Internet Explorer 8 blocked about four out of every five sites that attempt to trick visitors into downloading malicious software in browser security tests performed by NSS Labs, according to a report released yesterday.
In the Microsoft-sponsored tests, Firefox 3 came in at a distant second with 27 percent. Safari 4 scored 21 percent, Chrome 2 blocked 7 percent, and the Opera 10 beta was barely there with a 1 percent block rate. The tests did not include sites that use hidden exploits and drive-by-download attacks to attempt to install malware without your ever having a chance to recognize an attack.
According to the report, NSS Labs tested against a list of 2,171 608 socially engineered malware URLs, which it defines as “a web page link that directly leads to a ‘download’ that delivers a malicious payload whose content type would lead to execution,” over the course of 12 days in July. The tests focused on sites that try to trick you into doing the dirty work of installing the malware, such as sites that disguise malware as a video codec or player.
Blocking these sites is a good thing for any browser, but so is blocking exploit sites. Hidden attack code on exploit sites will search for software flaws in an ActiveX control or browser plugin, for example. If such a flaw exists, the attack code can install malware without having to trick you into downloading anything.
NSS Labs also tested against phishing sites, with much closer results. IE 8 blocked 83 percent of the information-stealing sites, and Firefox 3 blocked 80 percent. Opera 10 beta stopped the pages 54 percent of the time, Chrome 2 blocked 26 percent, and Safari 4 intervened for only 2 percent.
While these results may be fully legit and highlight a real advantage for IE, eyebrows go up whenever a company being tested is also footing the bill. NSS Labs could quell such skepticism by saying where it got its list of malicious URLs, and why it left out exploit sites. The company’s report doesn’t include this info, and NSS Labs hasn’t yet returned calls.
Update:
Rick Moy, President of NSS Labs, provided details about the company’s test methodology, URL sources and why it left out exploit testing.
Per Moy, the company’s methodology was in place before Microsoft contacted NSS Labs about performing the test. Microsoft asked plenty of questions about the methodology, but NSS Labs didn’t change the methods used for Microsoft’s test. Microsoft paid for a private report, and presumably could have chosen to not release the results had they not been complimentary, but Moy says Microsoft didn’t push to change the methodology or source URLs to favor its browser.
Those methods involve using an array of crawlers and spam traps to compile a list of potentially malicious list of URLs. NSS Labs also pulls in data from other sources, such as Sunbelt Software, Telus security labs, and Mailshell, Moy says, but the bulk of URLs is gathered by the company’s own crawlers.
Those URLs aren’t gathered until the test begins, so that the products are tested against current malicious sites. NSS Labs starts with a large list of suspicious sites (12,000 for this test), and then verifies which of those sites foist actual malware by testing downloads using sandboxes and other methods. The company first narrowed the 12,000 sites down to 2,171, and then further filtered it down to 608 verified sites that contain malware (I incorrectly only listed the 2,171 number above).
And finally, Moy says that testing exploit sites essentially breaks the test apparatus used by the company. If any one of its test machines became infected by malware as a result of visiting an exploit site, it would have to be fully rebuit before it could continue testing. NSS Labs’ method requires pointing all the browsers at a given site at the same time to see which browsers block a site, and these forced halts in the process would drastically slow things down.
Moy agrees that having test results for exploit sites would be helpful in gauging a browser’s overall security effectiveness, and noted that sites with socially engineered malware and even phishing will sometimes include exploits as well. But leaving them out of this analysis was a matter of understandable test bed limitations.