The tests and results are published in NSS Labs’ latest comparative analysis of browsers, and the results show a vast, but complex, difference in browser performance. Most effective is Internet Explorer (blocking 99.96% of the samples). Then comes Chrome (83.16%), and then a huge gap to Safari (10.15%), Firefox (9.92%), and Opera (1.87%).
The primary reason for the difference in effectiveness seems to be the efficiency of the different reputation-based blocking mechanism. “Microsoft’s ‘Application Reputation’ and Google’s ‘Download Protection’ are fundamentally both content agnostic malware protection (CAMP) schemes,” noted the report, “however the extent to which this technology is relied on is an important differentiator, as the technology is flawed.”
The flaw is that since the block is based on reputation rather than known bad content, it is susceptible to false positives. To avoid blocking what might be a legitimate ‘clean’ URL, the user is given the option of accepting or overriding the block. There remains, noted the report, “the danger that social engineering attacks can deceive users into bypassing the file blocking and installing malicious software.” The frequency with which users are allowed to override a block thus becomes an important issue.
Randy Abrams, research director for NSS Labs, put the effect of this into perspective. “Both Google’s Download Protection and Microsoft’s App Rep allow users to override browser protecting, however, Google relies on this less reliable protection mechanism nearly four times as often as does Microsoft. The net result is that IE 10 users are offered superior protection over Chrome users with one quarter the risk of making a bad download decision.”
Microsoft and Chrome use different reputation systems. Since the previous tests performed by NSS Labs, Chrome has incorporated its Download Protection mechanism. Without this new mechanism Chrome would perform almost identically to Firefox and Safari – blocking about 10% of the malware samples. However, with the reputation system, it leaps to 83.16%. But this, noted NSS, is almost identical to Internet Explorer without its own reputation blocking – which adds a further 16% success rate for IE.
The net result of the tests is that Internet Explorer is the clear leader in blocking malware downloads from the web. Chrome comes second; but both are far ahead of Firefox, Safari and Opera.
We should, however, include one note of caution. “An initial sample set of 11,296 unique and suspicious URLs entered the system; 754 URLs were found active and malicious, and met the criteria for entry into the test.” Furthermore, the test comprises US data only, and is therefore limited in both quantity and geography.
A spokesperson from Sophos, a company that is involved with the anti-malware testing standards organization (AMTSO), told Infosecurity, “Sample selection is extremely important. It is fair to consider whether there are enough samples being included – but actually it is not just about the number. All of them might be a single family, rendering the effective sample set size as 1. The distribution of the samples also needs to be considered: are the right malware families represented; is the sample set skewed towards one (or a few) malware families; do the samples adequately test the varying types of protection being tested?”
This, the spokesperson added, is a structural problem with all ‘real world’ type tests. They are resource intensive and often lead to the use of relatively small sample sets.