The Test of Time

Written by


So you’ve seen all the stories about the rising tide of Mac malware. Maybe you’ve noticed that the fanboi cries of “Macs are secure! There are no Mac viruses!” have been a little muted lately, and that OSX/Flashback managed to recruit a sizeable number of Macs into a botnet. Maybe you’re thinking there might be an argument for installing some security software on the Apple of Your Eye. But which one?
Well, you can look out some comparative tests – preferably not one of those magazine tests that give you no clue as to the methodology behind the test, or as to whether the journalist gave any thought to all those technical issues that I used to obsess about as a former tester and director of the Anti-Malware Testing Standards Organization (AMTSO) – but the sad fact is that Mac-specific tests by competent testing specialists are rarer than hen’s teeth.
But apparently some hens do have teeth. At any rate, AV-Comparatives, an Innsbruck-based testing organization with more experience and know-how than most, have just come up with a report comparing the performance of seven products ranging from free and for-fee AV scanners, to security suites, to a product it describes as “more ... a system-optimisation tool than a pure security suite.”
Make no mistake: while the range of functionality available in each product is described, this isn’t whole product testing (but then it isn’t claimed to be). The products have been tested against 477 samples “most of them belonging to the FlashBack and FakeAlert families” and collected over the last two years. (I’d describe FakeAlert as a generic label used by some specific vendors rather than as a family, but that probably won’t bother too many people outside the industry.)
The test was restricted to on-demand scanning, which isn’t best practice for modern testing. Some products might be disadvantaged because passive scanning may miss malware that would be detected if the malware was allowed to execute during an on-access scan. However, this is probably less of a problem than it would be in a Windows test, since Mac malware is – at present – less likely to be self-protected by the kind of anti-forensic obfuscation that much Windows malware uses. In any case, many modern scanners use emulative techniques for on-demand scanning that essentially allow a program to execute in a virtualized (emulated or sandboxed) environment, which levels the playing field somewhat. 
It’s also noticeable that a number of significant players in the Mac AV industry haven’t been included: in particular, Intego, an established company working exclusively on Mac products, Sophos, whose free-for-home-use Mac scanner has a very large user-base, and Trend Micro, which claims to have 97% of the Mac AV market.
While this is essentially a traditional static test, it’s significant because it’s one of the early indicators of an interest in mainstream testers in Mac testing. Not that there hasn’t been interest previously: I remember discussing the economic viability of Mac AV product certification with ICSA Labs at least ten years ago, though at present I can only find one product that has been certified by ICSA Labs on OS X. My guess is that in a year’s time, vendors with a Mac product will be much better represented in both comparative and certification testing.


What’s hot on Infosecurity Magazine?