Testing Anti-Malware Support

(I'm talking about real support, not tech support scam helplines...)

Many years ago at an EICAR conference in Munich, I sat in the bar one evening while some people in the security industry amused themselves by ringing each other's helplines. I'd be tempted to embarrass a few people by going into details, but the fact is it was so long ago (late 90s) that I can't remember anything particularly lurid (and some of those guys aren't even in the industry any more).

I did think at the time, though that it could make an interesting formal test, given a suitably well-designed methodology. But it's rarely been done, and I don't remember seeing it done particularly well. 

AV-Comparatives, a well-known testing organization, has just published two reports on the support offered by a number of vendors to users in the UK and in Germany. (Nine vendors, but not quite the same list in each case: not all the vendors have localized support in both countries.) Both reports are in English, and were apparently commissioned by PCgo and PC Magazin Germany. 

The tests looked at issues such as the ease of finding the vendor's support phone number, how long it took to get a response, how polite and professional the support staff were, and how well they dealt with fairly basic queries - three separate phone calls asking for help with activation and configuration. So the part of the test that deals with the actual phone calls seems very much a snapshot test that might get very different results at a different time, even if the testers got the same support person each time.  I'd rather see this part of the test repeated several times (perhaps with similar rather than the same questions) over time in order to get a better and more consistent picture of each helpline.

Still, the reports are not without interest, and I'd like to see something more substantial done along the same lines. In fact, it would be interesting to see AMTSO (the Anti-Malware Testing Standards Organization) generate some methodological guidelines on this kind of testing. I'm sure that after seeing these reports, there'll be plenty of security researchers with an opinion on how best to go about it. 

What’s Hot on Infosecurity Magazine?