The Anti Malware Testing Standards Organization (AMTSO) is “an international non-profit association that focuses on addressing the global need for improvement in the objectivity, quality and relevance of anti-malware testing methodologies.” As such it serves a valuable purpose. Testing AV products is a difficult process with different testers using different criteria on different malware samples; and the same AV product can appear to perform very differently under separate tests. AMTSO was established to provide a level playing field for the vendors by establishing credible and accepted testing standards.
The organization has been very quiet over recent months, with hints and rumors of big changes within. These may be needed. For example, a long-standing criticism is that AMTSO is effectively a closed club of and for the AV industry. While it says itself that “membership is open to academics, reviewers, publications, testers and vendors, subject to guidelines determined by AMTSO,” the actual members page on its website lists just one educational establishment, three test laboratories, two magazine testers, and 20 vendors. Furthermore, an annual membership fee set at €1250 is not conducive to the inclusion of individual and independent experts.
One further point to indicate that change is both necessary and perhaps already beginning is the exhortation, “Follow the AMTSO blog...” The AMTSO blog, however, merely says “…this blog no longer has any association with AMTSO, and is no longer maintained.”
Perhaps that change has started. AMTSO has announced a new service on its website: “a feature check solution for users of endpoint anti-malware security products.” Its purpose is to allow users to check the configuration of their installed AV product. For example, with just a few clicks Infosecurity confirmed that one free AV implementation is configured to check against malicious downloads, but is either unable or not configured to prevent phishing attacks.
Dr. Richard Ford, president and CEO of AMTSO, said “Whether you are a consumer or business, understanding if your endpoint anti-malware security product is configured correctly, detecting then blocking different types of attacks, is an essential element to giving the user or administrator the knowledge and comfort that their security solution is working on multiple levels to protect their valuable digital assets.”
At first glance this appears to be a huge shift of emphasis from defining test standards to helping the vendors’ customers. Luis Corrons, technical director at PandaLabs, pointed out that it is not a complete change. “This ‘Feature Settings Check’ is a small step within a bigger plan,” he told Infosecurity. “Even though this is something that end users will benefit from, it can be used by testers just to verify that the products they are about to test are running and configured as expected. For example, from a tester point of view it is not an easy task to verify if a product is able to connect to the cloud. Now they have a simple tool that will help them in that task.”
If Corrons is right, then we can expect to see further developments from AMTSO in the future – perhaps with a greater emphasis on allowing users to test their own products.