AMTSO, founded in 2008 to improve anti-virus testing methodologies, is currently engaged in a “re-engineering” to improve its credibility and acceptance in the security community.
Harley, a former board member, is recommending that AMTSO expand the range of stakeholders in the organization to overcome the impression that it is a vendor cartel.
“The credibility of the organization is influenced by the fact that it tends to be seen as a group of anti-virus vendors and testers”, which have a direct financial stake in how testing is carried out, Harley told Infosecurity. “It needs input from a wider range of stakeholders”, he advised.
AMTSO is holding workshops on how to accomplish its re-engineering. In a blog, Harley related how at a recent AMTSO workshop in Munich the membership discussed reforming its internal structure “in a way that looks more like a commercial enterprise (albeit run on a budget that at present would barely pay for a round of drinks at Google's Christmas party).” Part of the reform would include the appointment of a paid administrator; right now, the executive team is made up mostly of volunteers.
“In addition, the organization is considering returning to one of its early goals of monitoring and documenting ongoing testing, though probably in a less contentious form than its earlier review analysis process”, he wrote.
In a white paper, written before AMTSO began its re-engineering effort, Harley argued that “mistrust” of the anti-virus industry has created a barrier to the organization's attempt to raise the quality of testing.
To overcome this mistrust, Harley recommended that an independent group review AMTSO’s testing guidelines to restore their credibility.
“What if an organization (or a coalition of organizations) with more credibility (or at least a less compromised public image) were to take on the task of policing standards enforced through certification of product certification bodies, testing organizations, and perhaps even generalist reviewers?” he asked.
Harley recommended a number of organization types that could take on this policing role: academia, international standards bodies, the research community, even the testing industry itself.
In his interview with Infosecurity, Harley said he favors a group like IEEE because of the organization’s international credibility. IEEE has a “track record of bringing together various stakeholders, including the anti-malware industry, for various standards projects”, he explained.