Teradata and the Ponemon Institute have today published new research into the evolution of big data analytics as a security tool within industry. The overall conclusion is that its effectiveness is widely recognised; it is under widespread evaluation for implementation; but it is not yet extensively used.
The research shows that while cyber attacks are getting more serious, only 20% of organizations are more effective at stopping them. The two biggest threats are considered to be malware (scoring 9.6 out of 10) and malicious insiders (8.88). In both cases early recognition of related anomalous behavior on the network would lead to faster and more effective mitigation.
The reasons this is not yet happening are many and varied, but the most significant barriers are considered to be a lack of effective technology solutions (43%), insufficient visibility into the network (42%), and a lack of skilled or expert personnel (38%). Technically speaking, data analytics could be achieved by manual analysis of system logs – but this research suggests that the technology itself is insufficient because it doesn’t provide an adequate view the network, and that the organizations don’t have enough manpower or skill to analyze the data provided.
This might suggest a clear market opportunity for new specialist and automated big data analytics products.
From the research, it is tempting to see a relationship between the current use and effectiveness of analytics technologies with the availability of budget and skilled security staff. The research grouped respondents into three sectors: financial services, manufacturing and government. Within these it is generally accepted that financial services have the largest budgets and pay the better salaries; with government having the lowest budgets and salaries. Both observations are potentially born out by the Teradata/Ponemon research. While 67% of finance organizations are already using, or intend to implement big data technologies within six months, only 41% of government organizations say the same.
Similarly, 74% of finance organizations find the reduction of false positives in the analysis of anomalous traffic to be difficult or very difficult; but 87% of government organizations have a similar problem. This suggests that the finance industry has better product and more skilled staff – and resonates with an unrelated comment from John Colley, managing director EMEA of the (ISC)2 consortium of security professionals. “The organizations with the most money, such as the finance sector,” he said, “are solving their security problems by creaming of the best staff with higher salaries.” The corollary, of course, is that other organizations do not have access to the skill set necessary for detailed big data anomaly analysis.