From ABTA, to election hacking to WannaCry, it seems not a day goes by without a cyber-attack dominating the headlines. Cybercrime doesn’t discriminate; it affects organizations of all shapes and sizes. Added to this is the mounting pressure caused by the EU General Data Protection Regulation (GDPR) which will penalize organizations that do not comply with laws that aim to keep customer data safe. It’s imperative for organizations to re-evaluate their security posture and plan for the future.
Cyber-threats have always been a difficult problem for humans to handle, even when they are equipped with the right data and tools. However, the threats that plague organizations today have grown too big, too fast and too complex for existing solutions or methodology to control. Cybersecurity has become a machine-scale problem, and the threats of the future will require a machine-scale solution.
Security through big data analytics and machine learning
Organizations can no longer rely on static rules to alert to more advanced activity. Behavioral-based analytics, machine learning and more flexible analytic solutions are required to defend the enterprise. Engines that operate via old rules typically have limits on the types and amounts of data that can be evaluated meaningfully.
On the other hand, today’s big data-based systems and machine learning analytics can use many years’ worth of total enterprise visibility data sets to model normal behaviors, highlight latent threats and even imagine new risk scenarios. An organization must be constantly vigilant about what impact they may face, and that means having visibility of all the information at hand.
With complete enterprise visibility comes immense amounts of complex data to analyze. Traditional tools are simply not designed to handle the vastness of the information available nor the complexity of the analytics required. Machine learning has emerged as a viable tool in the security analyst arsenal, to process billions of records quickly and to indicate anomalous activities in a meaningful timeframe.
The majority of organizations still spend most of their time scrubbing their data to make it ready for machine learning analysis, but then don’t possess enough domain expertise to understand what the data is telling them. In response, the open source community has created cybersecurity projects like Apache Spot that are revolutionizing the way we look at cybersecurity.
The idea is to create a common open data model that any application developer can take advantage of, to bring new capabilities to bear on cybersecurity problems. It enables big data analytics and machine learning for advanced threat detection.
In today’s cybersecurity landscape, there is a flood of data that organizations need to constantly monitor. Cyber-criminals have discovered ways to learn from one another, while enterprises have been operating independently to fight threats.
By defining Open Data Models for data storage and access, this project creates a single data platform where all data in the enterprise can be processed in its entirety in a cost-effective manner. The same platform becomes the hub for all future analytics as opposed to yet another point analytic product with yet another data store.
This creates a standard for the data making it much easier to begin analytics and it allows for subject matter experts to help train the data set along the way so the security team and the data science team are working together on the same platform.
We’re all in this together
With community developed and shared machine learning modules, Apache Spot is changing threat hunting from shared indicators to shared algorithms. It has raised the bar higher in terms of capabilities, while at the same time making machine learning accessible to every organization instead of only the most mature. It provides organizations with the tools and knowhow to protect themselves in the midst of an evolving threat-scape.
For example, ransomware continues to be a global scourge. The WannaCry attack bought the NHS to its knees and infected over 230,000 computers in over 150 countries. This devilish piece of malware is a very real threat, and it will only continue to get worse.
Ransomware will evolve to make use of newer and harder to combat exploitation kits. Over time, ransomware may lead to new classes of threats where instead of simply being passive, monitored or encrypted, data is instead manipulated with malicious intent. Organizations will level the playing field by learning from one another and applying that shared intelligence not just in terms of IP, domains and hashes, but by sharing machine learning algorithms and enterprise visibility use cases to help protect themselves and one another.
It’s hard to over-emphasize the importance of cybersecurity to an organization and its data. The fact is, without a modern strategy, it’s simply not possible to keep up with the number, and frequency, of attack.
Organizations that have created a unified big data analytics platform are on the front foot. They are using the platform to ensure their security analysts have complete enterprise visibility and the ability to be creative and accurate in trialing new techniques to detect malicious behaviors.
For too long, organizations have been losing the battle against threats that are leading to financial and reputational damage. Fortunately, we can now turn the tide with technologies such as artificial intelligence and machine learning built into a robust platform that is already proving to reduce risk in measurable ways.