Security analysts must ensure that incidents are managed effectively with the least amount of impact on business operations or users. Without the right tools, security event and alert management can lead to analyst fatigue as they manage huge volumes of alerts and notifications generated by on-premise, cloud, and hybrid computing environments that consist of hundreds of services hosted on physical and virtual machines.
Additionally, security teams must also manage the incident response process from identification to escalation, communications, resolution, and follow up.
A survey published by the Cloud Security Alliance found that IT security professionals at times feel overwhelmed by the number of alerts and notifications. This then creates “alert fatigue” as the average enterprise generates nearly 2.7 billion actions in cloud services per month of which 2,542, on average, are anomalous. Of the 2,542 anomalous events, only 23.2 are actual threats, a ratio of nearly 110:1.
Machine learning and automation are going to play an increasingly critical role in how organizations handle their responses to incidents as security staff don't have the time to examine and research the details of every notification or alert.
Using Machine Learning for Duplicate Incident Extraction
One example of using automation and machine learning for security is duplicate incident extraction, as high alert numbers can lead to a high number of duplicates. Due to various attack vectors and different target endpoints, the same security incidents can register independently on different security platforms within an organization, leading to mundane and repetitive analyst work as they manually remove duplicate incidents.
With a machine learning enabled security platform, algorithms look for similar labels, email labels for phishing incidents, incident occurrence times, and common indicators to generate a list of duplicates. This type of threat identification and duplicate incident documentation eliminates menial analyst work, freeing them to focus on more critical problem-solving and high-quality tasks.
While machine learning algorithms can help proactively identify duplicate incidents and provide context, automation can help execute the repeatable steps involved in this identification at machine speed instead of the analysts having to wade into the mire of grunt-work.
Visualizing Critical Security Incidents using Machine Learning
Another machine learning use case to reduce analyst fatigue is using real-time data visualizations and maps of similar security events or incidents that have already occurred in the system. Using incident data and indicator details, machine learning algorithms can identify patterns and similarities and then visualize that data in actionable form.
In addition to reducing alert fatigue, real-time data visualizations increase analysts’ investigative capabilities by providing them with visual tools to better understand the larger security picture and how incidents are related.
With data visualizations that lack any type of machine learning capability, analysts may not draw patterns with similar incidents that have already occurred on the system, resulting in redundant rework for previously stored response processes.
Bringing Elements Together
An ideal application of both automation and machine learning would be bringing both the duplicate incident identification and map-based visualization together. Machine learning algorithms could spot similar and duplicate incidents – with automation executing the high-quantity tasks involved – before displaying an actionable visualization for security teams to base their decisions upon.
This combination also maintains a balance between automation and human-led decision-making. By displaying suspected similar and duplicate incidents but leaving the final decision with security teams (marking as duplicate, collapsing multiple incidents into one case etc.), the analyst still retains overall control and direction over incident resolution but is able to spend the majority of time productively rather than sifting through piles of alerts.
The Machine Learning Advantage
A critical advantage of machine learning when addressing analyst fatigue is its ability to enhance analyst capabilities with intelligent platforms that filter out duplicate incidents to reduce alerts, identify incident patterns and similarities, and help analysts understand the larger security picture.
Soon, using machine-learning platforms, security analysts will be able to regularly focus on more important decision-making in situations such as emerging threats, escalating incidents, troubleshooting, and case management.