On average, an IT executive has only seven minutes to determine whether their organization is under attack. This is according to a survey of more than 400 IT executives in the UK, France, Germany and Hungary in which respondents were asked about their ability to process and use valuable information from security alerts.
Today, most organizations have log files that record numerous security events from the operating system, applications or users’ actions. In fact, an astonishing 198 million logs on average are collected per day from IDS, DLP, SIEM and other user monitoring systems. For organizations trying to sift through and find positives among false alerts, it’s like finding the proverbial needle in a haystack.
Good data is needed for good analysis, so it is crucial to collect all relevant logs from all possible platforms in order to structure and classify them for further analytics.
Good News and Bad News
Organizations need to collect and store all log messages for a forensic investigation in the event of a breach. Processing and analyzing this magnitude of logs would require a huge task force and be quite costly, so organizations need to have processes in place to help them determine which log messages are relevant.
According to the aforementioned survey, organizations are only able to process a third (31%) of log messages, and so they need to decide where to focus their efforts. One option is fine tuning the alerting systems and continuously updating the rules and patterns. This means that there will be fewer alerts, but a higher proportion will be relevant.
On the other hand, if log analysis is too finely tuned to reduce false positives then it is more likely that the rate of false negatives would increase. It is important to find the right balance and not just blindly optimize for low false positive rate.
"The sheer volume of alerts received and the limited timeframe available to investigate indicates that manual efforts are not enough"
Either approach can work if the process is well balanced and the correct processes are in place to determine what security alerts are high priority. Rules- and pattern-based alerts are useful but can be costly to maintain and would be limited to detect the latest advanced threats.
So, What Can You Do in 7 Minutes?
The seven-minute decision window for just one security alert is drawn from our own survey results; an IT security team with an average 9.2 people collectively needs to examine an average of 520 security alerts a day. Theoretically one person will receive 56.5 security alerts a day – seven per hour. Factoring in a 10-minute break, one person has, on average, a maximum of seven minutes per alert to decide if it’s a sign of an attack and needs further investigation.
The sheer volume of alerts received and the limited timeframe available to investigate indicates that manual efforts are not enough. Automating the prioritization – where machines select unusual activities based on pre-defined rules and self-learning algorithms – allows teams to focus their efforts on tasks that require human intelligence, improving overall efficiency.
False Positives
Organizations have an average of 18% false positive alerts. Whilst no organization wants to spend time and resources chasing after false alarms, this is a relatively small proportion and suggests that organizations are finding the right balance in their security alerts.
On balance, it is better to ‘waste’ a small proportion of time and investigate a false positive rather than ignore suspicious activity. The key is to prioritize any unusual activity based on the potential risk it poses. This is where adding contextual information in addition to logs and analysis using big data algorithms provides the critical evidence needed to help prioritize activities and identify the true positives.
Organizations have finite resources and must make quick decisions on which security alerts to investigate. With many security teams now facing a tsunami of alerts, it is critical to have the right processes in place to allocate resources as effectively and efficiently as possible to minimize the risk of a data breach.
About the Author
Márton Illés is product evangelist at BalaBit, having started out as a security consultant, later becoming a product manager. Prior to that he worked as a CTO. He graduated with a MSc in computer science and economics at Corvinus University, Budapest. Márton is married and father to a son