Anyone working in the criminal justice system will be familiar with the term ‘CSI Effect’. Just as doctors are routinely challenged over diagnoses by patients who have googled their symptoms, the CSI Effect comes from the public’s new found ‘knowledge’ of forensic science.
Concerns exist that jurors are coming to court with unrealistic expectations of forensic science, to the extent that juries are thought less likely to convict when there is no forensic evidence produced. It’s also believed that the CSI Effect may be responsible for more unnecessary crime lab tests and calls for expert testimonies.
The problem is that forensic science is often portrayed as providing definite and irrefutable evidence of proof when the truth is that, outside of DNA analysis, forensic science should only be used as supplementary weight to support an allegation.
In reality, forensic science is used relatively sparingly, especially when eye-witness, circumstantial and alibi evidence is available. Its comparatively expensive, time-consuming and rarely the definitive evidence that TV suggests.
When it comes to cybersecurity investigations, instead of swabs, fingerprints and fibers, a key source of evidence are system logs. Everything from applications to devices is capable of generating an audit trail, ‘logging’ activities and events. At its simplest, if we have a record of logons to a system, and we know when our breach happened, we have a cyber ‘smoking gun’.
If we can use log data for a reconstruction post-attack, why can’t log events be used to pre-empt a breach, providing an early warning that suspicious activity is taking place? This is the promise of contemporary SIEM technology, an automated system to capture sufficient evidence to not just understand the timeline of a breach, but to detect the warning signs of an attack before it happens.
If audit logs provide all the answers, why are we still regularly hearing about cyber-attacks? One reason is that, just as DNA samples aren’t always available at a crime scene, audit logs may not be detailed enough, or may not even exist at all.
This is because both the activation and level of detail recorded in system logs is a configurable option, and for good reason. There is always a cost with logging; not just the processing resources to generate the logs, but for the storage resource needed.
For example, a developer trying to troubleshoot an application problem may want all activity logged whereas under normal circumstances logging would be reduced in favor of maximizing system performance. Why bother generating gigabytes of logs if not needed?
All of which is why, for security controls to be effective, the right audit policy is essential: A goldilocks policy that avoids auditing everything and generating unnecessarily large log files full of non-essential data, but one that logs precisely what is needed, and with the right level of detail.
The other reason why cyber-attacks are not being eliminated is because, like most forensic science techniques, pre-emptive SIEM analysis is far from reliable. Hair and fiber analysis relies on visual comparisons to provide, at best, a determination that samples are ‘similar’.
Likewise, just because we have an understanding of previous breaches and the indicators of suspicious behaviors that were significant then, this doesn’t necessarily mean this knowledge can be re-purposed as an advanced warning in the SIEM tool.
Not surprisingly, the degree to which behavior patterns can be used to reliably infer an attack is pretty limited and as a result, false positives are always going to feature. With alert fatigue one of the biggest enemies of effective security controls, it’s important to include additional dimensions and context to log data, which is why other controls like Change Control and File Integrity Monitoring are best achieved using tools other than SIEM.
In fact, in recent testimony to the Homeland Security Committee Hearing on Private Sector Data Breaches, John Gilligan, CEO for the CIS, stated: “deploying the top five CIS Critical Security Controls can reduce up to 90 percent of known pervasive and dangerous cyber-attacks”. For reference, the first six Basic CIS security controls, are:
- Inventory of hardware
- Inventory of software
- Vulnerability management
- Configuration hardening
- Control of admin rights
The sixth control covers log analysis.
In summary, while SIEM technology provides a great solution to the essential security control of audit log analysis, like most forensic science, it can never replace, nor exclude, other known and proven best practices.