As enterprises battle to keep pace with online traffic growth by increasing network speeds, they are inadvertently playing into the hands of cyber-criminals. Why? Because most security tools are not equipped to handle the increases in traffic speed as well as attack frequency. They are limited in how much traffic they can intelligently process and need to take on increasingly sophisticated functions to combat evermore advanced and voluminous cyber-attacks – essentially, there is too much data and too little compute. As a consequence, many existing security applications will be wholly ineffective in the very near future.
This is a damning assessment of the IT security industry and its future to be sure, but it can still be saved by a small yet powerful force…enter metadata.
Metadata is data about data. Using the telephone network as an analogy, metadata more closely resembles a phone bill than a recording. It doesn’t allow for continual analysis but can still provide rich detail, as well as summarizing hours of conversation (or data) in just a few lines. A phone bill can show who’s been talking to whom, at what time, for how long, from where, to where, everything but the content of the phone call itself. Looking at a phone bill, it’s easy to check for interesting patterns. For instance, frequent calls to the same number, calls at odd hours or to and from unusual locations, calls that are very long. Each of these can serve as clues that help to narrow investigations to only the most relevant conversations.
The same principle can be applied to security. Providing security tools with summary takes of the packet data traversing networks, metadata can become a powerful weapon for enterprises looking to separate signals from noise, reduce time to threat detection, and improve overall security efficacy.
Dealing with the Data Deluge
Common sense tells us that listening to every phone conversation in the world in totality to find the bad guys approach simply doesn’t work. Similarly, with network speeds scaling up from 10Gbps to 40Gbps and even 100Gbps – and according to the Verizon DBIR hackers only needing minutes to breach a network – looking at every network data packet continuously to find hackers is becoming untenable.
The mean time between packets for a 100Gbps link is 6.7 nanoseconds. For security devices, this means that all security processing must happen in this nanosecond time envelope, and must start anew when the next packet arrives. The steps associated with security processing include extracting the packet, loading it into memory, determining the type of application, performing the relevant checks for protocol conformity and anomalous pattern matching, performing signature lookups against known patterns, extracting attachments, computing hashes, etc. These are all computationally intensive unto themselves, and this is not even an exhaustive list of all of the security inspection, enforcement, and recording tasks that need to be performed.
In the future, it simply won’t be possible for security appliances to conduct all of the security functions required to find bad actors moving in high-bandwidth links. However, the job of the security appliance becomes more manageable if it can focus on analyzing the relevant data and this is where metadata comes in. This small but mighty as yet unleveraged security super power can help accelerate time to detection and expedite response to breaches by enabling SIEMs, forensic solutions, and other big data security analytics technologies to approximate where in the network data breaches may have occurred.
Analysis of metadata like NetFlow/IPFIX records, URL/URI information, SIP request information, HTTP response codes, and DNS queries can be conducted much faster than analyzing the full packet data. SIEMs make a first pass at analysis of metadata for instance, and then provide network proximity intelligence that lets firewalls, intrusion prevention systems and/or content security gateways do a focused deeper dive to verify the type of threat, if any.
Summary information or metadata can provide valuable clues to lingering threats inside networks. Behavioral and security analytics using metadata gives organizations an approximation of the location of hot spots or areas of suspected threat activity – much like a divining rod points its user in the direction of water. Rather than searching the entire network, security analysts can focus on the identified trouble spots and conduct a more thorough investigation by using traffic or packet analysis. Consider the example of a DNS request made by a laptop to a suspicious server. Metadata flags the anomalous request. Then using tools capable of deep inspection security pros can examine all connectivity to and from that device to ascertain if the endpoint is indeed infected, whether it has forwarded malicious data and what other devices might be implicated.
Security and behavioral analytics’ approaches that use metadata as a first step in the research can create approximations of where threats reside making the in-depth investigation more focused and the time to threat discovery shorter.
Use of metadata gives big data analytics tools the network context they require to build models of what is good and anomalous behavior inside networks. This kind of base-lining can be very difficult to accomplish because it usually entails retrieving information from many geographically disparate and hard to access sources. By contrast metadata is readily available and can be harvested from the network flows themselves without disruption or organizational coordination.
So in the not too distant future, ensuring that security models and analytic processes are informed by not only lots of data, but the right data, will be key to making threat detection pragmatic and effective in high-bandwidth networks. Metadata is key to that future of effective security and breach detection because it can expedite behavior base-lining as well as anomaly detection making effective big data security analytics a reality for the masses.