IT teams can mine the vast volumes of data captured by security software and even line of business applications, to spot areas where the business is at risk.
The growth of "big data" within companies, and the development of better tools to analyse it, is providing a new way for information security to monitor what is happening on their networks, or to scan customers' accounts for unusual activity. Used carefully, the tools can help spot, and close down, vulnerabilities at an early stage.
At a keynote panel at Infosecurity Europe 2013, CISOs from a range of businesses set out how data was helping them improve threat detection, and reduce data loss. Data mining can be especially effective in spotting unusual behaviour patterns that might pick up an intrusion, or an attempt at data theft.
"We use big data from various different sources to become more proactive and try to protect where threats will come from, and protect the organisation against those threats," said Craig Goodwin, director of information security at Monster Worldwide. "Information sharing can make the data even better."
This, he said, can support a move to "intelligence-led security". "Often it is not the direct threat that we detect," said Adrian Asher, chief information security officer at Skype. "What we detect is attackers learning our systems. That is highly valuable."
Using big data techniques for security does, though, present challenges. One is the potential cost of storing and analysing the data. Another is the need to comply with privacy and employment law.
"I want to be able to use the masses of data on the Philips network, but privacy concerns come into play," said Carl Erickson, CISO and director of threat management for the company. "You are not allowed to do profiling, so you may have to obscure identities. But I want to see as much as possible about what the attacker is trying to do."
Councils in particular have to balance the need to keep some information confidential, and to make others available to the public, said Simon Salmon, head of IT security at Nottingham City Council. But storing and gathering data from a range of security incidents can bring better insights than looking at attacks in isolation, he explained.
"If you take each piece of information, or each attack, in isolation, you are not making the most of each piece of data you have. We also try to share [security] data regionally and nationally, and also with the private sector." All organisations could improve their cybersecurity, Salmon suggested, if they shared more incident data.
But infosecurity teams are also drawing on data analysis work done by other departments, in particular marketing, and fraud prevention, which have been using data for longer. Security teams should also be tapping into the business intelligence and analytics tools these departments use, the panel agreed: it was rarely practical, or necessary, to invest in security-specific big data tools.
"There are no specific security tools, it is just data," said Asher. "We use tools for data mining that we have developed internally or are off the shelf. Used appropriately they give us that intelligence."
"I wasted a lot of time looking at IT security vendors and security specific tools: we have a lot of internal tools I could leverage," said Goodwin. "The fraud and compliance functions are a lot further along in using big data."
"You don't need anything security-specific," agreed Salmon. "But you need your information repositories mapped, and you need to know what you are reporting on. Security tools can do the heuristic piece, but we also use standard dashboards."
"We also use more off the shelf tools, but we are also capable of asking vendors for more functionality," said Erickson.
But, warned Andy Kellett, principal analyst for security at Ovum, some data analysis tools lack their own security safeguards. "You do have to check whether these new tools are up to the task of delivering these services," he cautioned.