April 2008 issue

White hats fight back

Danny Bradbury

The introduction of the profit motive has increased levels of expertise in the black hat community over the past five years, encouraging white hat researchers to rise to the challenge. Danny Bradbury looks into this research and find out how it works.

It's no secret that the black hat community has changed in the last five years. With the introduction of the profit motive, the level of expertise has risen substantially. On the whitehat side, the research community has stepped up to the challenge. But what does that research community look like, and how does it work? 

Research in the security field sits along a spectrum, with academics thrashing out lofty concepts at one end, and commercial researchers dealing with more immediate problems at the other. "Academia is good at more abstract tool development and research properties, looking a little further down the pipe than the short term," says Jon Ramsey, CTO of managed security service provider SecureWorks. "In the commercial world, we seem to be focussed a little more on the short term, and on what we can do to protect our clients in a more applied fashion." Ideally, the job of the commercial world is to take and package such abstract concepts and apply them in a commercial way.

Unfortunately, this transition is often plagued with problems says his colleague Joe Stewart, who works as a senior security researcher at the company. "When you try to take their ideas and apply them to the real world, a lot of assumptions break down," he warns, adding that he has experienced this after being approached by academics trying to form start-ups using technology that worked well in their labs.

"That's because the real world is very chaotic. There are anomalies, misconfigurations, crazy programming stuff that they will never see in their academic environment," Stewart continues. This can create the same kind of gap between those two communities as exists between scientists and engineers in other fields.

Pure research
One of the clearest examples of such problem lies in operating systems. In an ideal world, the work that people such as Microsoft security guru Steve Lipner did in the seventies on mathematically provable systems would have made its way into the commercial world. In the seventies, Lipner had an idea that formal specifications for provably secure systems would be developed. Models would have been created supporting those specifications, and then systems would have been written against those models. The result: a system mathematically verified as secure. 

If we could post a sticker on a Windows box saying 'unbreakable: mathematically proven', lots of consumers would be happy. But the world hasn't been kind to that idea, Lipner says. "Security isn't a closed form problem," he laments. "People come up with new ways of attacking, and as a result, we need to come up with new ways of defence. Theories of security come from theories of insecurity."

That doesn't stop academics trying to research more secure operating systems, however. Verifiable operating system projects still exist, and others are relying on new developments in hardware to rekindle secure systems development. "We're building a new operating system called Nexus," says Emin Gün Sirer, an associate professor at Cornell University, explaining that it uses the Trusted Computing Group's trusted platform module (TPM) to guarantee trust in an operating system. 

Microsoft removed much of the support for TPMs that it had originally scheduled for Vista in the form of its Next Generation Secure Computing Base (NGSCB). Could operating systems like Sirer's - built to be secure from the ground up, rather than frantically retrofitted with security after the fact - make it in the commercial world? Does it matter?
"We're not developing commercial products," he points out. That's not his job.

The further that research moves away from commercial utility and the more abstract it becomes, the 'purer' it gets. Ramsey praises such research. "You get bigger gains out of pure research than you do with short-term applied research," he says. "You get new techniques; revolution instead of the evolution you often get with applied research."

Certainly, some academic research efforts seem more directly mappable to commercial products than others. For example, Diffie and Hellman ceded a whole subsection of the industry with their paper "New Directions in Cryptography," which laid the foundations for the modern PKI industry. Some of the patents, such as the one that they claimed along with Ralph Merkle on cryptographic apparatus and method, become extremely valuable. 
Other areas of academic research in security seemed to be less explicitly tied to commercial opportunities. Donna Dodson, deputy division chief of the Computer Security Division at the National Institute for Science and Technology (NIST), describes how the organisation is researching a basic lexicon to describe computer security. "[Security] started out at its fundamentals without some of the vocabulary," she explains. "The variables and the complexity in terms of information science has grown dramatically, and that has made some of the problems of assurance much more difficult today." To build security and assurance you need metrics - a language to describe it. 

Keeping up with the bad guys
The more applied research gets, the more susceptible it becomes to technology cycles and trends. While NIST explores fundamental security concepts that aren't time-critical, SecureWorks' Stewart spends his days picking apart new malware instances to find out what they are doing, and to engineer solutions that react directly to those innovations from the black hat community. 

Recently, he has been discovering techniques that rehash those found in the early days of virus writing. For example, recent malware has used the master boot record on a disc as an infection vector. Still others use parasitic techniques which were first used in the 1980s, but which went out of style for a while with the emergence of macro viruses and the like. He engineers these into his software tool, Truman. He calls it a "sand net" tool for analysing malware activities without the use of virtual machines (because more malware is beginning to recognise virtualisation, and avoid running in such environments).
Stewart's research takes place at the thin end of the wedge. He is trying to spot and neutralise new techniques as they occur to protect his customers. His and Dodson’s activities bring their own benefits to the security realm, but in between them lies a wealth of interesting applied research that responds to longer term trends in the industry without being entirely abstract in its approach.

One example is Herdict, a project orchestrated by the Berkman Centre for Internet and Society. Herdict would not have been possible before the development of an internet in which large groups of people were constantly connected. The system, a PC client, examines the software running on a machine and measures its effect on resources such as memory and processor resources. It polls users on their satisfaction with their computing experience, and can then compare results from other Herdict users in the broader community.

"The basic idea is to use many of the tools of spyware, but for good," says Jonathan Zittrain, who founded the Berkman Centre. "You download it to your machine. In doing so, it gathers the machine's vital signs. How many popups it gets, how happy the machine is. It sends that back to be aggregated with everyone else's data, and it sends you a dashboard that helps you to make sense of it all."
Researchers on his team hope that the software may one day be suitable to spot code matching patterns distinct to malware. So, if a piece of software spreads rapidly throughout the Herdict community, contains no reviews, and seems to drastically affect client performance, it could be flagged as a potential problem.

Two-way communication
Such research offers tactical benefits stretching beyond the ad hoc daily battle between black hat and white hat, but with enough practical focus to have meaningful utility in the short to medium term. How can researchers battling daily with the criminal online community and academics working with abstract concepts, each already productive in their own right, work together effectively to produce more of these projects? It requires a two-way communication, says Ramsey.

"We communicate with academia, but it's marginally effective," he argues, adding that there isn't as much cross-pollination on the conference circuit as one would hope. "Go to RSA this year and look at how many universities are there."

One opportunity might lie in colocation deals between commercial organisations and academic researchers and companies, where the former becomes more intimately involved with the latter on-campus. That inspires mixed feelings in Sirer. "It grounds the research done at the university. It often channels money to the area if not the actual department involved, and it creates synergy between the company and the school," he says. "The downside is being too chummy with the company. If you're a graduate student and there's a big company on campus, it can colour how you look at the world."

There are other approaches. CERT is a federally funded R&D centre with the mission of technology transfer, for example, and university incubators can be useful. In that model, academic ideas can get funding and business experience. And of course, driving pure research within companies can be beneficial, although there is always the pressure to produce applied results, warns Ramsey. He recalls his time at Siemens, when he got to know the R&D department well. The company switched from a "corporate tax" model, in which divisions had to pay a proportion of income for R&D, to one in which the research group had to bid on cash for projects like everyone else. "The nature of the research projects changed dramatically," he recalls. "Not only did the nature change, but when projects were out of money, they just stopped."

In the world of security, then, as in many others, researchers must play a political game. All research is motivated by a particular goal, and the funding depends on how well the goals of the researchers match the goals of those with the money. While such political games play out, the black hats get their funding - lots of it - in entirely different and nefarious ways. No wonder they come up with so many new ideas.