!exploitable analyses data created by applications that have been crashed by fuzzer programs. Fuzzers bombard applications with unexpected data to try and cause them to stop working, putting them in a mode where they can potentially be manipulated.
Fuzzers have evolved over time. Randy Wiggington, the developer of Apple's MacWrite word processor, used to run a program called 'monkey' overnight against his program when developing it in the early eighties. It would automatically type gibberish into the word processor, trying to generate conditions that would confuse the system. Since then, fuzzers have become much more sophisticated, and can generate large numbers of crashes.
"When you're looking at a large number of crashes, it just doesn't scale. Not every developer can or should be a security expert," said Jason Shirk, program manager on the Secure Windows Initiative team in Microsoft's TWC. "The tool works by looking at crash dumps and analyzing them for security problems. We have to work out how bad they are."
A minor programming error may not be exploitable, whereas others may represent major security flaws, explained David Weinstein, the senior security development engineer who wrote the program. A divide-by-zero error may be inconsequential, for example, whereas a buffer overflow could allow the execution of arbitrary code. The relevance of these different types of flaw change over time, as blackhats become better at exploiting them.
The system is a plug-in for the popular Windows debugger WinDbg . "We look at what comes through WinDbg and analyze it, and tell you what we think the exploitability is, based on a number of things," Shirk said. These criteria include where the information came from, the type of crash (for example divide by zero or buffer overrun), and how much control the user has over the input data. The tool then categorizes the crash according to its exploitability.
"We also identify the uniqueness of the crash," added Shirk. Crashes can be caused by multiple factors, which could lead to large numbers of apparently different bugs that are really the same one, reached via different routes. "By assigning a value to it, we can see whether we've seen it somewhere else and cut down on the number of bugs that we have to look at."
In an internal contest, the company found 57 crash reports by using different fuzzers, and whittled them down to nine unique bugs using the tool. "Of those nine, only one came back from !exploitable as anything but 'probably not exploitable'," said Shirk, referring to the rankings used by the system. This enabled the company to hone in on a single bug.
In addition to using it internally, the company is recommending the tool to third party Windows developers and testers. Microsoft thought hard about releasing it, said Weinstein and Shirk. There is a traditional debate about how useful tools can be for blackhats wanting to compromise a system. They decided that anyone sophisticated enough to use the tool would already have the skills to find security flaws in a system. The tool would be much more useful for a legitimate developer trying to protect a system than for a blackhat trying to write malware to exploit it, they said.