Rooting out rootkits: Security software won't help, says Symantec

While rootkits make up a small percentage of malware, they are insidious because of their ability to remain on a machine undetected, explained John Harrison, group manager with Symantec Security Response.

A rootkit acquires and maintains privileged access to the operating system (OS) while hiding its presence by subverting normal OS behavior. A rootkit has three goals: to run without restriction on a target computer; to elude being detected by the computer or an installed security product; and to deliver its payload, such as stealing passwords or network bandwidth, or installing other malicious software.

“A lot people hear about rootkits and think of them as big, dark, scary things….We wanted to tell everyone about what a rootkit is, how it tricks the operating system by hiding below the system, so that when a security product asks if there is malware on the system, the rootkit tells the operating system to say, ‘No, there is nothing here’”, Harrison, who is a co-author of the white paper, told Infosecurity.

The rootkit is able to avoid detection by security software by “hooking” the OS’s application programming interface (API), by hiding in an unused space on the machine’s hard drive, or by infecting the master boot record (MBR).

“Some rootkits re-route OS APIs by changing the address of these APIs to point to their own code. This can be done both in user mode (where most applications run) and kernel mode (where device drivers run) and is often referred to as ‘hooking’”, the white paper explained.

“When an application calls a hooked API, the system looks up the address of the API in a table (such as the System Service Dispatch Table in kernel mode or the Im¬port Address Table in user mode). The operating system then executes the code at that address. If a rootkit has hooked the API, it has changed the address in the table to point to its own code so that its code runs, rather than the expected system functionality. This allows the rootkit to intercept requests that might reveal its presence”, according to the white paper.

In addition, the rootkit can hide in the hard drive’s unused space. “This unused space is invisible to normal OS APIs that are used to look for files on the hard disk. The rootkit will then modify a commonly used driver (such as atapi.sys) so that when that driver loads it will look in this unused space to find the rest of the rootkit’s code”, the white paper noted.

The rootkit may also infect the MBR in order to get its code into memory. “The MBR is used to bootstrap a system, helping make the transition from the hardware portion of a computer’s startup routine to loading the operating system itself. If a rootkit can control that process, then it can control what code gets loaded into memory before the OS even has a chance to protect itself”, according to the paper.

Harrison explained the Tidserv family of malware has been among the most popular rootkits used to infect systems. Tidserve arrives on a machine through a drive-by download, peer-to-peer file sharing software, bundled with other malware, or through a social engineering attack, such as through email or text messaging, he added.

“Just like with any other malware, one of the key purposes of the bad guys is to make money. They make money if they can keep something on your system….What better way to do that than with the stealthy aspects of a rookit hiding under the operating system”, Harrison said.

What’s Hot on Infosecurity Magazine?