How to Deal with Unknown Files Effectively

Written by

Michél Bechard explains how to combat the files that are gaining access to networks and causing mega breaches

Unknown files have become one of the biggest security threats to organizations, catching the attention of C-suite officers. Highly-publicized data breaches at brands like Target, Home Depot and Sony have raised awareness of the damage that unknown files can cause. What was once a simple computer virus has evolved into sophisticated malware that can be the basis for an advanced persistent threat (APT) attack designed to wrest sensitive data and wreak financial harm.

The simplest way to frame the issue is with the ‘three-file rule’, which argues that we can group files for security purposes into three types: the good, the bad and the unknown. Until recently, there have been two basic approaches for vetting files entering a system, each with limitations.

The first approach is legacy antivirus technology. This technology is able to identify and block a bad file but only if it’s in the malware database. All other files (unknown or good) are not recognized as bad and, by default, are allowed to pass through the filters and potentially infect a user’s system and compromise an organization’s network.

The second approach is whitelisting, which attempts to limit allowed applications and files that can be used inside the network to specific types or programs. Whitelisting, while effective, can be overly aggressive and introduce excessive administrative overhead – it’s difficult to keep up with new, legitimate applications being developed, upgraded and updated.

Neither approach addresses the unknown file issue.

It’s impossible to say how many unknown files are in a network with this approach because they only disclose their presence if you have a visible breach or data loss. Many infections go undetected, collecting and transmitting confidential data to their creators.

The challenge is thus how to manage unknown files without impeding day-to-day business and hindering the user’s experience. The default behavior of legacy security software is to allow unknown files to enter the organization. This ‘default allow’ approach to endpoint security needs to change immediately.

Sandboxing and Containerization

What’s needed is a change to the default method of handling unknown files. Instead of always allowing them into an endpoint (persistent default allow), or explicitly forbidding them (persistent default deny) an effective mechanism is to automatically sandbox the unknown at run-time.

Legacy sandboxing (i.e. a security mechanism for isolating a running program in a tightly controlled environment while the program is heuristically analyzed), however, has had limited success due to issues identifying the program’s activity as unknown and relegating the unknown files to the sandbox until a time when its true nature can be determined.

Next generation sandboxing, or ‘containerization’, which provides a jailed environment where programs can run isolated from the rest of the host environment, has proven to be a superior approach.

"Next generation sandboxing, or ‘containerization’... has proven to be a superior approach"

Several attempts have been made at containerization including creating ‘micro virtual machines,’ negatively impacting the end-user’s experience due to performance degradation through resource consumption. Other attempts at addressing this unknown file problem include browser-based solutions that monitor a user’s behavior with web applications. But protection is only provided for browsing, ignoring threats from downloads, email attachments, USB or other storage infections

The ideal approach would be to separate an unknown file and allow it to run safely without the ability to infect the host endpoint with the user’s environment accurately replicated in the container.

The Three-Stage Approach

The new idea for handling unknown files is using a three-stage approach. The first step uses the whitelist approach. If the program is located in the list of trusted applications then it requires no further scrutiny and is allowed to run on the endpoint.

Should the program not be found in the list of trusted applications it is then inspected for malicious content by cross-referencing the program against the antivirus signature database. If the program is identified on the AV database it is relegated to the quarantine container. This is the second step.

If the program isn’t identified on the whitelist or on the blacklist it is automatically contained

via an auto-sandbox approach. Containment, however, doesn’t have to use virtualization; it can rely on persistence, i.e. it creates a persistent layer on top of the user’s OS. Anything not on the blacklist and the whitelist can only run outside this layer and therefore cannot interact with the OS.

This three-stage approach minimizes any disruption to the user’s experience by providing the ability to safely run unknown files in a secure container without the usual performance degradation.

Michél Bechard is Director of Service Provider Technologies at Comodo.  You can reach him at michel.bechard@comodo.com

What’s hot on Infosecurity Magazine?