Independent test lab Broadband-Testing embarked on a performance validation of IT network firewall solutions from Cisco Systems, NetPilot, SonicWall and WatchGuard, benchmarking the performance of the vendors’ devices against key criteria such as network and application traffic performance with and without attacks. Differentiation among the four tested products appeared in security features, the variety of licensing options, differences in management GUIs and competitive pricing.
“Enterprises have much to lose by not taking the initiative to create and enforce a strict security regime. Instead they depend only on their security device vendors’ claims to assess the fortification of their network,” said Steve Broadhead, founder and director at Broadband-Testing. “The goal of this test was to look at different firewall solutions that sit at various price points and evaluate them on pure performance.”
In basic firewall mode, the products in general were able to get close to their claimed performance figures and were perfectly reasonable for the target market, the study found. However, with IDS/IPS functionality enabled, performance of the firewalls fell away markedly in all cases.
“We first looked at performance (throughput) levels and what can be sustained, or otherwise, as more and more firewall features are enabled,” explained Broadhead. “With IDS/IPS functionality enabled, performance [was sometimes] a fraction of what it was in basic firewall mode. While this may not be as significant a problem for a product aimed at supporting say 50–250 people as it is for a larger enterprise product, it is still of some concern, especially when combined with threat attacks.”
And to that point, Broadband-Testing tested the security appliances in both low- and high-traffic conditions to see how well they were able to defend against these attacks both in unstressed and stressed modes. The study found that differences in the products’ capabilities impacted their ability to prevent threats, depending on traffic conditions.
“Again we found differences in the product's capabilities to prevent threats depending on traffic conditions, though this was not totally consistent, as is often the case when a device is fully utilized (CPU, memory, etc),” Broadhead noted. “All the products tested let significant numbers of attacks through, despite being completely up to date in terms of signature databases, firmware and all aspects of configuration.”
Overall, the study found that mainstream, low-cost security appliances are not delivering quite what they say they will. “The reality is that the devices under test are designed for relatively low levels of use, with according price tags, so there is a level of compromise involved in terms of overall performance, especially when you consider the depth of features now expected to be part of an entry-level, enterprise-class firewall,” Broadhead said.
The takeaway? Security personnel need to be aware of exactly what their chosen appliances can or cannot do given the enterprise’s overall security profile. “Enterprise security validation needs to go beyond just looking for known risks or performance in isolation, it also needs to track the impact of security policies on performance and vice versa,” said Aswath Mohan, director, segment marketing for security, data center and cloud computing at testing company Spirent, whose products were used by Broadband-Testing to emulate a realistic mix of traffic that a branch office may commonly encounter.
“It should also test for abnormal traffic patterns that pose greater, yet currently unknown, risk to enterprise mobility", Mohand added. "Spirent has a long history of enabling enterprises and IT organization of all size better prepare the need work for new upgrades, and services deployments.”