Think bad bots on the Internet are just a nuisance and nothing more? Think again, writes Rami Essaid
Much of the non-human bot traffic online is malicious, and IT and security executives need to address the threats posed by bad bots or face possible data breaches and other risks to enterprise information resources.
Bots are programs coded to automate a certain task on the internet. While good bots make it easier for users to share information or conduct searches, bad ones are used for malicious reasons such as theft or damage.
In our 2015 Bad Bot Landscape Report, we found bots are the main culprits behind brute force attacks, competitive data mining, account hijacking, unauthorized vulnerability scans, and other attacks.
Clearly, there’s a growing volume of automated traffic online. While not all bots are bad, enough of them are that organizations need to be aware of the problem. Bad bots can come from virtually anywhere in the world. China in particular has become a major source of bad bots. The country is the leading source of bad bot mobile traffic in the world, with 31%. The three mobile carriers with the highest share of bad bot traffic are based in China.
However, the bad bot landscape continues to evolve rapidly, particularly with the dramatic increase in mobile bot traffic, and an ever wider range of geographic and ISP points of origin. With the advent of cheap or free cloud computing resources, anyone with basic computer skills can download open source software and get into the game. With the continued growth in mobile devices and business applications, expect the rise in mobile bot traffic to continue as well.
IT security teams need to make sure cyber-criminals and other nefarious actors can’t harvest organizations’ data or breach their defenses. Unfortunately, most companies still have no visibility or control over this malicious website traffic, as the emphasis for many IT security teams is mainly on protecting company IT assets and networks rather than website content. These sites may be outside the realm of the It security team if they are hosted by third parties, yet they still hold valuable company data in the form of intellectual property.
The Future Challenge
Like many other types of cybersecurity threats, bad bots are becoming more sophisticated, including the ability to mimic human behavior. Some of the simpler bots make little or no attempt to hide what they are; they can be identified by using bad user agents or by failing basic browser integrity checks.
More sophisticated bad bots, however, closely mimic human behavior such as web browsing. These types of bots can be nearly indistinguishable from actual human website visitors such as prospective customers.
In 2014, 41% of bad bots mimicked human behavior while 7% of bad bots disguised themselves as good bots. Webmasters allow the entry of good bots such as the Googlebot to their website infrastructure for SEO reasons. When a bad bot masked as the Googlebot enters a site, it can trigger a number of problems without even raising any red flags.
Companies can take steps to protect themselves against bad traffic and at the same time make the most of good bots. Firstly, have a good understanding of what traffic on the company’s website typically looks like. That way, when something is amiss it will be easily evident.
IT security teams should explore new techniques and tools, such as behavioral modeling and browser automation detection, for detecting bad bots. In addition, IT and security should work closely with other parts of the company to detect and protect against bad bots. For instance, the marketing department can notify IT security when there are signs of click fraud with a particular campaign.
From a security standpoint, companies need to be especially concerned about bad bot traffic from China and Russia. As the report notes, in 2014 China and Russia were the most blocked countries. If your company does not operate in these countries, it can use technology such as geo-IP fencing to provide protection against bad bots originating from these areas.
It would be a mistake, however, to focus only on a few specific regions when defending against bad bot traffic. There are many web hosts located worldwide that have little monitoring capability and few safeguards in place to prevent bad bots from originating.
It’s likely that the volume of bad bot traffic will continue to increase, particularly with the rise in mobile devices in the enterprise and the emergence of the internet of things. By taking the right steps now to learn about the issues and by working with other teams across the business, IT security teams can be better prepared to find these bots and keep them from becoming a security risk to the business as a whole.