Approximately 61.5% of all website traffic in 2013 was generated by bots and botnets, according to a report from web security firm Incapsula—up 21% from 2012. That means that only 38.5% of website traffic actually comes from real people doing real things.
Before IT runs screaming in the face of a Terminator-style apocalypse, the firm did say that the bulk of that growth is attributed to increased visits by good bots (i.e., certified agents of legitimate software, such as search engines) whose presence increased from 20% to 31% in 2013.
The increase from good bots can be explained for a couple of reasons. For one, the visitation patterns of search-engine type crawlers consist of re-occurring cycles. “In some cases we see that these cycles are getting shorter and shorter to allow higher sampling rates, which also results in additional bot traffic”, the firm said in its report. And, the emergence of new online services introduces new bot types into the pool. “For instance, we see newly established SEO-oriented services that crawl a site at a rate of 30-50 daily visits or more,” it said.
But even so, 31% of bots are still malicious. And there has been an 8% increase in the activity of “other impersonators,” a group of unclassified bots with hostile intentions. In terms of their functionality and capabilities, such “Impersonators” usually represent a higher tier in the bot hierarchy. These can be automated spy bots, human-like DDoS agents or a Trojan-activated barebones browser.
“The common denominator for this group is that all of its members are trying to assume someone else’s identity,” Incapsula said. “For example, some of these bots use browser user-agents while others try to pass themselves as search engine bots or agents of other legitimate services. The goal is always the same - to infiltrate their way through the website’s security measures.”
The generalized definition of such non-human agents also reflects on these bots’ origins. One way or another, these are also the tools of top-tier hackers who are proficient enough to create their own malware, the firm found. Where other malicious bots are agents of known malware with a dedicated developer, GUI, “brand” name and patch history, these “impersonators” are custom-made bots, usually crafted for a very specific malicious activity.
“The 8% increase in the number of such bots highlights the increased activity of such hackers, as well as the rise in targeted cyber-attacks,” Incapsula noted. “This is also reflective of the latest trends in DDoS attacks, which are evolving from volumetric Layer 3-4 attacks to much more sophisticated and dangerous Layer 7 multi-vector threats.”
While the relative percentage of malicious bots remains unchanged, the good news is that there is a noticeable reduction in spambot activity, which decreased from 2% in 2012 to 0.5% in 2013.
“The most plausible explanation for this steep decrease is Google’s anti-spam campaign, which includes the recent Penguin 2.0 and 2.1 updates,” Incapsula noted. “SEO link building was always a major motivation for automated link spamming. With its latest Penguin updates Google managed to increase the perceivable risk for comment spamming SEO techniques, while also driving down their actual effectiveness.”
Based on the firm’s figures, Google appears to have been able to discourage link spamming practices, causing a 75% decrease in automated link spamming activity.