Nearly every website containing a login page is under attack from bots looking to access the site, a new report has revealed.
Distil’s 2017 Bad Bot Report, which looks back at 2016’s bot activity, found that 96% of websites with a login elements were hit by bot activity, while 97% of sites with proprietary content and/or pricing were hit by unwanted scraping. Around one-third (30%) of websites containing forms were hit by spam bots. Nine out of 10 websites were hit by bots that got behind the login page, which means they could have accessed sensitive data.
Overall what makes websites attractive to bots includes: unique content and/or product and pricing information; sign-up, login, and account pages; payment processors; and web forms, such as contact, discussion forums, and reviews.
In total, 40% of all internet traffic recorded in 2016 came from bots, with bad bots accounting for half of that traffic, a 7% increase on the previous year. Bad bots are increasingly hitting larger websites, the report said. These bad bots can be used by rival companies looking for a competitive advantage via inventory levels or pricing information, or they can be used to cyber-criminals for fraud, data theft, brute force attacks or account hijacking, for example.
Good bots, on the other hand, are crawlers such as Bingbot and GoogleBot, which index the web so results appear in web searches. They can also be used by businesses to ensure their products can be found online, Distil said. Good bots accounted for 18.8% of all web traffic in 2016, down 30% from 2015.
In order to avoid detection, bad bots disguise their identity. Most (76%) claim to be one of the four biggest browsers - Google Chrome, Firefox, Internet Explorer/Edge or Apple’s Safari. Interestingly, 16% of bad bots self-identified as a mobile browser, such as Safari on iOS devices. This figure increased 43% on 2015’s figure, and Distil said it expects that trend to continue.
Bad bot authors are taking advantage of the low cost and scalability of the cloud, with Amazon AWS accounting for 16% of all bad bot traffic. In total, 60% of bad bot traffic came from data centers, 30% from residential ISPs and 10% from mobile operators. “It’s never been easier to build bad bots with open source software or cheaper to launch them from globally distributed networks using the cloud,” the report said.
The US accounted for 55% of all bad bot traffic in 2016, more than all other countries combined. That doesn’t necessarily mean that the people behind the bots are in the US, however.
“A spammer bot might originate from the Microsoft Azure Cloud, but the perpetrator responsible for it could be located anywhere in the world. Individuals building careers by attacking US web properties generally live in countries that don’t have extradition treaties with America. Thanks to virtual private data centers such as Amazon AWS, such cyber crooks leverage US-based ISPs to carry out their attacks as if they originated inside America,” the report said.
“Massive credential dumps like Ashley Madison and LinkedIn, coupled with the increasing sophistication of bad bots, has created a world where bad bots are running rampant on websites with accounts,” said Rami Essaid, CEO and co-founder of Distil Networks. “Website defenders should be worried because once bad bots are behind the login page, they have access to even more sensitive data for scraping and greater opportunity to successfully carry out transaction fraud.”