Online businesses’ risk from data theft due to web scraping—harvesting website info—has almost doubled, especially for sectors like travel sites.
An analysis of ScrapeSentry’s Global Scraping Intelligence Platform shows that 2014 saw an increase in overall scraping activity for the fifth year in a row. Across all sites, 22% of visitors to websites are identified as scrapers (up from 18% last year. And there has been a 17% increase in scraping attacks across all industries in 2014—the hardest hit site in the survey received 60% of its traffic from scrapers.
About 49% of all scraping activity originated from the US.
“We know that the means to scrape is now even more easily sourced and the profits from scraping can be considerable, so the incentive to continue to scrape and to invest in it as a business form in itself grows,” said Martin Zetterlund, founding partner of ScrapeSentry. “Scrapers are not going away, and the threat to online businesses will continue to increase. This year has shown some worrying growth in data theft, and we hope that all online businesses, and the ones in the at-risk sectors in particular, will take this report as a wake-up call to review how they are being affected by scraping.”
In terms of those at-risk segments, the travel industry has seen scraper numbers more than double from 15% of visitors in 2013 to 33% last year.
“Very recently we’ve seen airlines announce increases in booking fees to tickets booked through third parties, because scraping and the loss of revenue that causes is a very real threat to their businesses,” said Zetterlund. “The 2015 Scraping Threat Report clearly shows the challenges the travel industry and airlines in particular face.”
Online classified sites have seen a similarly dramatic increase, from 14% to 30% in the same time frame. Worse, ticketing sites have also seen a big increase in scraping traffic over the year, jumping from 9% to 27% of visits.
Other sectors saw upticks too: Online directories suffered from almost a third of visitors being scrapers at 30%, which is up 8% on last year; betting sites experienced high amounts of scraping traffic at 25% in 2014, thanks in part to the World Cup, when in 2013, they saw only 8%; and the general e-commerce sector experienced just a modest 3% increase in scraping, to bring it to 15%.
ScrapeSentry has also identified a botnet which has infected more than 1.3 million IP addresses over the course of 2014, and is seen to be growing by 10,000 new infections a week.