Major privacy concerns have been raised after researchers discovered terabytes worth of data scraped from the internet by the Pentagon on what appear to be law-abiding citizens around the world.
The data was found by UpGuard’s Chris Vickery, who discovered it exposed to the public internet as the result of yet another Amazon S3 database misconfiguration, back in early September.
In just one of the three buckets discovered, there are 1.8 billion internet posts dating back eight years, including content scraped from news sites, comment sections, web forums and social media sites like Facebook.
It relates to both American citizens and those from other countries, and linked to CENTCOM (US Central Command) and PACOM (US Pacific Command).
The data trove appears to have been managed by “VendorX” as part of a project known as “Outpost”, designed to monitor “high risk youth in unstable regions of the world.” There are also links here to the army’s “Coral Reef” program which helps the government better understand online connections between persons of interest.
“Taken together, this disparate collection of data appears to constitute an ingestion engine for the bulk collection of internet posts - organizing a mass quantity of data into a searchable form,” wrote UpGuard’s Dan O’Sullivan.
“Given the enormous size of these data stores, a cursory search reveals a number of foreign-sourced posts that either appear entirely benign, with no apparent ties to areas of concern for US intelligence agencies, or ones that originate from American citizens, including a vast quantity of Facebook and Twitter posts, some stating political opinions. Among the details collected are the web addresses of targeted posts, as well as other background details on the authors which provide further confirmation of their origins from American citizens.”
The findings are concerning because the government is legally prohibited from using the US military as a tool for law enforcement, except in cases of national emergency.
They also show the US army’s poor cybersecurity posture: PACOM and CENTCOM apparently have CSTAR risk scores of just 409 and 542.
“A simple permission settings change would have meant the difference between these data repositories being revealed to the wider internet, or remaining secure,” concluded O’Sullivan.