Pentagon Social Media Surveillance Data in Open S3 Buckets (Nov 2017)

Retrospective: this article looks back at events from November 2017, written in 2026 with the benefit of hindsight.

In November 2017, UpGuard researchers found three Amazon S3 buckets configured for public access that contained billions of social media posts collected as part of a US Department of Defense intelligence program. The buckets were linked to US Central Command and Pacific Command.

What was exposed

The data consisted of public social media posts and online comments scraped from around the world, apparently gathered for monitoring purposes. While the posts themselves were public, the collection revealed the scope and focus of a government monitoring effort.

How it happened

The same way as the other 2017 S3 exposures: buckets configured to allow access to anyone, or to any authenticated AWS user. In 2017, the "authenticated users" option was widely misunderstood. It did not mean users in your organization — it meant anyone with any AWS account.

Why it mattered

If defense organizations with substantial security budgets could misconfigure cloud storage, the problem was clearly about process and tooling, not just resources. The incident added to pressure for preventive guard rails rather than reliance on careful configuration.

Lessons in hindsight

• Understand what each permission setting really means. "Authenticated users" was a trap.
• Use preventive controls, not just detective ones. Organization-wide policies that make public access impossible remove whole classes of mistakes.
• Contractors and programs need oversight. Data collected for one purpose often sits in storage long after the project ends.
• Classify data by aggregate sensitivity. Public data, collected and organized, can become sensitive.

AWS later removed the confusing ACL options from the console default experience and made Block Public Access the default — a direct response to years of incidents like this one.