By Hamza Faisal

Security researcher HD Moore wanted to know what happens to a secret once you type it into an AI app. So he planted nearly two million fake ones in prompts sent through the services that power modern AI tools — and then watched where they surfaced. The answer, published this week by security firm Dragos, was not comforting: the AI supply chain leaked sensitive data like a sieve, and the companies doing the leaking had no idea.

Over five months, from July through November 2026, Moore and his team submitted around two million prompts containing unique, trackable secrets to widely used AI services. Each secret — an AWS access key tied to an empty account, an SSH password, a unique URL, or a reference to a private code repository — went into one prompt sent to one provider. Each was used only once, and Moore controlled the infrastructure that would receive any attempt to use it, so a touch would reveal exactly which link in the chain had leaked it.

Three providers were breached without knowing

The most serious finding was also the most direct. Moore found evidence that attackers had compromised at least three AI hosting providers and were siphoning off the prompt data flowing through them. The largest breach involved around 200 AWS keys, all stolen in a few short windows in late October. Every request sent during those windows was taken, Moore wrote, and the stolen keys were then used across Amazon services including S3 and IAM.

The thefts were not a one-off burst. The earliest confirmed activity dates back to the middle of July. One stolen AWS key, seeded on October 21, was first used the next day and was still being used three weeks later. None of the three providers knew any of this was happening — they learned about the breaches only when Moore notified them in November.

Middlemen logged prompts, and privacy settings were ignored

Outright compromise was only part of the picture. Many AI apps never talk to a model directly. They hand prompts to brokers such as OpenRouter or Vercel, often called AI routers or gateways, which forward each request to whichever inference engine fits — usually the cheapest. Moore found that multiple resellers in these marketplaces could view and retain customer prompts using standard logging, with no exploit needed.

The experiment also caught providers ignoring the privacy settings users had chosen. One Chinese AI lab fetched the secret URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of the requests marked as no data collection or Zero Data Retention. In another case, a unique URL seeded during the study showed up in an internal chat at a Chinese company 24 days after the original request.

What it means for anyone typing secrets into AI

The reason the leaks matter is what prompts actually contain. Prompts are encrypted in transit, but they are decrypted at each hop inside the inference network so routers can read them and decide where to send them. Along the way they frequently carry confidential source code, internal communications, strategic plans, customer information and credentials.

Moore's conclusion is blunt: anything placed in a prompt should be treated as potentially exposed. Never paste live credentials, API keys or private links into an AI tool, use redacted or placeholder values when testing, and check what a provider actually promises about prompt retention before trusting it with sensitive work. The findings were publicly summarised on November 19, and Moore said the picture is unlikely to improve soon.