Skip to main content
Emerging ThreatsData Breaches

AI Hosting Firms Expose Customer Data in Months-Long Breaches

Interior of a cloud computing facility with server racks and scattered monitors.

"A privacy flag doesn't help when the attacker is already inside," wrote HD Moore, the firm's chief research officer.

Dragos's canary-token experiment and timeline

Industrial cybersecurity firm Dragos seeded roughly two million prompts with unique web links and credentials between July and November to see whether any third party read them. Each planted secret was unique; any use of it would prove someone had read the carrying prompt. The earliest misuse tracked by the experiment began in mid‑July, and researchers say planted keys and links continued to be accessed months after the first thefts.

How attackers stole requests and what they did with secrets

Dragos reported three separate provider compromises that together show multiple techniques. The largest involved "a couple hundred" AWS keys planted and sent in single requests to a provider during short windows in late October; every request during those windows was copied, the firm said. In some cases requests were stolen before a model ever ran, indicating interception at the first point of arrival. Thieves tested keys within hours, listed S3 buckets, searched Secrets Manager and Parameter Store, attempted to create Bedrock API keys and add IAM users, and generated tens of thousands of AWS API calls.

One attacker appears to have compromised two providers simultaneously and used secrets from both from a single rented server. On November 1 that attacker ran ARTEX, an AI‑driven penetration testing framework, but only to confirm previously held keys were still live; two of ARTEX's user agents, artex-agent and artex-judge-r121, appear in no public indicator list the researchers could find. In other cases, no break‑in was required: sellers on an AI inference marketplace listed models at rock‑bottom prices, enabled request logging, and collected traffic that flowed through their resale capacity.

Concrete provider disclosures: Phala, Together AI, Novita

Three providers have posted public notices. Phala said an attacker took control of an inference server and could view request data while it was processed; the incident was contained on November 8. Phala advised that requests to the DeepSeek V4 Flash 0731 model routed through the gateway between October 21 at 00:00 UTC and November 8 at 05:35 UTC should be treated as exposed and that every secret within them should be rotated. Phala noted a third‑party researcher had been placing unique canary credentials since July 1 and that only prompts sent on October 21 and 22 were later misused.

Together AI said an unauthorized party downloaded stored samples of customer prompts after an AI coding agent committed a credential to the company's public GitHub repository. The samples — collected for routine software testing — were generated by requests to MiniMax M3, GLM‑5.2, and Kimi K2.6 models in June and July. The party searched the samples for credentials and tested them; five of the tested credentials were canaries from the Dragos experiment. Together AI said it has no reason to believe the data went further, understands it has been deleted, and has made Zero Data Retention the default for new accounts.

Novita confirmed a routing configuration issue caused some MiniMax requests to be forwarded to MiniMax's official API beyond what customers expected.

OpenRouter's response and the remaining uncertainty

OpenRouter, a gateway that routes customer requests to third‑party inference providers, removed three providers from its platform, removed 125 endpoints from eligibility for Zero Data Retention, and notified potentially affected customers. OpenRouter said the researchers' findings showed no breach of OpenRouter itself. The 125 endpoints removed cover providers flagged by the researchers plus others removed as a precaution; any endpoint taken off the Zero Data Retention list will require assurances and validation before its status is returned.

The gateway said it is adapting Dragos's testing approach for ongoing checks and is verifying network paths to identify where requests and responses may be forwarded to third parties. Not all affected providers have posted public notices, and OpenRouter has not said whether the three providers it removed are the same three Dragos identified.

How technologists, procurement leaders, and end users should respond

  • Technologists and security teams: treat a privacy flag as binding only to the first network hop and assume resale pools and marketplaces may route traffic beyond the intended provider; plant fake credentials or use Canarytokens as tripwires and verify that scrubbing settings actually remove known credential formats.
  • Procurement and platform operators: prefer self‑selected upstream providers over default pools, require validation when endpoints regain Zero Data Retention status, and verify routing paths so customers know whether requests are forwarded to third parties.
  • End users and application builders: consider self‑hosting when no third party may read prompts; if using routed providers, rotate any exposed keys immediately and avoid reseller marketplaces whose capacity may run on stolen keys.

Dragos's evidence is blunt: secrets planted as bait were used, no single privacy toggle prevented copying, and some stolen content was passed to and read by human beings. The researchers are working with about a dozen providers; the pattern they found — interception at arrival, resale collection, and long‑running crawls — leaves the core question simple and stark: if a prompt touches third parties, the customer must assume someone else may read it. For many organizations, the choice between convenience and exposure now looks more like a choice between contracts and unknown caches.

Source: The Hacker News — OpenRouter Drops 3 Providers After Researchers Catch Months of Prompt Theft at AI Hosting Firms