Attackers consumed model credits worth about $600,000 over three weeks after stealing an API key from AI safety research group METR, the organization disclosed in a security update published Aug. 31.
The March exposure: a public EC2 instance, a "vibe‑coded" app and a fail‑open flaw
METR says the chain of events began in March when a researcher ran agents on a personal Amazon EC2 instance that had been made publicly accessible behind Google authentication. The researcher’s “vibe‑coded” app stored an API key for METR’s public models account. A fail‑open flaw silently disabled the authentication for several days, leaving the instance exposed.
METR said it suspects the attacker located the exposed instance by mining certificate transparency lists for recently registered sites containing high‑signal terms related to language models and agents. The attacker then prompted an agent on the instance to reveal the model provider API key, added an SSH key for persistence, and used the credentials to consume large volumes of model credits for roughly three weeks.
What the attackers did with the stolen key — and why it was hard to spot
According to METR, the stolen credentials were used to generate model traffic that METR’s defenses found difficult to distinguish from legitimate evaluation activity. Researchers at METR routinely generate high volumes of model queries during evaluations, and the organization said it had no technical way to cap spending on free‑credit keys supplied by the model developer. The credits in question had been provided free of charge by an unnamed model developer; METR characterized the roughly $600,000 figure as the commercial value of those consumed credits rather than a direct monetary loss.
METR’s response to the discovery included revoking the researcher’s access, rotating credentials, wiping the researcher’s laptop, and alerting the model developer. The organization later added spend alerts to keys where possible.
Nobody's watching your logs at 2 AM.
Full SOC coverage without building one. Nubivance deploys and manages Rapid7 InsightIDR and MDR for organizations that need detection and response, not another dashboard.
Get coverageMay probing: large‑scale agent reconnaissance and a transcript viewer bug
METR said it was tipped off in early May that threat actors were targeting its infrastructure. The attackers showed a pattern METR described as financially motivated and possibly aimed at gaining access to frontier models. They heavily used agents to automate vulnerability discovery, deploying techniques that included credential stuffing, OAuth token grant attempts, scanning for new services, and attempts to phish staff.
Separately, METR inadvertently exposed a read‑only SQL query mechanism through its public transcript viewer. The organization said a bug could have been exploited to reach unpublished evaluation data, and that the database had been accidentally loaded with sensitive model data it was not meant to hold. An independent researcher disclosed the flaw to METR; the organization took the interface offline and paid a bounty. METR said the probing attackers had scanned the endpoint but did not appear to discover or exploit the bug.
Why METR says these were external attackers, not evaluation agents
METR emphasized that its disclosure concerned external attackers rather than AI agents acting inside its evaluations. METR reported that an initial scan had found no evidence of agents used in its evaluations attempting to hack third parties. The organization also said that after the incidents it runs public‑facing applications in an environment architecturally separated from its internal infrastructure, and described its wider security measures as accurate to July 30.
What this means for technologists, model developers, and researchers
- Technologists and security teams: METR’s account highlights how small operational errors — a fail‑open authentication flaw or an exposed personal instance — can expose high‑value credentials. Teams running agent frameworks or high‑volume evaluations will watch for ways to separate public experimentation environments from internal systems and for mechanisms to detect abnormal spend that mimics legitimate traffic.
- Model developers (providers of free credits): METR’s disclosure points to a gap in policy and tooling: the organization confirmed it had no way to cap spending on free‑credit keys. Model developers supplying free or trial credits may be pushed to offer spend limits, finer‑grained alerts, or explicit controls for keys intended only for research.
- Researchers and program owners: METR’s remediation steps — access revocation, credential rotation, device wipes, and paying a bounty after an independent disclosure — underscore operational actions research groups may need to adopt quickly when investigation finds exposure or unexpected data in production databases.
METR reported no evidence that sensitive information was accessed in either the March or May incidents. Still, the episode raises concrete operational and tooling questions rooted in the events themselves: how to prevent accidental public exposure of experimental infrastructure, how to detect attacker use of agents that mimic legitimate evaluation traffic, and whether model providers will implement mandatory spend controls for free credits. METR’s move to separate public‑facing apps from internal infrastructure and to add spend alerts where feasible are specific mitigations; whether they will be sufficient across a field where high‑volume evaluations are routine remains to be seen.
Source: https://www.infosecurity-magazine.com/news/attackers-steal-metr-api-key/
