"These models have been getting better and better and better, which is amazing to see, because they only get better when people try things and they fail, and they push the model vendors to keep pushing the boundaries," said James Spiteri.
James Spiteri and Elastic Security Labs' rapid experiment
Returning from the RSAC conference, one of Elastic Security Labs' researchers decided on a Friday to build a lightweight pipeline that used a live AI agent to monitor package repositories. James Spiteri, head of Elastic Security Labs, described how that quick experiment grew into a practical detection capability: the agent used a large language model to evaluate changes pushed to package repos and to alert on Slack if the model judged a change malicious. Spiteri said Elastic then moved from alerting to actively responding — reverse-engineering the incident, publishing detections and sharing findings in real time.
How the agent watched PyPI and npm
The pipeline was pointed at the top 15,000 packages on PyPI and npm by download count, a list chosen to cover the most widely used distributions. The agent's workflow was deliberately simple and focused: monitor changes as they were pushed, use an LLM to determine whether those changes looked malicious, and notify a human on Slack. Elastic has open sourced the resulting supply-chain monitoring tool, a detail Spiteri highlighted when discussing the project's success and lessons learned.
The Axios backdoor: minutes from push to alert
Three days after the experiment began, the agent triggered an alert minutes after an actor backdoored the latest distribution of Axios, the popular JavaScript library for making HTTP requests. Axios is distributed through npm, the default package manager for the GitHub-maintained Node.js runtime, and the library sees more than 100 million downloads per week. The researcher who received the alert reached out to Axios' maintainers on the social platform X, and Elastic Security Labs then took a hands-on role: the team reverse-engineered the malicious package, "found exactly what was happening," and published detections and findings as events unfolded late at night in America, according to Spiteri.
How other researchers and attribution entered the picture
Spiteri described a rapid, collaborative response: other security researchers began sharing findings, and the community response was "pretty incredible," he said. The incident was publicly tracked alongside reporting that tied the backdooring of the Axios library to North Korea (see: Backdooring of JavaScript Library Axios Tied to North Korea). Elastic's playbook in this case combined automated LLM-driven monitoring with fast, manual reverse-engineering and public sharing of indicators and detections.
What this means for open-source maintainers, security teams, and LLM developers
- Open-source maintainers: The Axios episode underscores how quickly a malicious change can be pushed and distributed — maintainers who oversee high-download packages face a fast-moving threat that can emerge within minutes of a distribution change.
- Security teams and SOC operators: The incident showed that a lightweight, agentic LLM pipeline can provide early warning and an immediate triage path; Elastic published detections in real time and used human reverse-engineering to validate and amplify the alert.
- LLM developers and researchers: Elastic credited continuous experimentation — and tolerating failure during that work — with improving model utility. Spiteri said potential future uses include identifying threat actors and emulating attacks to build better detections, and that code-analysis capabilities in the latest LLMs could bolster many security roles.
Open source release and next steps
Elastic has open sourced the monitoring tool that spotted the Axios backdoor. Spiteri framed the episode as proof that practical AI-driven tooling can catch supply-chain tampering when researchers iterate quickly and combine automated detection with human validation. He also noted that models "only get better when people try things and they fail," a contention he used to encourage continued experimentation with generative AI in security work.
The Axios alert is a compact case study: a fast-built LLM agent, focused monitoring of the most-downloaded packages, a near-instant alert after a malicious push, and an immediate human-led response that produced reverse-engineering results and published detections. The episode leaves a clear question for defenders and developers alike — how many more such lightweight, experiment-driven pipelines will be needed and adopted to provide continuous coverage across the long tail of open-source software?
Original story: How AI Supply-Chain Monitor Spotted Unfolding Axios Attack
