"The big hype around this model so far was primarily marketing," Daniel Stenberg wrote after a limited test of Anthropic's Mythos against cURL's codebase.
Daniel Stenberg, cURL, and the promise of Project Glasswing
Daniel Stenberg, the long-time developer of the open-source tool cURL, says he was promised access to Anthropic’s Mythos through Project Glasswing, a program that involves giving high-profile open source projects access via the Linux Foundation. Stenberg signed up to try Mythos but never received direct access himself; instead, someone with access ran Mythos against cURL’s git repository at a recent master-branch commit and sent him the resulting report.
What Mythos reported — and what cURL found after review
The Mythos scan returned a short report claiming five "confirmed security vulnerabilities" in cURL. Expecting a longer list, Stenberg and his security team reviewed the findings for hours and reduced that list to a single confirmed vulnerability. Three of the original items were false positives that pointed to cURL shortcomings already noted in API documentation, and the fourth was categorized as a simple bug rather than a security flaw.
The lone confirmed issue will be published as a low-severity CVE in sync with the planned cURL release 8.21.0 in late June. Stenberg stressed the flaw "is not going to make anyone grasp for breath." Mythos did, however, surface several non-security bugs whose descriptions and explanations Stenberg said were "well done" and which the team is working to fix.
Mythos in context: not inventing new classes of bugs
Stenberg placed Mythos alongside earlier AI-powered code-analysis tools rather than on a pedestal. He noted that AI tools like AISLE, Zeropath, and OpenAI Codex Security have already been used on cURL and that those analyses have triggered "somewhere between two and three hundred bugfixes merged in curl through-out the recent 8-10 months or so." He added that "probably a dozen or more" of those findings were confirmed vulnerabilities and published as CVEs.
On Mythos specifically, Stenberg concluded it "might be a bit better at finding things than previous models, but it is not better to a degree that seems to make a significant dent in code analyzing." He argued that AI tooling—Mythos included—primarily rediscovers established categories of errors: "AI tools find the usual and established kind of errors we already know about. It just finds new instances of them." He conceded it would be possible for AI to discover novel vulnerability types but remained unconvinced by the Mythos results he saw.
Human oversight, not AI alone
Stenberg emphasized that useful security work still requires human judgment. He recalled that some researchers assisted by AI have made valuable reports, but he also cited the recent closure of the cURL bug bounty earlier this year after a flood of "sloppy, useless bug reports" drove the team to suspend it. "Human researchers have always used tools when they look for security problems," he told The Register. "Adding AIs to the mix gives the humans even more powerful tools to use, more ways to find problems. I expect that many security bugs going forward will be found by humans coming up with new ways and angles of prompting the AIs."
Stenberg also framed source code as text and suggested that many security problem types may already be well understood: "Source code is text and it feels like maybe we already know about most ways we can do security problems in it."
What this means for technologists, open-source maintainers, and security teams
- Technologists and security teams: Expect AI tools to continue to find repeat instances of known vulnerabilities and to provide useful bug descriptions, but plan for manual triage; cURL’s review reduced five claimed vulnerabilities to one confirmed CVE.
- Open-source maintainers: The cURL case shows that receiving an AI-generated scan is only the start — Stenberg’s team spent hours verifying each finding and judged several to be false positives or non-security bugs.
- Project managers and procurement leaders involved with programs like Project Glasswing: Promises of access may not equal immediate, hands-on access — Stenberg was told he would receive Mythos access but to date has only received a report someone else generated.
Anthropic’s Mythos, in Stenberg’s telling, can produce helpful analyses and identify defects that human teams will then fix. But after testing — albeit indirectly — on a mature, heavily-tested codebase that has already been subject to multiple AI and traditional analysis tools, Mythos did not reveal a trove of previously unknown, high-severity security problems. The immediate, practical next steps are concrete: Stenberg’s team will publish a low-severity CVE alongside cURL 8.21.0 in late June, they will fix the non-security bugs Mythos identified, and Stenberg continues to wait for the direct access to Mythos he was promised.
Read the original report: Anthropic’s bug-hunting Mythos was greatest marketing stunt ever, says cURL creator
