"Imagine if AI is a birthday cake in the middle of this room, but you don't know how it got there," Ian Swanson, VP of AI security at Palo Alto Networks, told The Register. "You don't know the recipe, you don't know the ingredients, you don't know the baker. Would you eat a slice of that cake?"
Ian Swanson and the visibility problem
Swanson's metaphor captures the crux of a growing security gap: traditional software bills of materials (SBOMs) catalog packages and dependencies, but modern environments steeped in models, agents, prompts and datasets harbor components SBOMs were not designed to list. The gap has a name in security circles now — "shadow AI" (a descendant of the older "shadow IT") — and it includes unsanctioned vibe coding platforms, employee-spun agents, and external chatbots that may receive sensitive corporate inputs. Without a way to inventory these AI-specific ingredients, defenders lack the visibility needed to secure them.
What an AI-BOM records
An AI bill of materials, or AI-BOM, aims to map the full set of AI artifacts and their relationships: models, datasets, SDK libraries, MCP servers, ML frameworks, agentic skills, prompts, and other AI tools — plus how those components interact and connect to workflows. Amy Chang, Cisco's head of AI threat intelligence and security research, told The Register that organizations "want a way to be able to identify what AI assets exist in their environment" and that "a tool like the AI bill of materials is one of those first places that you can start to get a better understanding of what exists."
Vendors are already extending the concept beyond the final artifact. Ziad Ghalleb, Wiz technical product marketing manager, said Wiz's own AI-BOMs "also accounts for all of the tools in the developers' workstation, such as a laptop or integrated development environment, that went into building the AI application." He added that AI-BOMs should include "the identities that are attached to these AI workloads" — the non-human identities and permission sets tied to models, agents, and other AI systems — because those identities determine access and risk.
Cisco's Model Provenance Kit and its data
Cisco open sourced an AI-BOM scanner and, on a subsequent Friday, released a Model Provenance Kit as a separate open source repository. In a blog announcing the work, Chang and other researchers described the kit as a "DNA test for AI models." According to the description, the kit determines provenance using one of two modes: compare or scan. Compare mode takes any two models and shows their similarity across metadata, tokenizer structure, weight-level signals and a composite score. The blog also describes a mode called "Scam mode" that starts with a single model and matches it against a database to determine likely lineage candidates.
To support provenance matching, Cisco published a model fingerprint database covering about 150 base models across more than 45 families and over 20 publishers. Chang said the provenance checks operate in two gate stages: first by comparing metadata between a base model and a fine-tuned version, and second by looking at weight-based signifiers to provide a "verifiable, repeatable and provable way to attest" which models are actually deployed and in use.
Palo Alto Networks' incident, poisonings and runtime monitoring
Defenders are racing not only to catalogue AI assets but to protect them from manipulation. Swanson recounted a response in which a criminal group used AI to scout a victim's attack surface and locate exposed endpoints. The attackers obtained internal system prompts — the instructions given to an AI workload — and modified them to force the AI to perform actions it should not, such as stealing data and sending it to an external email account. He said an AI-BOM that captures "state and understanding of state changes" would have enabled investigators to see that a system prompt had changed and to flag the alteration for follow-up.
Beyond altered prompts, supply-chain style attacks against models and agent skills are a particular concern. Swanson warned that "skills that people use in coordination with a lot of these coding assistants are pretty easy to tamper with," and that a skill meant to provide a weather forecast "shouldn't also steal credentials or leak secrets." AI-BOMs and continuous scanning can reveal such tampering, and at runtime defenders should monitor communications to detect exfiltration attempts.
AI-BOMs also help identify compromised open-source libraries running inside AI applications. Both the recent wave of poisoned npm and PyPI packages and the earlier Shai-Hulud worm credential stealer targeted code commonly integrated into AI systems; an AI-BOM allows teams to query for related libraries even absent a CVE and to remove malicious versions from their environments, Ghalleb said.
How technologists, policymakers, and enterprise defenders are responding
- Technologists and security teams: Use AI-BOMs to inventory models, datasets, prompts and agentic components; include developer workstation artifacts and non-human identities to assess permissions and attack surface.
- Policymakers and regulators: Watch provenance tools as they relate to compliance. Chang pointed to the European Union's AI Act, which mandates documentation of training data, methodology and risk assessments for "high-risk systems."
- Enterprise procurement and risk leaders: Adopt provenance and fingerprinting checks — Cisco's model fingerprint database (about 150 base models across 45+ families and 20+ publishers) and open-source kits provide an early, auditable mechanism to establish which models were actually used in production.
AI-BOMs do not promise to solve every problem overnight, but the tools Cisco and others have open sourced give organizations concrete ways to reduce the unknowns in their AI stacks. If, as Swanson asked, many organizations are already "eating the cake," these bills of materials and provenance kits are the first cutlery: they let security teams see the ingredients, trace the recipe and spot when something in the kitchen has been tampered with.
