“We are running this as a security industry as a sprint—oh my gosh! … This is going to be the long haul. We're going to be doing this forever,” AWS Chief Security Officer Stephen Schmidt warned, describing an accelerating cycle of discovery, patching and renewed attacks as open-source AI spreads.
Nvidia, Amazon, Meta, Google, and Microsoft embrace open-weight models
On July 24, Nvidia CEO Jensen Huang — joined by Amazon, Meta, Google, Microsoft and other signatories — said in a statement that “Open-weight models—AI models that anyone can download, inspect, modify, and run on their own infrastructure—are an important part of that foundation.” The declaration marks a visible shift by firms that previously emphasized proprietary, closed models as essential to safety. The same week Microsoft executives touted a catalog of “over 11,000 models,” while AWS CEO Andy Jassy noted more than 10 models available through Amazon’s Bedrock platform.
AWS on how attackers are “poisoning” open-source libraries
AWS security teams report adversaries using increasingly subtle techniques to compromise open-source code. Rick Anthony, Sr., who manages Amazon Inspector, described malicious packages that “are doing real useful benefits” while hiding harmful behavior that triggers only in narrow circumstances — for example, when a user issues a prompt with a typo or when additional code is added later. These attack chains are being enabled and accelerated by AI coding agents that can craft convincing contribution histories and realistic release cycles, Anthony said.
Built by Nubivance.
OSINTSights' secure edge-first architecture, AI content pipeline, and serverless ops are designed by Nubivance. We do this for clients too.
Talk to us →Attackers exploit AI-based reviews and nation-state advantages
AWS warns that attackers are taking aim not only at human reviewers but at AI review systems themselves. “What we're going to see is attackers not only try to fool the humans, but try to fool the AI by giving it enough evidence to convince it that what you're running is ‘OK,’” Anthony said. Schmidt singled out adversaries such as China and Russia — and also named North Korea — as being “in a great position to carry out such attacks” because they do not face penalties for testing against real-world targets, a constraint that alarms AWS security leadership.
Red teams, patches, and the cost of continuous response
AWS has deployed red teams that run with AI agents to find vulnerabilities before adversaries do, and to simulate the kinds of attacks that might be used against emerging open-weight models. But Schmidt emphasized that discovering a flaw is only step one: producing a usable patch, rapidly distributing it, and then validating the patch against novel adversarial behavior is slow and resource-intensive. He said AWS tests patches for both performance and resilience to “certain kinds of adverse behavior, because we know that the adversaries are going to go after them as soon as we release them to the public.”
Anthropic’s absence and its safety demands
Notably absent from Huang’s signatory list was Anthropic. The company’s CEO issued a separate statement this week: he does not call for banning open-weight models outright, but “supports mandatory safety testing for all models, as well as other measures to curb China’s ability to copy powerful models like Mythos,” the CEO wrote. Anthropic researcher Julie Merz added a sharper warning in a post on X, predicting that “This time next year there will be the internet hitting every rural hospital/city council/etc at once with crypto locker attacks,” and criticizing a “shocking lack of imagination in a lot of the CEOs/influencers pushing open models.”
How AWS, open-source maintainers, and the Pentagon are responding
- AWS: Expanding red teams that use AI agents, accelerating patch development and validation, and integrating tools such as Amazon Inspector to detect malicious contributions in open-source code.
- Open-source maintainers: Facing a rising workload as attackers create realistic-looking contributors and poisoned packages; maintainers will need to vet contributions more deeply as AI agents flood ecosystems with plausible code.
- The Pentagon: Seeking AI it can control and operate without dependence on large, targetable data centers — a factor cited in the broader shift toward open-weight models that can run on local infrastructure.
The facts laid out in recent statements and interviews sketch a clear tension: the same openness that democratizes advanced AI — giving startups and small players access to models they can inspect and run locally — also expands the attack surface and rewards adversaries able to experiment at scale. Big Tech has repositioned itself from owner-builder to platform-and-security provider, investing in catalogues, tooling and inspection services even as it warns that defenders must move as quickly as attackers. The crucial unanswered operational question is whether the industry can compress discovery-to-patch cycles enough to keep pace with adversaries who can weaponize AI-guided development at will.
