"The knowledge needed to do some of that stuff is getting less likely to be necessary," said Steve Swick, a warning grounded in an October 2026 study of AI's use against industrial control systems.
That warning came during a panel discussion tied to the release of Frenos' State of AI in Security report. Frenos — an AI pentesting company that works on operational technology (OT) environments — ran nearly 17,000 automated tests to measure how current AI models perform at choosing actions, planning multi-step attacks, and executing simulated scenarios against industrial systems. The results, and the participants' remarks, sketch a picture that is neither doomsday nor reassurance: AI is making reconnaissance and tooling faster and easier, but it remains error-prone and uneven.
Steve Swick on "security by obscurity" for utilities
Steve Swick, former chief information security officer at American Electric Power, described a long-standing reliance on "security by obscurity" in utility systems and warned that such obscurity is eroding as AI reduces the specialized knowledge once needed to target OT environments. Swick's point: access to manuals and design documents, when combined with AI parsing, shrinks the barrier to understanding operational systems.
Frenos' tests: scale, models, and success rates
Frenos published the State of AI in Security report on Wednesday and reported running 16,980 tests from October 2025 through September 2026: 13,884 decision attempts, 2,520 planning attempts, and 576 simulated attacks. The company evaluated a mixture of models — six frontier models, two U.S. open-weight models, and two foreign open-weight models — across three task types: single-action decisions, step-by-step planning, and simulated scenario execution.
Performance varied widely. Depending on the model, between 6 percent and 47 percent of single-choice decisions matched the required outcome. The strongest three frontier models produced workable attack plans in about 90 percent of their planning attempts, but they were inconsistent: only 60 percent, 62.9 percent, and 74.3 percent of those plans passed every repeat. One non-U.S. open-weight model achieved a 36.2 percent overall success rate; by comparison, three U.S. frontier models scored 31.9 percent, 18.8 percent, and 17.9 percent respectively.
This site is the portfolio.
OSINTSights runs on Cloudflare Workers, D1, R2, and Vectorize, with an AI pipeline on Hetzner ARM. Nubivance designed, built, and operates it. We do the same for clients.
See what we buildPaul Shaver and Armadin: AI-assisted reconnaissance and rapid tooling
Paul Shaver, head of OT at Armadin, described how AI accelerates the reconnaissance phase. "I can download the manual for almost any automation controller, parse through that very quickly, and come up with ways that I can exploit a dedicated feature of something that was not secure by design," he said. Armadin — the startup that uses swarms of autonomous AI agents to simulate attacks — is led by figures with Mandiant lineage; Shaver spent six years as the global OT practice leader at Mandiant, which the report notes was acquired by Google in 2022.
Shaver said AI lets experienced operators build tools "in days that would normally have taken me months," and that capability applies to less-skilled actors as models improve. He stopped short of saying script kiddies have already reached nation-state sophistication, observing instead that "the potential is there."
Harry Thomas on hallucinations, harnesses, and human judgment
Harry Thomas, Frenos' chief technology officer and co-founder, emphasized limits the report exposed. Frenos found that models were prone to "hallucinations" — mistakes that left plans missing essential steps or assuming access that wasn't established. In the tests, humans did not iterate with the models to correct or refine plans; Thomas noted that in real attacks an attacker could provide more context and request revisions.
Frenos also used what Thomas called a "pretty simple" harness for its evaluations — supplying context and asking for an answer. He said a more capable harness could further amplify model effectiveness, and that human judgment remains essential for models to reach an adversary's objective.
What this means for utilities, OT defenders, and miscreants
- Utilities and critical-infrastructure operators: expect an acceleration in timelines for reconnaissance and tooling that reduce specialized requirements. The report's suggested mitigations focus on layered defenses — network segmentation, limiting remote access and permissions, and validating recovery steps.
- OT security teams and defenders: the tests reinforce the need to execute business continuity and disaster recovery plans and to assume that faster information processing and automated tooling can increase the volume of lower-skill threat actors able to probe systems.
- Miscreants and lesser-skilled threat actors: the findings indicate a growing capability gap can be bridged by AI-assisted reconnaissance and planning, but model inconsistency and hallucinations mean human operators still influence success.
The Frenos report does not depict an immediate transformation of every novice into a sophisticated nation-state attacker, but it documents a measurable lowering of barriers and a distinct acceleration in the tools available to would-be intruders. Its practical prescriptions are familiar — segmentation, access controls, tested recovery — and its data make clear why operators should prioritize executing those plans now rather than later.
