Skip to main content
CybersecurityVulnerability Management

AI Pentesting Vendors Face Scrutiny as Security Gaps Persist

Empty cybersecurity testing laboratory with computer workstations and network equipment in a bright, neutral space.

In 1H 2026, 23% of vulnerabilities VulnCheck added to its exploited list were hit on or before disclosure day — a blunt metric that underscores one bottom-line dynamic: attackers move first. (Mandiant M-Trends 2026.)

Mandiant M-Trends 2026 and VulnCheck timing

The headline is simple and consequential: exploitation increasingly precedes fixes. The source material cites Mandiant’s M-Trends 2026 finding that, on average, exploitation now starts before a patch exists, and a separate VulnCheck statistic showing that nearly a quarter of newly listed exploited vulnerabilities in the first half of 2026 were targeted on or before disclosure day. That reality compresses the window defenders have to find and remediate flaws — and it reframes what an effective pentest must deliver.

Google’s Threat Intelligence Group and an AI-crafted zero-day 2FA bypass

AI is not just accelerating conventional discovery; it is being used to develop novel exploits. The Google Threat Intelligence Group reported a threat actor who used AI to develop a zero-day exploit that bypassed two‑factor authentication through a logic flaw — a class of defect conventional fuzzing and static analysis can miss. The implication in the source material is direct: defenses and assessments must reach beyond signature or pattern matching and reason about logic and trust relationships inside an application.

How agents learn your app: starting maps and CyCognito timing

Agentic pentesting's value depends on what the agent knows before it launches an attack. Recon and analysis are real attack work: in CyCognito’s model, that phase consumes 20–30 hours compared with a manual pentest’s 63–95 hours. Vendors diverge on whether they rebuild a map from scratch on every run or start from a maintained site model that can carry forward context between tests. The source stresses one principle: a starting map should be a head start, not a boundary — the agent must keep discovering beyond its initial model if it is to uncover chained or context-dependent flaws.

Proof, guardrails, and the Reflectiz insurer case

Findings that look suspicious are not the same as validated, exploitable vulnerabilities. The source lays out what proof must include: exploitation of the suspected issue, reproducible evidence, and artifacts your team can verify. It gives a concrete example from an authorized assessment of a major insurer’s production website, described in a Reflectiz case study on The Hacker News: an AI pentesting agent discovered that a one-time passcode could be sent to any email address, proved the flaw by logging in with a code sent to its own inbox, and was then stopped by vendor-implemented guardrails. That episode illustrates both the power and the danger of running autonomous tests in production — and why robust, enforced limits matter.

Run a bake-off: the 15 checks, the 36-point scorecard, and common red flags

Because most vendors “demo well,” the source recommends a structured bake-off on your own application rather than relying on vendor demos. Practical, repeatable rules are provided:

  • Run every vendor on the same target, in the same week, with the same app, test accounts, and scope.
  • Log three timestamps for each run: access granted, discovery complete, and first test.
  • Score every check with a per-check scoring sheet; the full checklist includes 15 checks and a 36-point scorecard that weights safety and impact.

The listed checks cover attack chaining, roles and cross-role access, coverage proof, data handling, guardrails, and retesting. Key red flags include agents that stop at login, cannot explain or prove what they cover, or have no hard limits when running against production.

What this means for technologists, procurement leaders, and insurers

Technologists and security teams should insist vendors demonstrate that agents can reason about application context, reuse and extend a starting map, and produce reproducible proof — not just suspicious findings. Procurement leaders should require a same-target bake-off using the checklist’s scoring sheet and log the three required timestamps to compare vendors objectively. Insurers and other organizations that test in production must enforce and verify guardrails; the Reflectiz example shows both the downsides of production testing and the necessity of stopping tests when exploitation is proven.

The central judgment the source leaves on the table is clear: the best AI pentesting agent is not the one that gives the slickest demo, but the one that can show what it tested, prove what it found, and run safely on your application. Use the 15 questions, the red flags, and the 36‑point scorecard to force vendors to prove those three things before you hand over access.

https://thehackernews.com/2026/11/ai-pentesting-checklist-for-web-apps-15.html