"When used to generate long-form technical content, large language models can deliver 'significant inaccuracies, unusual conclusions, and inconsistent writing styles,'" Nate Pors, senior incident commander on the Cisco Talos Incident Response team, wrote in a blog post describing a controlled experiment with AI-written incident reports.
Cisco Talos' diagnosis: four predictable failure modes
Pors laid out why the Talos team found large language models (LLMs) unreliable without guardrails. First, LLMs can use different data for each query, making "it’s difficult to rely on an LLM for repeatable, standardized research outcomes." Second, models may reach different conclusions from the same data — Pors gave an explicit example: a model might suggest "a full organization-wide password reset in one instance and a targeted reset in another," and then “often defaults to whichever recommendation it generates first.” Third, because LLMs generate content token-by-token they can produce documents with different structure and formatting on each run, which Talos called "problematic for professional environments" that need standardized layouts. Fourth, "AI can discard data," meaning output might ignore critical information.
Mitigations Talos tested: granular prompts, fixed sources, and style rules
To reduce those failures, Talos imposed constraints on the model. The team wrote granular, single-task instructions that focused the LLM on “a specific, small portion of the report,” a method Pors said significantly reduced the "risk of hallucination or cross-contamination between sections." They also fed the model explicit guidance about which sources to use and set firm rules for style and format.
Operationally, Talos learned to treat each report as isolated work: editing multiple sample reports in a single session produced cross-contamination “even if the notes used to generate the first report were deleted from the project’s reference documents.” The practical fix Talos recommends is to start a new session and re-enter prompts for every separate incident report.
Measured benefits: half the drafting time and a blind QA pass
Talos reports concrete time savings. Using their constrained approach, "the time required to draft an incident report based on a tabletop exercise fell by 50 percent," Pors wrote. In a blind quality-assurance test, reviewers were unaware the report was AI-generated; the peer reviewer, professional editor, and management reviewer "all made complimentary comments about the report." The peer reviewer noted that the "incidence of typos and grammatical errors was far lower than in the average report."
Persistent weaknesses: cross-contamination, poor grammar checks, and weak recommendations
Even with safeguards, significant issues remained. Cross-contamination persisted across sessions unless new sessions were started. Talos also developed a spelling-and-grammar-checking prompt that "hallucinated numerous grammar issues," "failed to identify actual issues," had a success rate below 50 percent, and "would behave inconsistently, sometimes catching issues and sometimes overlooking them." Pors concluded that this grammar-checker was "currently unsuitable for production use."
Separately, the team found the LLMs could generate recommendations that were "duplicative, irrelevant, or not actionable." Pors warned that "if this were used in a production environment without manual checks, it could result in poor-quality recommendations in a final report." The blog emphasizes that human authors must "take ownership of every word of the final report."
What this means for technologists, procurement leaders, and editors
- Technologists and security teams: The Talos experiment suggests LLMs can speed drafting of standardized outputs — but only when inputs are controlled and each report is produced in a fresh session; use in complex investigations (multiple system logs) remains a higher-risk proposition.
- Procurement and enterprise leaders: Cisco’s finding that time-to-draft fell by 50 percent will be attractive, but buyers should factor in the need for manual review and additional operational safeguards before adopting LLM-assisted reporting in production.
- Professional editors and peer reviewers: The blind-test result — reviewers complimenting an AI-assisted draft while unaware it was machine-generated — shows potential value, yet the team’s grammar-check prompt failing in production underscores that human editing remains essential.
Pors summed up the trade-off plainly: Cisco concluded the method "could be adapted to any cybersecurity reporting use case with standardized inputs and predictable outputs," but only with explicit, human ownership of the final text. For now, Talos’ work shows that AI can accelerate routine reporting — and that organizations must still guard hard against inconsistency, cross-contamination, and hallucinated recommendations if they plan to rely on it.
