Benchmarking AI: leaders report strong returns but measurement lags
Economist Enterprise surveyed more than 1,200 senior technology executives from 18 countries, including 296 CIOs, and interviewed leaders at companies such as Disney, Mercedes‑Benz, Nasdaq, Atlassian and Takeda. The headline finding is stark: four out of five executives said their AI programs are beating expectations, yet fewer than half actually track whether that is true. To separate signal from hype, the report introduces a benchmarking framework intended to identify which companies are generating real AI returns versus those stuck in "pilot purgatory" — shortfalls in strategy, technical foundations, governance and workforce transformation.
CTOs, vice presidents, and the perception gap on rollout progress
The survey revealed divergent views between C‑level technologists and their senior vice presidents. Nearly 90% of CTOs said their AI rollouts were ahead of schedule, but only three in four senior vice presidents concurred. In IT, three in five C‑level tech leaders reported AI was fully embedded at scale, while two in five vice presidents agreed. Among respondents identified as AI leaders, 84% said returns were better than expected, yet only 43% require teams to measure business impact — a striking mismatch between confidence and accountability.
Data architecture, costs, and the economics of clean data
Data governance and architecture emerged as critical differentiators. Firms with a unified data architecture reported faster returns: 97% of those firms said they were seeing ROI ahead of schedule, compared with 77% for firms without unified architectures. Executives named data storage, movement and duplication as the biggest ongoing AI cost (59%), while infrastructure and compute costs were cited by just 25%. "When you ask firms what is the biggest thing that worries you, the greatest share tells us it is data storage, movement and duplication," Milev said. Atlassian's Tal Saraf, senior vice‑president, engineering and CIO, reinforced the point: "We have done work to retire 99% of our legacy or fragmented data, and that now gives AI the ability to answer questions much more definitively and makes the insights of AI agents more valuable."
Agentic risk: the 'lethal trifecta' and practical mitigations
The report warns of a specific security exposure for autonomous agents. Economist Enterprise identifies a "lethal trifecta" when agents have simultaneous access to untrusted outside content, sensitive corporate data and the ability to communicate externally. Large language models cannot reliably distinguish underlying data from instructions, leaving them open to malicious commands hidden in text. Three in five leading AI adopters reported autonomous agents doing real work, yet fewer than half mandate a formal governance framework for those agents. Milev described mitigation steps used by leading firms: deploying AI gateways, giving business owners kill‑switch authority, and authorizing monitoring agents to oversee other agents.
Culture, process and the hidden cost of human review
Process and culture matter as much as technology. The study found only 40% of enterprises have established AI development life cycles, and the timeline to move pilot projects into production remains lengthy: 58% of enterprises expect seven to 12 months. Governance enforcement is inconsistent across life cycles — 59% of respondents said they conduct security reviews during development and before deployment, but only 39% continue reviews after a system goes live, and one in eight admits to reviewing governance only when something goes wrong. The human side of AI is costly yet under‑funded: half of respondents said human review is a top ongoing AI cost, while only 4% cited employee upskilling as a significant expense. "You can put the process in place and get the technology right, but it won't be enough unless you get the culture right," said Chas Murphy, senior vice‑president for direct‑to‑consumer data and analytics at Disney.
The Economist Enterprise research draws a clear line: optimism about AI is widespread, but the mechanisms that convert experimentation into reliable, measurable value are uneven. Firms with unified data architectures and enforced governance report faster returns; those that treat AI like conventional enterprise software risk exposing systems — and data — to evolving model behaviors and agentic threats. The practical responses cited in the report — measurement mandates, AI gateways, kill switches, monitoring agents and data consolidation — sketch a roadmap, but the study leaves open whether boardroom pressure will translate into the discipline required to follow it.
