"Citizens will be on their best behavior because we’re constantly recording and reporting," Larry Ellison said.
Automated enforcement as "speed cameras on steroids"
The essay warns that near‑future AI surveillance systems will do more than watch: they will notice, retain, tie infractions to official records, notify the person involved and alert authorities — and "maybe also the general public." The systems are cast as "automated speed cameras, but on steroids": able to enforce not just speed limits but "any other rule you can imagine," and to fine people immediately rather than sending a ticket in the mail.
China's scale and a public shaming example
China is presented as a real‑world demonstration of these techniques. The essay states the country "has over 600 million surveillance cameras, increasingly powered by AI and facial recognition to enforce legal and social rules." It cites the case of Lao Duan: after losing his job and failing to repay loans, he was blacklisted; when he visited Beijing the city's AI system identified him at a major intersection and displayed his face, name and citizen ID on a large electronic billboard with a message that he was an "untrustworthy person." The essay says similar systems are being deployed across China and integrated with online monitoring, censorship and social credit systems.
US Department of Homeland Security and global experimentation
The essay reports that AI surveillance is being experimented with worldwide — "in North America, South America, Europe, Asia and Africa." It states that "the US Department of Homeland Security is rapidly increasing its use of AI‑based surveillance, including facial recognition and the monitoring of social media accounts, to keep tabs on immigrants, dissidents, journalists, legal observers and protesters." While such systems are framed as maintaining security and public safety, the essay argues their "real aim is often social control."
Mechanisms of chill: Jon Penney's framework
Drawing on Jon Penney's new book Chilling Effects: Repression, Conformity, and Power in the Digital Age, the essay lays out how surveillance produces societal chilling. Penney identifies surveillance, personalization, uncertainty and authority as mechanisms that escalate chilling effects: people self‑censor, conform and become more compliant. The essay emphasizes that the fusion of persistent data collection, AI analytics and real‑time identification automates tasks that once required human analysts, producing a "supercharged societal level of chilling effects" in which fear, self‑censorship and groupthink displace dissent, creativity and innovation.
Technical and legal risks: bias, unauditable systems and loss of accountability
The essay lists concrete public‑policy challenges posed by AI surveillance: technical biases, unauditable systems, and "inflexible automated law and social rule enforcement" that can promote discrimination and undermine transparency, accountability and the rule of law. It contrasts modern AI‑enhanced surveillance with earlier, human‑centred domestic programs, calling the FBI's mid‑20th‑century wiretapping, mail opening, informants and paper index cards "genuinely archaic" by comparison — and even East Germany's cold‑war human surveillance less sweeping than what is now possible.
What this means for technologists, policymakers, and targeted populations
- Technologists and security teams: the essay implies a need to recognize that AI can "spy on the contents of our communications" and to confront the analytical power now automating surveillance tasks once done by humans.
- Policymakers and regulators: the essay urges concrete policy choices — bans on facial recognition and other identification tech, "robust new privacy and data protections," AI regulations to curtail invasive uses, and structural reforms to scrutinize and break up "powerful state/tech cartels."
- Targeted populations (immigrants, dissidents, journalists, legal observers, protesters): the essay highlights they are already subjects of expanding AI surveillance and warns that chilling effects will disproportionately limit risky experimentation, activism and social reinvention for disfavored groups.
The essay concludes with a stark, prescriptive note: the chill of AI‑powered mass surveillance "will suffocate the very foundations of healthy democratic societies," but it insists choices remain. Concrete policy moves — bans, privacy safeguards, AI limits and structural reform — are presented not as abstract options but as the specific levers the essay says can slow or prevent the implementation of mass AI surveillance.
Original essay (appeared in The Guardian; reposted at schneier.com)
