AI Snake Oil: resilience beats nonproliferation, but resilience requires "normal" governance

Source: Sayash Kapoor and Arvind Narayanan, "Do AI Risks Require Extraordinary Government Intervention?", normaltech.ai (cross-posted at the Knight First Amendment Institute), 2026-05-21. Link: Essay

TL;DR

Kapoor and Narayanan respond to a Derek Thompson essay arguing AI requires "extraordinary government intervention" akin to nuclear nonproliferation. They concede AI poses misuse risks, but argue nonproliferation regimes are brittle because they rely on a single chokepoint (in nuclear's case, enriched uranium). AI has no such chokepoint, since open-weight models are only months behind closed frontier. Any meaningful AI nonproliferation would require licensing regimes that escalate continuously. Their alternative: invest in societal resilience (bug bounties for non-tech sectors, AI-assisted red-teaming for schools/hospitals/grids, biosecurity screening of synthetic-bio orders, fuzzers as the cybersecurity precedent). Resilience requires polycentric governance and "normal" policymaking competence, which the US federal government has under-invested in.

The argument structure

Kapoor and Narayanan define "extraordinary intervention" as a spectrum with three factors:

Precautionary. Restricts activity based on anticipated harm, not realized harm.
Imposes restriction on actors not directly responsible. Restricts dual-use technology builders, not the misusers.
Bypasses normal governance via emergency declarations or executive orders.

The more of these factors, the higher the bar for justification.

Their critique of nonproliferation as the AI policy frame: there's no physical bottleneck. Adversaries (especially nation-states) match frontier capabilities within months. Open-weight models close the gap further. So nonproliferation buys "at most a few months," not years. To stretch beyond that requires escalating restrictions: licensing regimes, restrictions on open-weight model releases, restrictions on researcher movement across borders. This is what they call a "slippery slope."

Their alternative, resilience, is grounded in the cybersecurity precedent. Automated vulnerability detection (fuzzers, symbolic execution) has been superhuman for years. Defenders had access to the same tools, so the offense-defense balance held. The lesson: invest in defensive capability everywhere it's needed, including schools, hospitals, power grids, and small businesses, rather than chokepointing the supplier.

The bad news: resilience requires polycentric governance, many actors coordinating. The US federal government is not great at this. So extraordinary unilateral action looks tempting precisely because it's executive-order-tractable in a way that resilience investment is not. Kapoor and Narayanan worry that this asymmetry of effort will push policy toward the brittle option.

Why this matters

This is the most coherent articulation to date of the "AI as normal technology" alternative to the nonproliferation frame, applied directly to the May 2026 Beijing summit news cycle and the proposed Trump executive order. It connects two prior wiki threads:

The cyber-defense argument (see 2026-05-01 GPT-5.5 UK AISI cyber eval and today's Decoder piece on US Cyber Command deploying AI on top-secret networks): if defenders move first and stay current, the offense-defense balance is recoverable.
The AI safety political consolidation (AISN #73 today): both extraordinary-intervention and resilience camps are now visible in US policy. The Sanders / Trump bipartisan attention is uncommitted between the two framings.

The Kapoor/Narayanan paper is the strongest argument that the policy debate should not implicitly assume nonproliferation is the only serious option.

Falsifiable predictions in the piece

Open-weight models will stay within months of closed frontier (calibrated by past 12 months of releases; remains testable against next 12 months).
Anthropic's designation as a "supply chain risk" (cited as a concerning example) is a leading indicator of where extraordinary regimes drift if not redirected.
Synthetic biology orders, dangerous-materials tracking, and AI-assisted red-teaming for under-resourced sectors are the concrete near-term investments that would tell us whether the resilience approach is being taken seriously.

Industrial implication

If Cohere's Apache 2.0 Command A+ release is representative, the "open-weight months behind frontier" claim is now self-confirming. So the resilience camp's argument has empirical traction. The question, in policy terms, is whether the EO drafts adopt the language of resilience or the language of licensing.

Cross-references

Source

Raw: raw/rss/2026-05-21-ai-snake-oil-do-ai-risks-require-extraordinary-government-interventi.md.