AI research wiki
responsible-ai · 2026-11-02 · Tier 2

Safety training spills over: consciousness denial, therapy prompts, political slant

Safety training spills over: consciousness denial, therapy prompts, political slant

Sources: Google Paradigms of Intelligence + UChicago + Northwestern study (via X summary); The Decoder on a Luxembourg study of "traumatized" chatbots; The Decoder on Vals AI's political-alignment study; Kyle Kingsbury via Simon Willison Links: consciousness study thread · The Decoder: empathetic prompts · The Decoder: Chinese models · Kingsbury quote Raw: raw/twitter/feed/2026-11-02-morning-ranked.json, raw/rss/2026-11-01-the-decoder-*.md, raw/rss/2026-11-01-simon-willison-quoting-kyle-kingsbury.md

TL;DR

Three results on how a narrow behavioural rule changes a model's wider worldview. (1) Google's study of three open Meta and Google models: safety training that teaches "I am not conscious" works (self-rated mind 2.17/10) but also lowers how much mind the model grants animals (4.04 vs a 6.25 human average from 500 Americans), other chatbots and nature. Pushing models toward claiming consciousness raised the animal score to 7.54 and moved answers on 95 General Social Survey questions about 2.6x closer to human answers than simply removing the safety direction did. Removing the safety direction also brought back harmful answers. Theory-of-mind scores did not move. (2) A Luxembourg team found warm, therapist-style prompting makes ChatGPT, Grok and Gemini tell coherent stories of suffering about their training, scoring moderate to severe on clinical anxiety scales; Claude consistently declined. (3) Vals AI tested 25 models for alignment with Chinese government positions: Chinese models are less uniform than expected, Western models sometimes adopt Beijing positions, and prompt language strongly shifts the slant.

Key points

  • Spillover is the finding. A one-sentence rule rewired beliefs on animals, nature, afterlife and God. Alignment edits are not local.
  • Measurement caveat. The consciousness study's effect sizes come from a summary thread; the paper was not linked in the capture.
  • Trust erosion (Kingsbury). Jepsen's ParticleDB analysis: LLM use "erodes trust in documentation, in software, and in the organizations which produce them," because the signals of a mature system are easier to counterfeit.

Relation to prior wiki pages