Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
Researchers find that large language models retain correct answers internally even when refusing to output them, challenging assumptions about refusal mechanisms.
- LLMs retain correct answers in hidden states even when refusing to output them.
- Refusal mechanisms operate as output-layer suppression rather than knowledge deletion.
- Bidirectional activation patching reveals causal asymmetries in refusal behavior.
- Study suggests current safety mechanisms may be less robust than previously assumed.
A new study published on arXiv examines how large language models (LLMs) handle refusal scenarios, where they decline to answer a prompt. Using controlled experiments with bidirectional activation patching, researchers discovered a fundamental asymmetry in how refusals are implemented. Even when an LLM generates a refusal, the correct answer remains linearly recoverable from its internal hidden states. This suggests that refusal is not a deletion of knowledge but rather a suppression at the output layer. The findings challenge existing assumptions about model interpretability and safety mechanisms, highlighting the need for deeper investigation into how LLMs manage conflicting internal representations.
Developers working on LLM safety and interpretability must reconsider refusal mechanisms.
Companies relying on refusal behaviors for safety or compliance need to reassess their approaches.
Investments in AI safety and interpretability tools may see renewed interest due to these findings.
The study raises questions about how LLMs handle conflicting information internally.
- bidirectional activation patching
- A technique used to analyze causal relationships in neural networks by swapping activations between matched trajectories.
- hidden states
- Intermediate representations of data within a neural network that encode information at various processing stages.
Cloud-Based Artificial Intelligence Classification of Common Intracranial Tumors on Magnetic Resonance Imaging - Cureus
Improving the matrix multiplication exponent with modern optimization and AlphaEvolve
AutoSR: Automatic Symbolic Regression by Searching Research States
Towards Computational Provenance: Carrying Causal-State Evidence in Generated Text
Proteus: Incremental Memory Activation for Long-Context Sequence Modeling
AI vs AI: Can artificial intelligence contain the fake news epidemic that it has helped unleash? - Genetic Literacy Project
Researchers explore whether AI can detect and mitigate fake news, a problem partly fueled by AI itself.
AI and the New Age of Bioweapons - Foreign Affairs
A Foreign Affairs analysis warns that AI could dramatically lower the barrier to creating bioweapons, accelerating proliferation risks.
Artificial Intelligence: Organizations Across the Americas Urge the IACHR to Address the Environmental and Social Impacts of Rapidly Expanding Data Centers - elciudadano.com
Organizations across the Americas have formally requested the Inter-American Commission on Human Rights (IACHR) to investigate the environmental and social consequences of rapidly expanding data centers, driven by artificial intelligence development.
Suburban man allegedly used AI to create child sexual abuse material: Prosecutors - NBC 5 Chicago
A suburban man is accused of using AI to create child sexual abuse material, according to prosecutors.
Appeals court flags AI-generated fake cases in San Antonio ISD lawsuit - KSAT
A federal appeals court in Texas flagged AI-generated fake cases in a lawsuit involving San Antonio ISD, raising concerns about the reliability of AI in legal filings.
BusinessAnthropic’s annualized revenue surges to $65B
Anthropic’s annualized revenue has skyrocketed to $65 billion, adding $18 billion in just two months.