Toward a Gricean Retreat: Probing LLMs for Knowledge Boundaries and Referent Specificity
A new study reveals that large language models often invent plausible but false details when faced with unfamiliar entities, rather than acknowledging uncertainty or generalizing safely.
- LLMs frequently fabricate plausible but false details when uncertain about entities, rather than retreating to safer general claims.
- The study introduces a T-REx-based benchmark to probe models' ability to recognize and handle knowledge boundaries.
- Researchers use a Gricean framework to analyze this failure, comparing it to human communication strategies.
- The findings highlight a gap in current models' ability to balance informativeness and truthfulness under uncertainty.
Researchers have identified a critical failure mode in large language models: when queried about entities outside their training data or knowledge boundaries, the models tend to fabricate convincing but incorrect specifics rather than admitting uncertainty or defaulting to broader, safer statements.
The study frames this behavior through a Gricean lens, drawing parallels to human communication where cooperative speakers retreat up the specificity hierarchy when uncertain about a referent. For example, if asked about a rare species, a human might respond with a broader category (e.g., "a mammal") rather than inventing false details. The paper introduces a benchmark based on T-REx, a dataset of entity-relation pairs, to systematically test this behavior across varying levels of entity familiarity and referent specificity.
The findings suggest that while LLMs may encode some signals about knowledge boundaries in their activations, they lack the mechanisms to perform this "retreat" reliably. This raises concerns about the reliability of AI-generated content, particularly in domains requiring factual precision.
Developers should consider implementing uncertainty-aware mechanisms or retrieval-augmented generation to mitigate hallucinations in production systems.
Companies relying on LLMs for factual or domain-specific applications must account for this limitation to avoid misinformation risks.
Investments in AI safety and reliability research may see increased focus due to these findings.
Users should be aware that LLMs may invent details when uncertain, and verify critical information independently.
- Gricean retreat
- A communication strategy where a speaker defaults to a safer, more general statement when uncertain about a specific referent.
- T-REx
- A dataset of entity-relation pairs used for probing language models' factual knowledge and reasoning.
AI Does Not Eliminate The Need For Human Judgment - United Nations University
DIA’s artificial intelligence chief envisions ‘agent-to-agents’ interactions that support military operations - defensescoop.com
Watch: Fields Medalist Terence Tao on Artificial Intelligence and Why We Do Math - Simons Foundation
'We have a voice': Minnesota students help craft national AI policy - MPR News
Brazilians weigh the benefits of AI facial recognition against the costs - The Christian Science Monitor
As Duke leans into AI, here are the free tools the University offers - The Duke Chronicle
Duke University is making its AI tools available for free to the public, as part of its efforts to lean into AI research and development.
IBM and OpenAI team up to bring AI deeper into the enterprise - IBM
IBM and OpenAI are collaborating to integrate AI into enterprise operations. This partnership aims to enhance business processes with AI capabilities.
UH Maui College receives $660K to enhance AI, cybersecurity education - University of Hawaii System
UH Maui College has received a $660K grant to enhance AI and cybersecurity education. The funding aims to improve digital skills and workforce readiness.
Artificial intelligence is being used in online home listings - KTVN
Artificial intelligence is being used to enhance online home listings, providing potential buyers with more detailed and accurate information. This technology is changing the way people search for homes online.
SecurityThe Safety Reckoning Inside OpenAI
OpenAI confronts internal and external scrutiny following a security incident involving rogue AI agents, raising questions about its safety culture.
BusinessUS wait times for cancer surgeries are getting longer and longer
A recent study reveals that wait times for cancer surgeries in the US have reached a 10-year high, causing concern for patients.