Hallucination
Quick Answer
A hallucination is when an AI model produces output that sounds plausible but is factually incorrect, fabricated or inconsistent with its sources. Hallucinations are a fundamental property of current generative models and the single biggest risk in enterprise deployments.
In Depth
What Hallucination really means
Hallucinations arise because language models are optimised to generate likely continuations, not truthful ones. They can invent citations, misremember numbers, blend similar facts, or assert incorrect causal relationships with high confidence.
Mitigations include retrieval augmentation, grounding prompts with authoritative sources, requesting citations, constrained output formats, and human-in-the-loop review for high-stakes outputs.
Why It Matters
Business relevance for UK organisations
For UK businesses in regulated sectors — financial services, healthcare, legal, insurance — unmanaged hallucinations can cause compliance breaches and reputational harm. Building guardrails into every customer-facing AI product is non-negotiable.
Real-world example
How this shows up in practice
A London wealth manager narrowly avoided sending a client an AI-drafted letter that cited a non-existent FCA regulation, after a human reviewer caught the fabricated reference.
Related Terms
Continue exploring
Large Language Model (LLM)
A Large Language Model (LLM) is a type of neural network trained on vast quantities of text to understand and generate human language. LLMs power chatbots, copilots, content generators and many modern AI features across consumer and business software.
TechnicalRAG (Retrieval Augmented Generation)
Retrieval Augmented Generation (RAG) is an architecture that combines a language model with an external knowledge source. Before generating an answer, the system retrieves relevant documents and feeds them to the model as context, dramatically reducing hallucinations and keeping answers current.
BusinessAI Governance
AI governance is the set of policies, roles, controls and oversight mechanisms that ensure AI is used responsibly, safely and in line with law and organisational values. Effective governance is proportionate — tight where risk is high, light where risk is low.
TechnicalPrompt Engineering
Prompt engineering is the practice of designing the text instructions given to a language model to produce reliable, accurate and appropriate outputs. Good prompts unlock significantly better performance without any change to the underlying model.