Failure Modes

At a glanceFailure Modes

Triage order. Every question splits the system in half before it splits again — the fastest bug is the one you localise, not the one you theorise about.

Why it mattersKnowing how it works is not knowing how it breaks

Everything else on this site teaches you to build. This page teaches you to debug — and that is the difference between someone who has demoed AI and someone who has shipped it. Production AI fails differently from production software. There is no stack trace. The system does not crash; it returns a fluent, confident, well-formatted, wrong answer, with a 200 status code, in 900ms, and your monitoring stays green.

So the discipline is different. You are not looking for an exception — you are looking for the stage that went wrong in a pipeline where every stage produces plausible output. This page is organised the way you will actually use it: symptom → likely cause → what to check first.

The methodBisect the pipeline before you theorise

A RAG or agent system is a chain: query → retrieve → assemble context → generate → post-process. Every debugging session is the same move — cut the chain in half and ask which side the failure is on. The single highest-value question in applied AI:

Teams routinely spend a week on the wrong half of that fork. The check costs 30 seconds: log the retrieved chunks, and read them yourself before you read the answer. If the answer is not in what you retrieved, nothing downstream matters.

The academic version of this fork: Barnett et al. engineered three production RAG systems and catalogued seven failure points, of which three are retrieval-side and four are generation-side. Their conclusion is the uncomfortable one — "validation of a RAG system is only feasible during operation," and robustness "evolves rather than [is] designed in at the start."

Symptom 1RAG returns garbage

The most common production complaint, and the most commonly misdiagnosed — because "the answer is wrong" is a symptom of at least seven distinct diseases. Barnett et al. name them; this is the map from their failure points to what you actually do about it.

Failure points from Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System (2024), drawn from research, education and biomedical case studies. The "half" and "first check" columns are the operational reading.

Chunk boundaries — where meaning gets amputated

Chunking is where most RAG quality is silently lost, because the failure is invisible at index time and only shows up as bad answers months later. A chunk is embedded as if it were a standalone document. It usually is not. Split a 10-K at a fixed 500 tokens and you get a chunk reading "revenue grew 3% over the prior period" — no company, no year, no segment. It is lexically fluent and semantically orphaned. It will never be retrieved for "How did ACME grow in Q2 2025?" because it does not contain "ACME", "grow" is a weak signal, and the embedding has nothing to anchor to.

Anthropic's Contextual Retrieval attacks exactly this: prepend a short, LLM-generated description situating each chunk in its parent document before embedding it. The measured effect on top-20 retrieval failure rate:

Source: Anthropic, Introducing Contextual Retrieval. Note the shape of the result — hybrid search (dense + BM25) and reranking are doing enormous work. If you are running pure dense retrieval with no reranker, that is the cheapest fix available to you, and you should do it before anything more exotic.

Embedding mismatch and the query/document asymmetry

Two failures that look identical from the outside and have completely different fixes.

Embedding mismatch is the mechanical one: the vectors in your index were produced by a different model — or a different version, or a different pooling/normalisation config — than the vectors you are querying with. The two live in incompatible spaces, so cosine similarity returns something, but it means nothing. The tell is diagnostic gold: results are not just bad, they are uncorrelated — nonsense with confident scores, and the same nonsense for every query. Ordinary bad retrieval degrades gracefully and stays topically adjacent; a space mismatch does not. Check the model ID and dimension stamped on the index against the one your query path loads. If your index has no such stamp, that is the bug — fix that first. Re-embedding the corpus is the only fix, which is why changing embedding model is a migration, not a config change.

Query/document asymmetry is the subtle one. Sentence-Transformers draws the distinction explicitly: symmetric search has query and corpus entries of similar length and kind (finding duplicate questions — you could swap them); asymmetric search has a short query hunting a long passage ("What is Python" → a paragraph). Their guidance is blunt — it is "critical that you choose the right model," and a model trained for symmetric similarity applied to asymmetric retrieval is miscalibrated for exactly the length and content mismatch your product is built on. Nearly every RAG system is asymmetric. A meaningful number are built on symmetric-trained embeddings because that was the default in the tutorial.

The asymmetry is real even with the right model: a question and its answer do not look alike. "Why did the deployment fail?" shares almost no surface with a log excerpt that explains it. This is what HyDE (Gao et al.) exploits — generate a hypothetical answer to the query, embed that, and search with it, so you are matching document-shaped text against documents. The encoder's "dense bottleneck filters out the incorrect details" of the invented answer while keeping its relevance structure. You are trading a model call for retrieval quality — the right trade when queries are short and vague, the wrong one when latency is the constraint.

Lost in the middle — and why more context is not more knowledge

The most expensive false intuition in RAG: the context window is 200K, so let us just put everything in it. Liu et al. showed models do not read the context window uniformly. Performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts" — and, critically, this holds even for models explicitly built for long context. Your top-ranked chunk sitting at position 12 of 20 is in the worst seat in the house.

It gets worse when you remove the crutch. Needle-in-a-haystack tests mostly pass because the needle and the question share literal words — the model can pattern-match without understanding. NoLiMa rebuilt the test with minimal lexical overlap, forcing genuine latent association. Across 13 models all claiming 128K+ context: strong below 1K, and at 32K, 11 of the 13 fell below 50% of their own short-context baseline. GPT-4o, one of the best performers, dropped from 99.3% to 69.7%. The authors attribute it to the attention mechanism struggling in long contexts when literal matches are absent — and note that reasoning models and CoT prompting do not rescue it.

Symptom 2Hallucination — a taxonomy, because the kinds have different fixes

"The model hallucinates" is not a bug report; it is a genre. Treating all fabrication as one problem is why teams throw RAG at it, watch it not go away, and conclude the model is bad. Ji et al.'s survey gives the load-bearing split:

Definitions quoted from Ji et al., Survey of Hallucination in Natural Language Generation. The distinction matters commercially: intrinsic hallucination is a solvable engineering problem, extrinsic hallucination is a product-design problem about what you are willing to claim.

The survey also separates the causes, which is where fixes actually come from:

And then the structural cause. Kalai et al. (OpenAI) argue models hallucinate because training and evaluation reward guessing over admitting uncertainty — like a student on an exam where blank scores zero and a guess might score one. Benchmarks "overwhelmingly penalize uncertainty," so a model that says "I don't know" loses to one that bluffs. Their proposed fix is socio-technical: penalise confident errors more than abstentions.

The operational consequence is immediate and under-appreciated. If your own eval marks "I don't know" as a failure, you are actively training your product to lie to your users. Score abstention separately from error. An honest "not in the documents" is a correct answer to an unanswerable question — and per FP1 above, your corpus is full of them.

Symptom 3Agents loop, stall, or confidently do the wrong thing

Agents fail in ways that look like bugs and are usually design. The best evidence base here is Cemri et al., who annotated 1,600+ execution traces across 7 multi-agent frameworks and derived MAST — 14 failure modes in 3 categories, at 0.88 inter-annotator agreement. The headline finding is the one worth internalising: failures cluster in system design, inter-agent misalignment, and task verification — that is, in specification and coordination, not in model intelligence. A smarter model does not fix a system whose agents disagree about what they are doing.

Symptom taxonomy informed by Cemri et al., Why Do Multi-Agent LLM Systems Fail? (MAST); the compounding-over-length mechanism connects to exposure bias (Ji et al.) and lost-in-the-middle (Liu et al.).

Symptom 4Quality dropped and nothing changed

The most disorienting failure, because your diff is empty. Something did change — just not in your repo. In rough order of how often it is the culprit:

The non-determinism one deserves its mechanism, because the folk explanation is wrong and the wrong explanation makes it unfixable. "GPU concurrency plus floating-point non-associativity" is not the cause: "even on a GPU, running the same matrix multiplication on the same data repeatedly will always provide bitwise equal results." The real cause is that inference kernels are not batch-invariant — their numerics shift with batch size — and batch size varies with server load, which is other people's traffic. So your output at temperature 0 depends on how busy the provider was. Two properties compose into the surprise: kernels lack batch invariance (mathematical) and load varies nondeterministically (operational).

Symptom 5The eval passes and users still complain

Your dashboard is green. Support is not. One of these is measuring your product; it is not the dashboard.

Symptom 6Cost exploded and traffic did not

Requests flat, bill up 4×. Cost per request is not a constant — it is an emergent property of behaviour you did not change directly. The suspects, in order:

Caching mechanics and multipliers from Anthropic's prompt-caching documentation; reasoning-token billing and the 25,000-token reservation guidance from OpenAI's reasoning guide. Both were accurate when fetched — re-check before you budget on them. The Costing page covers the token math these failures distort.

The playbookThe first fifteen minutes

When someone says "the AI is broken" — in order, no skipping.

ReferencesWhere these claims come from

Every external claim on this page is sourced. Where a number appears it came from the paper or the vendor doc, not from folklore.

Papers describe the failure; vendor docs describe the billing and the lifecycle. Both move. Re-check pricing, deprecation and caching pages quarterly — the mechanisms are stable, the numbers are not.