Chain-of-thought prompting asks the model to think out loud, boosting accuracy on hard problems.
ConceptWhat it is
Chain-of-Thought (CoT) prompting instructs a model to generate intermediate reasoning steps before producing a final answer, rather than jumping straight to a conclusion. It exists because language models perform markedly better on multi-step math, logic, and planning tasks when they externalize reasoning rather than trying to compute the answer in one leap.
The technique can be triggered simply by adding "think step by step" or by providing worked examples that show the reasoning trace explicitly.
How it worksThe mechanics
The prompt asks the model to lay out its reasoning in numbered or sequential steps before stating a final answer; each generated token conditions on the prior reasoning steps, effectively giving the model more computation and intermediate state to work with than a direct answer would allow.
At a glanceSee it
Self-consistency samples many independent reasoning paths and votes for the majority answer, trading extra compute for robustness — unless a shared bias tilts every path the same wrong way.
The real question is not whether to reason but how — skip chain-of-thought on simple lookups, then pick zero-shot or few-shot elicitation by whether you have good worked examples on hand.
When to use itWhere it fits
- Multi-step math, logic, or planning problems.
- Tasks where showing reasoning improves auditability and trust.
- Debugging why a model reaches a particular conclusion.
- Complex classification requiring weighing multiple factors.
When NOT to use itLimits & anti-patterns
- Simple factual lookups where reasoning adds only latency and cost.
- Latency-sensitive applications needing instant single-token-style answers.
- Cases where exposing the reasoning trace to end users creates confusion or leaks internal logic.
Trade-offsAdvantages & costs
Advantages
- Significantly boosts accuracy on multi-step reasoning tasks.
- Makes model decisions more auditable and explainable.
- Cheap to trigger, often just a phrase change.
- Compatible with few-shot examples for even stronger results.
Trade-offs & costs
- Increases token usage and response latency.
- Reasoning traces can look convincing while still being wrong.
- Not always reliable for factual grounding without retrieval.
- Verbose output may need to be hidden from end users.
ExampleIn the real world
Khan Academy's Khanmigo tutor uses chain-of-thought prompting so the model walks through math problems step by step, matching how a human tutor would explain a solution.
ToolsHow to implement it
- OpenAI o-series modelsbuilt-in extended reasoning beyond manual CoT prompting.
- Anthropic Claude extended thinkingnative chain-of-thought reasoning mode.
- LangChainchains that separate reasoning steps from final output.
- Ragasevaluates whether reasoning chains actually support the final answer.
Cost & effortWhat it takes
Higher token and latency cost than direct answers, often two to five times more output tokens; worth it when accuracy on hard tasks matters most.