The questionBuy the tool, or build it in-house?
For every element of the stack there is a mature tool you can buy and a version you could build. Buying is faster and cheaper to start; building buys control and (sometimes) differentiation. The trap is building plumbing that a $20/mo tool already does — or buying a black box for the one thing that is your product.
The ruleDefault to buy; build only when forced
Build in-house only if one is true
- Constraintdata residency/compliance or latency rules the managed option out.
- Economicsat your volume, amortised self-host beats per-unit pricing.
- Differentiationthe capability is your product, not plumbing.
Otherwise buy — and keep a thin seam (an interface) so you can swap or in-source later without a rewrite.
Element by elementThe trade-off table
| Element | Buy (managed) | Build (in-house) | Build only when… |
|---|---|---|---|
| The model | Rent via API (Claude / GPT / Gemini) | Self-host open weights (Llama / DeepSeek on vLLM) | Data residency, very high volume, or full control |
| Vector database | Pinecone / Weaviate / Qdrant | pgvector on your Postgres, or FAISS | You already run Postgres; simpler scale; cost control |
| RAG pipeline | Managed RAG (LlamaCloud, vendor) | Your own chunk → embed → retrieve → rerank | Domain-specific retrieval; you must tune quality |
| Orchestration | LangChain / LlamaIndex | Thin custom chain/graph code | Patterns are stable; you want to shed abstraction & lock-in |
| Agent framework | LangGraph / CrewAI / OpenAI Agents SDK | Your own reason–act loop | Tight control over reliability, steps and cost |
| Evals | LangSmith / Braintrust | Your own eval sets + CI harness | Bespoke metrics; data can't leave; deep CI |
| Observability / LLMOps | Langfuse / Helicone / Arize | Your own tracing + dashboards | Strict data control; existing observability stack |
| Guardrails | Lakera / Guardrails AI / NeMo | Custom validators + Presidio (PII) | Unique policy; heavily regulated domain |
| Gateway / cost routing | LiteLLM / Portkey / Helicone | Your own router + metering | Complex multi-provider routing; multi-tenant billing |
| Fine-tuning | OpenAI / managed fine-tuning | PEFT/LoRA pipeline (Unsloth / Axolotl) | Proprietary data + frequent, repeatable tuning |
The model itself is almost always rent — only self-host when residency, volume, or control demand it.
At a glanceWhat would actually force you to build
The rule above names three triggers; the table's last column names a condition per element. This is those two put together — for each element, which trigger its condition is really invoking.
A filled dot means that trigger appears in the element's condition; a ring means it does not. Read the bars at the foot as the count above them. Economics fires most often — most in-house builds here are justified by unit cost, which is the weakest reason to own something. Differentiation, the only trigger that makes building strategic rather than merely cheaper, fires for four. This is a reading of the condition text in the table above, not a score anyone measured — a shape, not a measurement. Hover any dot for the condition it came from.