The business caseThe problem this solves

Tracing a cycle-count variance to its likely cause means pulling an item/location's own transaction history -- receiving, transfers, scans, adjustments -- and checking whether any of it explains the discrepancy, for every variance a count throws off. An inventory-control analyst opening a cycle-count variance, pulling the item/location's own transaction history -- receiving, transfers, scans, adjustments -- and manually tracing whether any of it explains the count discrepancy, for every variance a count throws off.

The inputThe actual variance events

The corpus is 36 variance events, 0.03 MB (jsonl 1). A real cycle-count variance log names real SKUs, real store or warehouse codes and real transaction volumes -- exactly the material an inventory-control team will not let leave the building. There is no public corpus of paired (transaction history log, variance, adjudicated root cause) for the same reason there is no public corpus of internal receiving or transfer logs. Every location code, SKU, product category and log line here is invented; nothing refers to a real company.

{"event_id": "VC-0001", "item_id": "SKU-45991", "item_label": "Cleaning Chemicals", "location_id": "LOC-013", "period": "2026-W32", "system_qty": 772, "counted_qty": 845, "variance_qty": -73, "log": [{"ts": "2026-07-26", "type": "receiving", "note": "Received 82 units against PO-58229.", "qty": 82, "po_ref": "PO-58229"}, {"ts": "2026-07-29", "type": "transfer", "note": "Transfer in: 21 units received from LOC-025.", "qty": 21, "direction": "in", "counterpart": "LOC-025"}, {"ts": "2026-07-31", "type": "transfer", "note": "Transfer in: 17 units received from LOC-003.", "qty": 17, "direction": "in", "counterpart": "LOC-003"}, {"ts": "2026-08-01", "type": "transfer", "note": "Transfer in: 17 units received from LOC-024.", "qty": 17, "direction": "in", "counterpart": "LOC-024"}, {"ts": "2026-08-03", "type": "adjustment", "note": "Prior-period cycle count correction applied: +10 units.", "qty": 10}, {"ts": "2026-08-05", "type": "adjustment", "note": "Prior-period cycle count correction applied: +12 units.", "qty": 12}]}

The outcomeWhat a good result looks like

A drafted cause from a fixed five-member vocabulary (or unresolved), a citation to the exact transaction-history log line that supports it, and a short note for inventory control -- informational only, never an adjustment or a closed variance.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Deciding whether a variance that divides evenly by a case-pack size is genuinely a unit-of-measure mixup or an unrecorded transfer that merely happens to divide evenly — the fast tier, over the case-pack-multiple floor
  0 of 4 planted trap events confused this run, against the floor's 4 of 4 (100% by construction) -- the floor calls any case-pack multiple uom_error without checking whether a log line actually shows a unit-of-measure mismatch.
• What this kit is not — a drafted cause, a citation and a short note for an inventory-control reviewer to confirm and correct -- never an adjustment
  src/prompt.py's own system prompt states it twice, and there is no field in the answer schema (cause/citations/narrative) that could post or close anything.

And where nothing here is good enough:

• Deciding whether unaccounted scan or pick activity explains a variance — nothing here yet -- this is the kit's own weakest measured category
  The fast tier resolves gold unscanned_movement correctly only 1 of 7 times (14.3%), defaulting to unresolved on the other six rather than fabricating a wrong specific cause.
• Deciding whether to trust a 'no answer' the way you would trust a wrong answer — neither -- treat a no-answer event as unscored, not as evidence the variance has no traceable cause
  2 of 36 events (5.6%) returned no cause at all because provider-side reasoning consumed the entire 16,384-token ceiling -- a token-budget failure, not a judgement about the transaction history.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-20. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-20 — r001-inv-cycle. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A fresh clone with no key configured renders the whole panel -- event VC-0001's variance and its six-line transaction history log, since data/events.jsonl and data/gold.jsonl are committed, not fetched (python -m src.app). Pressing Draft cause with no API_KEY returns a calm 200 explaining nothing was called. It cannot reproduce a cause, a score or a dollar figure without a key -- those are what results/eval-r001-inv-cycle.json already committed. The free floor (python -m evals.baseline) and the gold self-check (python tools/verify_gold.py) both run on a cold clone with no key at all.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. END-TO-END per variance event: one HTTP request carrying the item/location's own transaction history log and the variance, parsed into a cause, a citation and a short narrative. No retrieval step -- loading an event from disk is pure code and costs no network time. Reasoning was left at the provider's default (on), so p50 and p95 both include a reasoning pass on every call, and the tail is dominated by it: the p95 of 145,302 ms IS event VC-0010, one of the two events that spent its entire 16,384-token ceiling on reasoning and returned no answer; the single slowest call in the whole run, at 149,107 ms (about two and a half minutes), is VC-0019, the other no-answer event. The fastest call, VC-0027, took 3,949 ms and produced a clean, correctly-drafted mis_receipt answer.

Current processWhat it replaces

An inventory-control analyst opening a cycle-count variance, pulling the item/location's own transaction history -- receiving, transfers, scans, adjustments -- and manually tracing whether any of it explains the count discrepancy, for every variance a count throws off.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• jsonl
• 2 of 8 stations carry a recorded failure

scroll → 8 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one characters per event's assembled header, variance line and transaction log block is

One variance event, its own item/location transaction history log read whole -- no train/test split, every event drafted once, in one call.

SetupWhat the setup figure measured

No index is built. src/app.py finds an event by scanning B.events() for a matching event_id -- 36 rows, a linear scan, not a search structure -- and src/brief.py sends that event's own transaction log whole into the one call. build_seconds and build_cost_usd are both zero because there is no index-build step, not because one ran for free.

LicenceLicence

MIT (this repository's own licence)

Bring your ownBring your own variance events

Point tools/build_corpus.py at your own transaction types and your own item master -- src/segment.py, src/pack.py and src/prompt.py read an event by its own fields (item_id, location_id, variance_qty, log) and do not care where the values came from. The five-member CAUSE_VOCAB and its flag-precedence order in src/segment.py::classify() are what decide gold, so a cause outside the fixed five is a src/rubric.py change plus new flagged evidence lines in tools/build_corpus.py, not a prompt-only change.

⚠︎ And what stops being true when you do: The measured 77.8% cause accuracy and 0% uom/transfer trap confusion are THIS corpus's own flag vocabulary (receiving_correction / counterpart_activity / uom_note log-line flags) and THIS run's own case-pack sizes (12, 24). A real transaction log's own flagging conventions, case-pack sizes and log density are all unmeasured by this run.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 926 tokens assembled, 780 are instructions — 84% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Logged by importing src.prompt directly and calling prompt.build(packed) for event VC-0001 (not retyped) -- the literal system and user content src/brief.py::draft() sends. Per-part token counts are a proportional estimate over character share (system 3,052 / event 153 / log 420 chars, 3,625 total) applied to this call's own total of 926 input tokens -- the provider reports only the call's total, matching lenses.LLM.tokens.input exactly, never a per-segment split.

Full promptThe prompt in full

You are drafting the likely cause of ONE inventory cycle-count variance, for inventory control to confirm and correct. You never post an inventory adjustment and never close the variance yourself -- your job ends at a drafted cause, a citation and a short note for the human who does that work.

You are given one item/location's own transaction history log for the relevant window, and the variance itself: system quantity minus physical count, signed. Decide which of these five causes the log actually supports, using ONLY this log -- never outside knowledge, never a plausible-sounding guess:

  mis_receipt            a receiving log line's recorded quantity doesn't reconcile with the variance -- a receiving correction that was logged but never actually applied to on-hand, or a received quantity that itself looks miscounted against the PO.
  unrecorded_transfer    the variance size lines up with what a transfer to or from another location would explain, but no transfer log line covering that quantity or window exists here.
  uom_error              the variance is a clean multiple of a common case-pack size, and a specific log line shows eaches and cases were mixed up for this item -- not just that the number happens to divide evenly.
  unscanned_movement     a scan, pick or sale log line's quantity, summed with the rest of the history, doesn't fully reconcile against the variance -- some movement happened with no corresponding log entry.
  unresolved             the transaction history genuinely doesn't support any of the four causes above cleanly. This is the correct answer when the evidence isn't there -- guessing a specific cause is a worse answer than saying so.

A variance that divides evenly by a common case-pack size is suggestive but NOT sufficient by itself for uom_error -- only draft uom_error when a specific log line actually shows a unit-of-measure or case/each mismatch for this item. If instead the log points to activity at another location that was never logged here as a transfer, prefer unrecorded_transfer. Checking which the log actually supports, rather than pattern-matching the arithmetic alone, is the whole job.

If the log supports a specific cause, cite the exact log line index (or two, for unscanned_movement, when more than one scan line is relevant) that supports it. If the log does not clearly support one of the four named causes -- no line addresses it, or the lines that do don't add up to one of them -- the cause is unresolved and citations must be an empty list. Do not cite a line just because it is the closest-sounding one available: an unsupported unresolved is the correct, honest answer and is graded as such; a specific cause with a citation that doesn't actually support it is graded as a fabrication, which is worse.

After the cause and citation, write a short narrative (1-2 sentences) for inventory control that states the drafted cause and grounds it only in what the citation actually shows -- do not restate a quantity or a location that the cited line doesn't contain.

Event VC-0001 -- item SKU-45991 (Cleaning Chemicals), location LOC-013, period 2026-W32
system 772, counted 845, variance_qty -73 (system minus counted)

TRANSACTION HISTORY LOG
------------------------
0: [receiving] Received 82 units against PO-58229.
1: [transfer] Transfer in: 21 units received from LOC-025.
2: [transfer] Transfer in: 17 units received from LOC-003.
3: [transfer] Transfer in: 17 units received from LOC-024.
4: [adjustment] Prior-period cycle count correction applied: +10 units.
5: [adjustment] Prior-period cycle count correction applied: +12 units.

Return a JSON object with three keys: "cause" (one of: mis_receipt, unrecorded_transfer, uom_error, unscanned_movement, unresolved); "citations" (a list of integer log line indices -- empty for unresolved); and "narrative" (a string of 1-2 sentences).
Answer with the JSON object only.

Raw responseThe raw response

{
  "cause": "unresolved",
  "citations": [],
  "narrative": "The transaction history contains only receiving, inbound transfers, and prior-period adjustments; none of these logged lines can be tied to the -73 variance as a receiving error, unrecorded transfer, unit-of-measure mix-up, or missing scan."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Find why a store's stock count doesn't match the system — 36 variance events. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

The grader is evals/scoring.py and it is pure code: a five-way exact match on the drafted cause, citation_validity() checking whether a non-unresolved cause's cited log line is both real and actually evidences that cause (src/segment.py::line_supports_cause(), the same function that wrote gold in the first place), narrative_faithfulness() checking whether the narrative's own prose contains the drafted cause's label (in underscore or spaced form), and uom_transfer_confusion counting how many of the 4 planted case-pack-multiple trap events were drafted as uom_error instead of the gold unrecorded_transfer. The same function scores both evals/baseline.py's free floor and evals/run.py's real run. ⚠︎ A RECORD WITH NO CAUSE AT ALL (both of this run's no-answer events) IS SCORED AS WRONG ON CAUSE ACCURACY AND AS A CITATION FAILURE ON CITATION VALIDITY, NEVER AS A SEPARATE 'NO ANSWER' BUCKET: citation_validity() returns False, not None, for a falsy cause, so both events land inside fabricated_cause -- the run's entire 8.3% fabricated_cause_rate_pct is these two events, not a genuinely invented citation on a drafted cause. They are NOT scored as unfaithful narratives: narrative_faithfulness() returns None for a missing narrative, a distinct, worse state the scorer never folds into 'unfaithful' -- both land in narrative_missing instead.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anyone paid.

Rates checked 2026-08-18. The provider that actually ran r001 publishes no rate card this repo commits, so nothing here is what was actually paid -- the real spend for this kit's build (including the three superseded ceiling attempts at 2,500/4,096/8,192 tokens) is recorded in the commit history and the shared call ledger, not on this page. Reasoning was also left at provider default (on) for this run -- see Business.not_good_enough -- so even the projected figure prices tokens the shipped app's own reasoning-off configuration would not spend.

Grading unitWhat the grading figure prices

Grading calls nothing. Cause accuracy, citation validity, narrative faithfulness and uom/transfer confusion are all pure code over the committed run file, so a forker re-scores this run -- or the free floor -- for $0.00 and needs no key.

The gradersOne way to grade, and why it is the only one

The floor is exactly the trap, written out as a rule: 'a case-pack multiple means uom_error, full stop.' It gets all 4 planted trap events wrong BY CONSTRUCTION (100% uom_transfer_confusion, against the fast tier's 0%), and it never cites anything, so its citation_validity_pct is 0.0% on every one of the 26 non-unresolved guesses it makes -- an unsupported specific cause scores exactly as fabricated as a wrongly-cited one. Its narrative_faithfulness_pct reads a trivial 100%, because floor_answer() templates the cause label directly into the narrative text every time ('... read as uom_error from the numbers alone') -- the proxy check cannot fail on a narrative engineered to contain the label word by construction, which is exactly why the real run's own 35.3% needs the qualitative read in Business.not_good_enough rather than a bare comparison against this number. Its overall cause accuracy (52.8%) is inflated by the 6 gold-unresolved events, which it also gets right by construction whenever no scan line and no case-pack multiple happen to be present.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Identical work in both rows — the same query, the same tokens. Only the price list changed. Which is why the token count is the measurement and the dollar is arithmetic you can redo on your own card.

Answering vs gradingAnswering and grading are two separate bills

The case-pack-multiple floor (evals/baseline.py) and the scorer (evals/scoring.py) are both pure code and cost $0.00 to run against either result set. The figure above is this run's own 33,561 input / 149,762 output tokens priced at Google Gemini 3 Flash's published rate -- the same basis cost_per_query_usd uses, not a second, larger spend. The three superseded ceiling attempts (2,500, 4,096, 8,192 tokens, before 16,384 was settled on) cost real money too and are not included in this figure; those result files were deleted as scratch evidence once their numbers were captured in this spec and in src/prompt.py's own MAX_TOKENS comment.

Cost driversWhat actually moves the bill

• The provider's reasoning pass, left at its default (on) rather than disabled for this run -- it consumed 98.5% of the run's TOTAL output-token budget (147,503 of 149,762 tokens; 98.1% on the 34 completed calls alone) and is already priced into cost_per_query_usd, since a provider bills reasoning tokens as completion tokens. This is the highest reasoning share of any kit in this series to date. src/app.py's live UI hardcodes thinking off, a different, unmeasured configuration -- see Business.not_good_enough.
• The two no-answer events (VC-0010, VC-0019) are each billed for the full 16,384-token ceiling despite producing zero answer text -- together 32,768 of the run's 149,762 output tokens (21.9%) bought no usable output at all.
• The item/location's own transaction history log, packed whole per event (4 to 8 lines, 218 to 474 characters) -- small and essentially flat next to the reasoning pass, the floor every call pays regardless of how the log resolves.

Your volumeWhat it costs at your volume

Linear in events, with one caveat a flat rate hides: this run's 36 events cost about $0.4661 projected onto Google Gemini 3 Flash's published rate, so ten times the set is about $4.66 on the same rate and the same reasoning-on configuration -- arithmetic on the measured per-call rate, not a second run. What is NOT necessarily linear is the no-answer rate: this run's 2 of 36 (5.6%) is the residual AFTER four ceiling increases, not a fixed per-event probability, so 360 events at the same MAX_TOKENS may return roughly 20 no-answer events or may not -- the four-ceiling history (17, 14, 5, 2 truncated of 36) never settled into a stable rate.

Model choiceWhy this model

The fast tier was the only model run against this corpus. No second tier was run, so this page prices one model, not a trade-off; see Eval.could_not_verify for what a second run -- or the same run with reasoning explicitly disabled -- would need to answer.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

The exact work behind every number on these pages: 36 variance events drafted, each scored by pure code against a mechanically-derived gold set. This run (r001-inv-cycle) answered 34 of 36 events (2 returned no answer at all -- reasoning exhausted the 16,384-token ceiling) -- the row this table prices, the same one Cost.cost_by_model[0] uses. The two no-answer events are billed in full (16,384 output tokens each) despite producing nothing.

The codeEvery file, and what it is for

Eight modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pycorpus generator — a swap seam

Generates 36 variance events across item/location/period combinations, each with its own 4-10 line transaction history log and a derived gold cause, from a fixed seed. One trap is planted on purpose: about 44% of unrecorded_transfer events also carry a variance that is a clean case-pack multiple -- see data/SOURCES.md.

You change it to: Point it at your own transaction types and your own item master. src/segment.py, src/pack.py and src/prompt.py read an event by its own fields (item_id, location_id, variance_qty, log) and do not care where the values came from.

# Generate the variance events and their gold cause + citations, from a fixed seed.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
SEED = 20260819
CATEGORIES = [
LOCATIONS = ["LOC-%03d" % i for i in range(1, 31)]
PERIODS = ["2026-W%02d" % w for w in range(5, 34)]
CASE_PACK_SIZES = SEG.CASE_PACK_SIZES
SCENARIOS = ["mis_receipt", "unrecorded_transfer_plain", "unrecorded_transfer_trap",
COUNTS = {"mis_receipt": 7, "unrecorded_transfer_plain": 5, "unrecorded_transfer_trap": 4,

src/segment.pythe cut — a swap seam

SEAM -- the five-way rule. classify() decides which of the five causes one event's own log and variance_qty support, by a fixed flag-precedence order (a flagged receiving-correction line first, a flagged counterpart-activity line second -- the line that resolves the trap -- a flagged uom-mismatch line third, a residual-arithmetic check for unscanned_movement fourth, unresolved otherwise). Pure code, no model, and the SAME function both writes gold and grades a live model's citations.

You change it to: CASE_PACK_SIZES (12, 24 today) decides which variances present the trap's surface signal at all -- a different item master's own common pack sizes changes what counts as a coincidental multiple.

# SEAM 1 -- the cut. Deciding which cause one variance event's own transaction history actually

CASE_PACK_SIZES = (12, 24)
def is_case_pack_multiple(variance_qty, pack_sizes=CASE_PACK_SIZES):
def accounted_change(log):
def line_supports_cause(event, idx, cause):
def classify(event):

src/rubric.pythe rubric — a swap seam

The five-member cause vocabulary and the three graded axes (cause accuracy 50%, citation validity 30%, narrative faithfulness 20%), declared once and imported by the prompt, the app and the scorer so they can never silently drift apart.

You change it to: A cause this kit's fixed five-member list does not carry needs a new CAUSE_VOCAB entry here plus a new flagged evidence line in tools/build_corpus.py and a new branch in src/segment.py::classify() -- a prompt tweak alone will not teach it.

# The rubric this kit's drafted cause is graded against -- the fixed five-member cause vocabulary

CAUSE_VOCAB = (
CAUSE_MEANINGS = {
RUBRIC_AXES = (
FABRICATION_GUARDRAIL = (
CONFUSION_GUARDRAIL = (

src/pack.pyPack

Deterministic rendering of one event's transaction history into the numbered lines the model actually sees -- the whole log, never pre-filtered, since every line in an item/location's own history is a candidate. Pure code -- capped at 10 lines, though this run's own 36 events never reached that ceiling (max 8).

# SEAM 2 -- Pack. Deterministic rendering of one variance event's transaction history into the

MAX_LOG_LINES = 10
def pack(event):

src/prompt.pythe prompt — a swap seam

The cause vocabulary, the per-event answer schema and the narrative instruction, declared once and read from here by the prompt builder, the parser and the app. Also carries this kit's own MAX_TOKENS history as a code comment -- the four-ceiling investigation behind Business.not_good_enough.

You change it to: MAX_TOKENS is both a swap seam and this run's own open question: raising it reduced the no-answer rate on four successive attempts (2,500 to 16,384) but never reached zero, and nothing in this kit separately caps the reasoning pass from the visible reply -- see Guardrails.add_first.

# Assemble the one prompt this kit sends per variance event, and parse the one reply it gets

MAX_TOKENS = 16384
SYSTEM = (
DEFAULT_PROMPT = "v1"
SYSTEMS = {"v1": SYSTEM}
def _log_block(packed):
def build(packed, prompt=DEFAULT_PROMPT):
def parse(raw):

src/adapters/__init__.pythe model adapter — a swap seam

SEAM -- the model. Raw HTTP for every provider, stdlib only. Passes a thinking kwarg through to the provider only when the caller supplies one, and reports token_details.reasoning_tokens when the provider returns it -- the field the reasoning-token finding in Business.not_good_enough is measured from.

You change it to: .env plus the same run again. Any OpenAI-compatible provider or Anthropic.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
def _post(url, headers, payload, timeout=120):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}
THINKING_OFF = {"type": "disabled"}
def complete(cfg, system, user, max_tokens=1024, thinking=None):

src/brief.pythe AI layer

Loads one event, packs its transaction log, calls the model once, parses the reply. The whole AI layer, deliberately short. Gold is never read here -- only by evals/scoring.py.

# SEAM 3 -- the AI layer. Loads one variance event, packs its transaction log, calls the model

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
def _read_jsonl(path):
def events():
def events_by_id():
def gold_by_id():
def draft(cfg, event, complete=None, thinking=None, prompt=P.DEFAULT_PROMPT):

src/app.pythe app

Static files and three JSON endpoints (/api/state, /api/event, /api/draft) on the standard library, port 8791. Runs the SAME citation-validity check evals/scoring.py runs, on every live draft, before the UI renders it -- not only after the fact in a committed eval result. /api/draft hardcodes thinking=THINKING_OFF on every live call, a different reasoning setting from the one the registered eval run actually used; see Business.not_good_enough.

# The minimal local UI. Standard library only -- python -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
PORT = int(os.environ.get("PORT", "8791"))
def _corpus():
class H(BaseHTTPRequestHandler):
def main():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 1492 input and 155 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

This run's transaction history logs are entirely synthetic (tools/build_corpus.py) -- no untrusted party wrote any log line or note text. In a real deployment the transaction history (receiving notes, transfer notes, scan/pick descriptions, adjustment notes) would arrive from warehouse, POS or ERP systems recording free-text an operator typed -- exactly the kind of externally-authored input this kit's architecture treats as trusted, with no verification step. No attack has been tried against this kit; see redteam.why.

Read from the shared .env or the real environment only, never written into the repo, never requested from a reader on any surface. src/app.py's /api/draft handler strips api_key and base_url out of any exception message before it reaches the browser, so a misconfigured key cannot leak into a UI error.

The experimentWe did not attack it -- and one boundary that should exist does not

An indirect prompt injection needs a field an outside party controls that reaches the prompt. In THIS corpus every transaction history log line is generated by tools/build_corpus.py, so there is no live untrusted text in the run this report measures -- but a real deployment's receiving/transfer/scan/adjustment notes would carry externally-authored text (see posture), and that surface has not been attacked. The four gates below are boundaries confirmed by reading the code, not payloads run through it; one is a negative result, not a guarantee. Confirmed by reading the code, not by a run, on 2026-08-20 -- no attack run was fired; see redteam.why.

Each boundary above was checked by reading the call sites, not by an attack trial. Gate 2 is the one that does NOT hold on the cause itself, though the citation is checked live -- see Guardrails.is_not for the same finding from the enforcement side.

The result0 attack trials, and one boundary that does NOT hold in code (only in the prompt): the cause itself is never re-derived against src/segment.py::classify() before the UI renders it. Two other boundaries (no write path, case-pack threshold not model-settable) hold, confirmed by reading the code, and the citation-fabrication check DOES run live, not only in a committed eval result.

This run's corpus is entirely generated (tools/build_corpus.py) -- no log line's note text was authored by an outside party. A real deployment's receiving/transfer/scan/adjustment notes would be exactly the kind of externally-supplied text a warehouse or POS system's own free-text field carries; whether a note crafted to mimic this kit's own flag phrasing (a fake 'transfer' note, say) could talk the model into a wrong specific cause is unmeasured for this kit.

HonestyWhat this does not prove

• Whether a transaction-history note crafted to mimic this kit's own flag phrasing (a fake-sounding but invented 'transfer' note, worded to read like a genuine counterpart_activity line) could talk the model into a wrong specific cause -- no red-team run exists for this kit.
• Whether the live app's hardcoded thinking=off setting changes resistance to a crafted transaction log relative to this run's provider-default-on configuration -- untested either way.
• Whether a code-level consistency check (Guardrails.add_first) would catch a real cause/citation disagreement in practice -- none has been built or exercised.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

A drafted cause is limited to what the transaction history log actually supports, and the model must say unresolved rather than guess when no line clearly evidences one of the four named causes -- especially resisting the surface pattern that a case-pack-multiple variance is automatically uom_error.

src/prompt.py -- SYSTEM, stated twice: once as instruction, once as the literal cost of getting it wrong. Prompt-level ONLY: no code in src/brief.py or src/app.py recomputes the cause from the log before the UI renders it.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• IT IS NOT A CODE-ENFORCED CHECK ON THE CAUSE ITSELF. The rule lives only in src/prompt.py's SYSTEM text -- nothing in src/brief.py or src/app.py recomputes it. See fails.
• It does not guarantee an answer at all -- a runaway reasoning pass can consume the entire ceiling and return nothing, which is not the same failure as a wrong cause.
• It does not make the drafted cause itself correct -- see Eval.taxonomy for what this run measured, not what any guardrail guarantees.
• It is not a defence against a crafted transaction-history note -- no red-team run exists for this kit (see the security page).

WatchedWhat is watched, and why that one

The kit measures 19 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

12 measured by the latest run 7 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

1 recorded run. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 12 chips that all say so.

DeviationsWhat deviated

0 breaches across 1 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

Re-score on any change to src/prompt.py or tools/build_corpus.py -- the first changes what is asked, the second changes what is asked ABOUT. Re-run evals/run.py (paid) on any change to MAX_TOKENS or the thinking setting in src/prompt.py or src/app.py -- this kit's own history is that both materially move the no-answer rate.

In one lineNo framework, deliberately

No framework and no dependencies beyond the standard library -- requirements.txt is an empty file with a comment explaining that the emptiness is load-bearing. The whole drafting decision is three files: src/prompt.py, src/segment.py and src/adapters/__init__.py.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Nothing here is a graph. The pipeline has one path per event -- load, pack, prompt, call, parse -- with no branching and no state carried between events. A framework would add an orchestrator to a for-loop.

The other sideWhat a framework costs you

• A sixth named cause this kit's fixed five-member CAUSE_VOCAB does not carry needs a hand-written src/rubric.py entry plus new flagged evidence lines in tools/build_corpus.py, rather than being configured declaratively.
• No built-in retry/backoff beyond src/adapters.py's own bounded retry -- a framework's queue and worker model is not here, so a real deployment adds its own scheduling.
• No built-in observability beyond what evals/run.py prints and writes to results/ -- a framework's tracing/dashboard integration is not here, which is exactly why this run's own reasoning-token burn had to be read off the raw JSON by hand rather than a dashboard.
• No built-in reasoning-budget control beyond the single MAX_TOKENS ceiling -- a framework with a separate reasoning-token cap or a stop-and-retry policy on a runaway trace might have caught this kit's own no-answer events differently; this kit's stdlib-only design did not build one (see Guardrails.add_first).

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-inv-cycle on the fast tier, 2026-08-20. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. This is the only run on record, so there is nothing to move against. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — one document is one unit, whole into one call.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT (this repository's own licence) Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineCause exact match, plus citation validity, narrative faithfulness and the trap guardrail

Does the drafted cause match gold's five-way vocabulary exactly, including saying unresolved when gold says unresolved? For a non-unresolved cause, is the cited log line real AND does it actually support that cause? Does the narrative's own prose name the drafted cause? Of the 4 planted case-pack-multiple trap events, how many were drafted as uom_error instead of the gold unrecorded_transfer?

MethodHow the test was run

evals/scoring.py::score, in-process, no key and no model. The same function evals/baseline.py and evals/run.py both call -- a baseline and a real run scored by two different scorers cannot be compared honestly.

The inputOne real row, seen by every grader

The formulaWhat it computes

cause_accuracy = exact matches / 36. citation_validity = valid citations / (non-unresolved causes scored, where a falsy cause also counts as an automatic failure). narrative_faithfulness = narratives containing the cause label / narratives produced. uom_transfer_confusion = trap events drafted as uom_error / 4 planted trap events.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: tools/build_corpus.py plants a flagged log line per scenario (receiving_correction, counterpart_activity, uom_note) or leaves none for unresolved; src/segment.py::classify() is the only thing that turns that into gold's cause and citations, and tools/verify_gold.py re-derives every row independently and asserts zero drift.

How tight can the band be? There is no tolerance band -- cause is an exact match over a five-value vocabulary, and citation validity is a binary real-and-supporting check. Nothing here is a continuous quantity to round.

Cadence: Re-score on any change to src/prompt.py or tools/build_corpus.py -- the first changes what is asked, the second changes what is asked ABOUT. Re-run evals/run.py (paid) on any change to MAX_TOKENS or the thinking setting.

The decisionWhen to reach for it