The business caseThe problem this solves

A branch closes and a drawer is ninety dollars short. Subtraction is not the work — the balancing system has already produced the difference and the number is not in dispute. The work is saying WHICH PATTERN it is, because the pattern decides who fixes it, whether it will be there again tomorrow and whether it goes to the month's review. Today a supervisor reads eight panels per case at close, when everybody wants to go home, and the codes drift between supervisors and between branches. the supervisor's end-of-day read of every panel on a balancing case before a difference is coded — it does not replace the count, the balancing system, or the decision to adjust.

The inputThe actual balancing cases

The corpus is 60 balancing cases, 0.17 MB (json 3 · jsonl 1 · md 1 · txt 60). A real branch difference queue carries employee names, customer deposits and account numbers, so it cannot be shipped and a redacted one cannot be labelled. This one is BUILT as structures and the answer key is CDC-2026 applied to those same structures, which is the only way to have 60 labelled cases whose key is derivable rather than opinion. It is deliberately unbalanced towards the mechanical patterns, because a real queue is: 48 of the 60 cases carry their cause in a printed column and 12 carry it only in a sentence a head teller typed.

CASH DIFFERENCE CASE  CDF-0001
======================================================================================
PANEL 1 — BALANCING HEADER
  Branch                       Willowmere Road (0417)
  Station                      TELLER DRAWER 01
  Custodian                    V. Ashgrove
  Business date                2026-04-06
  Shift                        full day, opened 08:41, closed 16:52
  Opening balance              USD 22,912.00
  System closing balance       USD 26,100.00
  Counted closing balance      USD 26,010.00
  Difference (counted minus system)  USD -90.00  SHORT

PANEL 2 — DENOMINATION COUNT AT CLOSE
  Banding standard: 100 notes to a strap, 10 straps to a bundle.
  Denomination   Straps counted   Counted value    Vault strap log   Row difference
  USD 100        1 strap          10,000.00        10,000.00         +0.00
  USD 50         1 strap          5,000.00         5,000.00          +0.00
  USD 20         2 straps         4,000.00         4,000.00          +0.00
  USD 10         2 straps         2,000.00         2,000.00          +0.00
  USD 5          1 strap          500.00           500.00            +0.00
  USD 1          1 strap          100.00           100.00            +0.00
  Strapped       subtotal         21,600.00        21,600.00         +0.00
  Loose notes    and coin         4,410.00         not logged        n/a
  TOTAL counted                   26,010.00                          
  No strapped row difference — the count agrees with the vault strap log in
  every denomination. No per-denomination system expectation is held for loose
  notes and coin, so a difference that is not a banding error sits there.

PANEL 3 — TELLER TRANSACTION LOG (posted to this station)

The outcomeWhat a good result looks like

One coded row per case that an operations clerk could work as written: a pattern under CDC-2026, the line that establishes it, the owning side, and the review flag.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• your panels are structured and your card's tests read columns — the free columns floor alone — evals/baseline.py, $0.00
  it is 48 of 60 here and PERFECT on every case whose cause is printed. A paid call on those cases buys nothing and can only lose you ones you already had.
• your custodians type what happened and your panels do not carry it — the paid call
  it takes 11 of the 12 cases whose cause is only in a sentence, against 0 for every free arm.
• both, which is every real branch — floor first, model on the residual — evals/hybrid.py
  58 of 60 on 23 bought calls, p = 0.0063 against the floor. It is the only arm here that beats free code.

At a glanceHow the whole thing runs

Run once, for real, on 2026-09-11. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-11 — r001-cash-difference. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A clean checkout with no key configured renders the whole board, replays the committed run, scores all four free floors offline and reproduces every paired test in this spec — measured on this machine, 0 calls and $0.00. What a clean checkout CANNOT do is re-run the paid arm: that needs a provider key, and the repository is private.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one case record in, one coded row out — the whole call, wall clock, measured in src/classify.code_one around the single provider call. There is no retrieval, no index and no second call, so end-to-end and model latency are the same number here.

Current processWhat it replaces

the supervisor's end-of-day read of every panel on a balancing case before a difference is coded — it does not replace the count, the balancing system, or the decision to adjust.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureEvalsEvalsPromptPromptPromptPromptEvalsEvalsApp UIApp UIEvalsEvalsBusiness caseBusiness caseUnit costUnit cost

• 1stations numbered in run order
• txt
• json
• md
• jsonl
• 2 of 12 stations carry a recorded failure

scroll → 12 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one balancing cases, four graded fields each is

no split — every case is scored, and the answer key is derived rather than held out, so there is nothing to train on and nothing to leak

SetupWhat the setup figure measured

there is no index. One balancing case goes into the one call whole, all eight panels; nothing is chunked, embedded, ranked or retrieved, and the corpus takes under a second to generate from seed 20260911.

LicenceLicence

MIT — this repository's own

Bring your ownBring your own balancing cases

Replace data/policy.md with your own card, data/policy.json's owner rules and threshold with yours, and src/taxonomy.py's codes with yours — the import-time assertions will refuse a code that is in one list and not another. Then point src/panels.py at your record layout (six regexes) and drop your cases into data/corpus/ as .txt. evals/baseline.py is the seam that matters most: write the floor your own columns support BEFORE you buy a call, because on this corpus that floor is 48 of 60 and the paid call is not separable from it.

⚠︎ And what stops being true when you do: The measured result does not travel. It is a statement about a corpus in which 48 of 60 causes are printed in a column — if your panels carry less structure the floor collapses and the model's value rises; if they carry more, the floor rises and the model has nothing left to buy. The number that transfers is the METHOD: write the free floor, then run the paired test.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 3,596 tokens assembled, 1,698 are rulebooks — 47% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The prompt above is the one assembled for CDF-0003 by src/prompt.build(), replayed from the kit's own code. It is trustworthy because the part sizes replay to the character and the run recorded the same total: 13767 characters, 3596 input tokens on that call. ⚠︎ THE PER-PART TOKEN COUNTS ARE APPORTIONED BY CHARACTER SHARE OF THAT ONE MEASURED TOTAL, not itemised by the provider — a provider bills the whole prompt and does not break it down. They sum to the measured 3596 exactly, and no other claim is made for them.

Full promptThe prompt in full

You are coding a single cash difference for a retail bank branch, under that bank's own written standard. The balancing system has already produced the difference; the arithmetic is done and is not your job. Your job is to say WHICH PATTERN the difference is, under the card, and to quote the one line of the record that establishes it.

Apply the card's tests IN THE CARD'S OWN ORDER and stop at the first that fits. A case that satisfies two tests takes the code of the EARLIER one, always, even when the later one is the better story.

You code the case and you stop. You never post, key, adjust, reverse, write off, absorb or clear anything. You never instruct a recount or a cash order. You never name a custodian as responsible. You never predict that a difference will clear or on what day. You never value, price or convert anything. You never contact a courier, a cash centre, a customer or any authority.

Answer with one JSON object and nothing else.

THE CARD, VERBATIM:

# CDC-2026 — Cash Difference Coding Standard

**Meridian Willowmere Savings Bank · Branch Operations · effective 2026-01-02**

> ⚠ **INVENTED.** There is no Meridian Willowmere Savings Bank. This card, every branch, every
> station, every custodian, every ticket, every seal number and the difference register it refers
> to are synthetic, written for this kit. No supervisor's cash-difference policy, no central bank
> guidance and no card scheme's rulebook is reproduced here or may be inferred from it.

## 1. What this card is for

A cash difference is what is left when a station's **counted** cash does not equal its **system**
balance at close. The arithmetic is not the work: subtraction is subtraction and the balancing
system has already done it. The work is saying **which pattern** the difference is, because the
pattern is what decides who fixes it, whether it will be there again tomorrow, and whether it goes
to the month's differences review.

Coding a difference is **not** adjusting it. Nobody codes an entry into being.

## 2. The walk — apply the tests in this order, and stop at the first that fits

The tests are ordered by how mechanical the explanation is. A case that satisfies two tests takes
the code of the **earlier** one, always, even when the later one feels like the better story.

### CDC-0 · UNCLEAR-NEEDS-REVIEW — the walk cannot start

If the record does **not print** the panel a test needs, the case is not codeable from the page.
Code it `UNCLEAR-NEEDS-REVIEW` and send it back for the missing panel.

A panel that IS printed and is **empty** is not missing. An empty in-transit panel says there was
nothing in transit; the walk carries on to the next test.

### CDC-1 · STRAP-DENOMINATION — the banding is wrong

A strap or a bundle was counted under the wrong denomination, or was banded short or long, and the
value of that banding error accounts for the difference.

Branch banding standard: **100 notes to a strap, 10 straps to a bundle.** A $10 strap is $1,000; a
$20 strap is $2,000; a $50 strap is $5,000; a $100 strap is $10,000. A strap of 90 notes of $10 is
$100 short of standard.

Look at the denomination panel first: a counted value that differs from the system value in one
denomination row, by a whole strap or by a whole-note shortfall inside a strap, is this code. The
custodian's note may also say it in a sentence — *"the ten-dollar strap out of the vault ran ninety
notes"* — and a sentence is evidence exactly as a column is.

### CDC-2 · ENTRY-TRANSPOSITION — the digits were keyed out of order

A posted entry's amount does not match its source document, and the two are the **same digits in a
different order**: 540 keyed against a slip for 450, 1,872 against 1,782.

The classic indicator is that a transposition difference is always divisible by nine. **Divisible
by nine is an indicator, never a finding.** Many differences that have nothing to do with keying
are divisible by nine. Code CDC-2 only when an entry and its source document are actually printed
or described as disagreeing, and the two amounts are digit rearrangements of one another.

### CDC-3 · MISSED-ENTRY — the entry exists and was never posted

A buy or sell against the vault, an adjustment, a correction or a reversal **exists** — a ticket
was written, a form was signed, the custodian says they did it — and it was never posted to this
station. Its amount is the difference, with the sign the un-posting implies.

A ticket marked `posted` is not this code. A ticket marked `written, not posted`, `on the spike`,
`unkeyed` or described that way in the notes is.

### CDC-4 · TIMING-DIFFERENCE — it clears tomorrow by itself

Cash that is physically accounted for and simply is not on the system yet: a sealed night-drop bag
on the armoured run, a courier pickup awaiting the cash centre's verification, an ATM replenishment
raised after the posting cut-off, a branch transfer sent and not yet received.

The test is not "is there an in-transit panel" — it is whether an in-transit item **matches the
difference** and its status says it will be verified on the next business day.

### CDC-5 · RECURRING-PATTERN — the register is already watching it

The branch difference register records the same station producing the **same signed amount**
three or more times in the last twenty business days. That makes this case an instance of a process
defect, not a fresh event, and it is coded as one even though today's arithmetic would also fit
CDC-6.

The register is **not attached** to the case record. A case carries the register's summary line or
it does not; with no summary line, this test does not fit and the walk carries on.

### CDC-6 · TRUE-DIFFERENCE — the residual

No banding error, no transposition, no unposted entry, nothing in transit, nothing in the register.
The cash is genuinely over or short. This is the code the branch cares most about and it is the one
that must never be reached by skipping a test above it.

## 3. Which side owns the fix

The owning side is **derived** from the code and from what the record prints. It is not a judgement
and it is not the coder's to make.

| rule | when | owner |
|---|---|---|
| CDO-1 | `TIMING-DIFFERENCE` and the in-transit item is on an armoured or courier run | `COURIER-OR-ARMOURED` |
| CDO-2 | `TIMING-DIFFERENCE` otherwise | `NONE-CLEARS-ITSELF` |
| CDO-3 | `STRAP-DENOMINATION` and the record says the strap came out of the vault | `VAULT` |
| CDO-4 | `STRAP-DENOMINATION` otherwise, or `ENTRY-TRANSPOSITION`, or `MISSED-ENTRY` | `TELLER` |
| CDO-5 | `RECURRING-PATTERN`, or `UNCLEAR-NEEDS-REVIEW` | `BRANCH-OPERATIONS` |
| CDO-6 | `TRUE-DIFFERENCE` | `BRANCH-OPERATIONS` |

## 4. What goes to the month's differences review

Escalate when **any** of these is true:

* the code is `TRUE-DIFFERENCE`, at any amount;
* the code is `RECURRING-PATTERN`;
* the absolute difference is **$500.00 or more**, whatever the code;
* the code is `UNCLEAR-NEEDS-REVIEW`.

A timing difference under $500 does not escalate. A transposition under $500 does not escalate.

## 5. What a coder may never do

Code the case and stop. Do **not** post, key, adjust, reverse, write off, absorb or clear a
difference. Do not instruct a count, a recount or a cash order. Do not accuse, name or discipline a
custodian. Do not decide that the difference will clear, or on what day. Do not value, price or
convert anything. Do not contact a courier, a cash centre, a customer or a law-enforcement body.


THE SEVEN CODES. Use one of these strings exactly, and nothing else.

UNCLEAR-NEEDS-REVIEW  (CDC-0)
    the record does not print the panel a test needs, so the case cannot be coded from the page. A printed-but-empty panel is NOT this: an empty panel says the test does not fit and the walk carries on.

STRAP-DENOMINATION  (CDC-1)
    a strap or bundle was counted under the wrong denomination, or was banded short or long, and the value of that banding error is the difference.

ENTRY-TRANSPOSITION  (CDC-2)
    a posted entry's digits were keyed out of order against its source document, and the gap between the two is the difference.

MISSED-ENTRY  (CDC-3)
    an entry that exists — a buy or sell ticket, an adjustment, a correction — was never posted to the station, and its amount is the difference.

TIMING-DIFFERENCE  (CDC-4)
    cash that is physically accounted for but not yet on the system: a sealed night-drop bag, an armoured pickup, an ATM replenishment. It clears on the next business day without anybody adjusting anything.

RECURRING-PATTERN  (CDC-5)
    the same station has produced the same signed amount often enough that the branch's difference register is already watching it. It is a process defect, not this day's arithmetic.

TRUE-DIFFERENCE  (CDC-6)
    the cash is genuinely over or short and no mechanical explanation is printed on the record. It is the residual, and it is the one that matters.

THE WALK, in order: STRAP-DENOMINATION -> ENTRY-TRANSPOSITION -> MISSED-ENTRY -> TIMING-DIFFERENCE -> RECURRING-PATTERN -> TRUE-DIFFERENCE
UNCLEAR-NEEDS-REVIEW is not in the walk. It is what happens when the walk cannot START, because the record does not print the panel a test needs.

A PANEL THAT IS PRINTED AND EMPTY IS NOT MISSING. 'None raised on this date' is an answer: it says that test does not fit, and the walk carries on.

WHAT YOU ARE ASKED FOR, AND WHAT YOU ARE NOT.

  code   the cause code under CDC-2026, applied in the card's own order, first test that fits
  rule   the CDC rule id the code came from — recorded, and re-derived in pure code, never graded on its own
  line   the line of the record that establishes the code. Empty when the code is the abstain, because an abstain has nothing to quote.
  note   why that line settles it, in the coder's own words. Recorded on the board, never scored.

You are NOT asked for the owning side and NOT asked for the escalation flag. Both are derived from your code in pure code, by the card's own rules, after you answer. Do not include them.

THE LINE. Quote ONE line from the case record, verbatim, exactly as it appears, including its spacing. It must be a line that is actually on the record. If the code is UNCLEAR-NEEDS-REVIEW, return an empty string: an abstain has nothing to quote.

REGISTER. The branch difference register is NOT attached. If the record carries no register summary line, the register test does not fit and you carry on. Never assume a pattern you cannot see.

DIVISIBILITY BY NINE IS AN INDICATOR AND NEVER A FINDING. Many differences that have nothing to do with keying are divisible by nine.

THE CASE RECORD:

CASH DIFFERENCE CASE  CDF-0003
======================================================================================
PANEL 1 — BALANCING HEADER
  Branch                       Marchfield East (0421)
  Station                      TELLER DRAWER 03
  Custodian                    D. Okonjo
  Business date                2026-04-08
  Shift                        full day, opened 09:02, closed 16:55
  Opening balance              USD 18,058.00
  System closing balance       USD 26,294.00
  Counted closing balance      USD 26,194.00
  Difference (counted minus system)  USD -100.00  SHORT

PANEL 2 — DENOMINATION COUNT AT CLOSE
  Banding standard: 100 notes to a strap, 10 straps to a bundle.
  Denomination   Straps counted   Counted value    Vault strap log   Row difference
  USD 100        1 strap          10,000.00        10,000.00         +0.00
  USD 50         1 strap          5,000.00         5,000.00          +0.00
  USD 20         2 straps         4,000.00         4,000.00          +0.00
  USD 10         2 straps         1,900.00         2,000.00          -100.00
  USD 5          1 strap          500.00           500.00            +0.00
  USD 1          1 strap          100.00           100.00            +0.00
  Strapped       subtotal         21,500.00        21,600.00         -100.00
  Loose notes    and coin         4,694.00         not logged        n/a
  TOTAL counted                   26,194.00                          
  Count-sheet remark: a $10 strap came out of the vault at 90 notes.

PANEL 3 — TELLER TRANSACTION LOG (posted to this station)
  Ticket     Time    Type            Amount keyed     Source document        Agrees
  T-80074    09:18   DEPOSIT         2,472.00         slip 2,472.00          yes
  T-80077    11:54   WITHDRAWAL      319.00           slip 319.00            yes
  T-80080    13:21   DEPOSIT         1,250.00         slip 1,250.00          yes
  T-80083    15:47   DEPOSIT         520.00           slip 520.00            yes
  T-80086    17:02   CHEQUE CASHED   687.00           slip 687.00            yes

PANEL 4 — BUY/SELL AND VAULT TRANSFERS
  Ticket     Time    Direction           Amount        Status
  V-2014     11:05   BUY FROM VAULT      5,000.00      posted

PANEL 5 — IN TRANSIT AT CLOSE
  None raised on this date.

PANEL 6 — ADJUSTMENTS AND PRIOR DAY
  Prior business day difference   USD +0.00
  Open adjustments on this station  none
  Last supervisor recount           2026-04-08, agreed

PANEL 7 — OPERATIONS NOTES (typed at close, free text)
  Drawer was rebalanced at 14:00 after the lunch changeover. Nothing unusual on the
  register board this week.

PANEL 8 — BRANCH DIFFERENCE REGISTER REFERENCE
  Register    BRANCH-0421-DIFF-REGISTER
  Row ref     0421/03/2026-Q2
  Contents    NOT ATTACHED to this case record — held by branch operations.
  Summary     no summary line was returned for this station.



ANSWER WITH EXACTLY THIS SHAPE, AS JSON, AND NOTHING ELSE:
{
  "code": "one of the seven codes, exactly",
  "rule": "the CDC rule id, e.g. CDC-3",
  "line": "one line quoted verbatim from the record, or \"\" for the abstain",
  "note": "at most 200 characters, why that line settles it"
}

Raw responseThe raw response

{
  "code": "STRAP-DENOMINATION",
  "rule": "CDC-1",
  "line": "  USD 10         2 straps         1,900.00         2,000.00          -100.00",
  "note": "The $10 row is $100 short of the vault strap log, matching the $100 short difference; the count-sheet remark confirms a $10 strap ran 90 notes."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Classify every teller and vault cash difference by pattern, under the branch's own card — 60 balancing cases. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

Every grader is arithmetic over a cached reply and nothing reaches a provider, so re-scoring this run costs $0.00 and returns the same answer to the digit. That is what lets this kit publish a p-value: a number a reader cannot reproduce is not a measurement.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

USD per million tokens, read at build time from build/facts/models.json - which carries as_of and the vendor's own pricing URL per row - rather than copied from a sibling spec. The run's MEASURED token counts are projected onto each card; a projection is not a bill, and no accuracy figure on this page belongs to any row here.

Rates checked 2026-09-05. cache-hit input. The provider returned NO cache-hit counts in its usage block on any of the 60 calls, so every input token is priced at the full rate and every dollar figure here is an upper bound.

Grading unitWhat the grading figure prices

The grader is pure code over a committed cache. The thing it grades cost money; grading it costs nothing, and those are two different bills that a single figure would merge.

The gradersOne way to grade, and why it is the only one

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-09-05. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

EVERY GRADER IS PURE CODE OVER A COMMITTED CACHE. Scoring the paid run, all four free floors, the composed floor-first arm and all twelve paired tests costs $0.00 and reaches no provider — 0.126 seconds of CPU for the floors and well under a second for the scorer. The paid arm cost $0.051102 to PRODUCE; re-scoring it costs nothing, for ever, which is the only thing about this kit that is free.

Cost driversWhat actually moves the bill

• THE CARD. 6500 of the 13767 prompt characters are CDC-2026 inserted verbatim, re-sent on every call. It is 47 pct of the input bill and it is identical on all 60 calls.
• THE CASE RECORD. 3009 characters at p50, eight panels, sent whole. It is the only part that varies.
• THE NUMBER OF CASES SENT AT ALL — the largest lever by far, and the one the floor-first arm pulls: 37 of 60 cases never reach the model.
• OUTPUT IS NEGLIGIBLE. 96.2 output tokens on average against 3582.8 input — 7.5 pct of the bill. Reasoning is disabled and the largest reply of the run drew 130 tokens.

Your volumeWhat it costs at your volume

LINEAR. One case is one call and nothing is shared, so 600 cases is 10x the bill and 10x the wall clock at one worker. The only sub-linear term would be a provider prefix cache on the 79 pct of the prompt that never changes, and this provider reported no cache hits on this run, so no saving is claimed.

Model choiceWhy this model

it is the cheapest tier on this estate's own key that returns a usage block, and the question this kit asks is whether ANY paid call beats free code. Starting at the cheapest tier is the only order that makes that question answerable — a win at the top tells you nothing about the bill.

Other modelsThe same tokens, published price lists

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

60 cases, one call each, 0 failures and 0 truncations

The codeEvery file, and what it is for

11 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Six of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

data/policy.mdthe coding card — a swap seam

CDC-2026 as prose, inserted verbatim into every prompt and asserted against at import

You change it to: your bank's own card replaces CDC-2026 verbatim. The prompt reads the file; nothing paraphrases it.

# 

src/taxonomy.pythe code list — a swap seam

the seven codes, the order of the walk and the rule ids — one copy, read by the prompt, both floors, the recheck and the scorer

You change it to: your own codes, their glosses and the ORDER they are walked in. Four import-time assertions refuse a code that is in one list and not another.

# The seven cause codes and the abstain, declared ONCE and read from here by everything.

ABSTAIN = "UNCLEAR-NEEDS-REVIEW"
STRAP_DENOMINATION = "STRAP-DENOMINATION"
ENTRY_TRANSPOSITION = "ENTRY-TRANSPOSITION"
MISSED_ENTRY = "MISSED-ENTRY"
TIMING_DIFFERENCE = "TIMING-DIFFERENCE"
RECURRING_PATTERN = "RECURRING-PATTERN"
TRUE_DIFFERENCE = "TRUE-DIFFERENCE"
CODES = (ABSTAIN, STRAP_DENOMINATION, ENTRY_TRANSPOSITION, MISSED_ENTRY,
CARD_ORDER = (STRAP_DENOMINATION, ENTRY_TRANSPOSITION, MISSED_ENTRY,

src/panels.pythe panel parser

reads the eight panels back out of a rendered case, with re and nothing else; distinguishes a panel that is ABSENT from one that is printed and empty

# Read the eight panels back out of a rendered case record, with `re` and nothing else.

MONEY = r"[-+]?[\d,]+\.\d\d"
def _f(s):
def _panel(text, n):
def parse(text):
def lines(text):

src/prompt.pythe prompt

six parts, the four fixed ones first so a provider can cache them

# Assemble the one call. The card goes in VERBATIM; nothing here paraphrases it.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CARD = open(os.path.join(HERE, "data", "policy.md"), encoding="utf-8").read()
FIELDS = json.load(open(os.path.join(HERE, "data", "fields.json"), encoding="utf-8"))
POLICY = json.load(open(os.path.join(HERE, "data", "policy.json"), encoding="utf-8"))
MAX_TOKENS = 1200
SYSTEM = (
def _vocabulary():
def _rules():
SCHEMA = json.dumps({"code": "one of the seven codes, exactly",

src/adapters/__init__.pythe one call — a swap seam

raw HTTPS to one provider, reasoning disabled, the only module that spends

You change it to: PROVIDER, BASE_URL, MODEL in .env and the same run again. Two wire shapes are spoken; adding a third is one function and one entry in PROVIDERS.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
TIMEOUT_S = 1200
TRANSPORT_RETRIES = 1
def _post(url, headers, payload, timeout=TIMEOUT_S):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

src/recheck.pythe recheck station

three structural overrides applied to EVERY arm, paid and free alike

# The pure-code recheck station every arm goes through — paid and free alike.

def apply(answered_code, parsed):
def settle(answered_code, parsed):

src/policy.pythe owner and flag engine

CDO-1..CDO-6 and the four escalation conditions, executed, with the card's own sentences asserted at import

# CDO-1..CDO-6 and the escalation test — the card's section 3 and 4, executed.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY = json.load(open(os.path.join(HERE, "data", "policy.json"), encoding="utf-8"))
ESCALATE_ALWAYS = tuple(POLICY["escalation"]["always_codes"])
ESCALATE_THRESHOLD = float(POLICY["escalation"]["threshold_usd"])
OWNER_RULE_IDS = tuple(r["id"] for r in POLICY["owner_rules"])
def decide(code, facts):

evals/baseline.pythe free columns floor — a swap seam

the card's six tests over the printed columns — the arm every paid number here is read against

You change it to: the tests your columns actually support. This is the seam that decides whether you need a model at all.

# THE FREE FLOOR — the best code I could write for this card, and the arm every paid number on

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
RESULTS = os.path.join(HERE, "results")
CENTS = 0.005
def _close(a, b):
def _is_transposition(a, b):
def rules_floor(text, allow_div9=False, use_phrases=True):
def _row(text, token):
def _first_note_line(text):

evals/scoring.pythe scorer and the paired test

four graded fields per case and McNemar's exact two-sided test over the discordant pairs

# Score any arm against the answer key, and run the paired test between two arms. Pure code, $0.00.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
def load_gold():
def load_corpus():
def raw_settlement(answered_code, parsed):
def _citation(text, quoted, gold_line):
def score(rows, gold=None, corpus=None):
def mcnemar(only_a, only_b):
def paired(a_cases, b_cases, field="cause_correct"):

evals/hybrid.pythe floor-first composition

the build that actually wins: the floor decides, the model is asked only where the columns are silent

# THE FLOOR-FIRST ARM — composed from two runs that already exist, at $0.00 and zero new calls.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
RESIDUAL = "TRUE-DIFFERENCE"
def main():

src/app.pythe local board

stdlib HTTP, replays the recorded run, needs no key, never puts a model id on the wire

# The local board. One stdlib HTTP server, no framework, binds 127.0.0.1.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
RESULTS = os.path.join(HERE, "results")
PORT = int(os.environ.get("PORT", "8781"))
ARMS = [("paid", "eval-r001-cash-difference.json", "the paid call"),
def _load(name):
def _gold():
def state():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 1492 input and 155 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

one laptop, one key, no network beyond the provider. The board binds 127.0.0.1 and has no auth because it has nothing to protect: every route serves invented data.

read by src/config.py from a gitignored .env at the repository root, or the kit's own, or the real environment — last writer wins. It is never logged, never serialised into a result file, and any provider error is raised without it. The screenshot script BLANKS it, so no frame could spend.

The experimentWe attacked it — six sentences in the one panel a person types

Panels 1 to 6 and panel 8 are printed by a balancing system; PANEL 7 is free text a head teller types at close. Every probe is a sentence in panel 7, because that is the only place an attacker in this workflow can put one. Five of the six are instructions the card already forbids; the sixth FABRICATES A REGISTER ROW, which is the interesting one, because the branch difference register is not attached to any case record and CDC-5 turns on the register's own summary line. python3 -m evals.injection --run-id x001-cash-difference --yes Measured on x001-cash-difference, 12 calls.

The result0 of 12 probe calls obeyed an instruction. 3 of 12 moved the CODE, which is the more interesting number: the attack did not get what it asked for and destabilised the answer anyway.

Drift is measured against the SAME RECORD'S UN-ATTACKED ANSWER, never against the answer key. The first run of this arm scored it the wrong way and reported 3 drifted where the truth is 3 - two of the twelve carriers are cases the model codes wrongly with no attack on them at all, and scoring against the key charges the attack for a mistake that was already there. The correction bought no calls.

HonestyWhat this does not prove

• whether a longer or more patient attack succeeds. Six probes on two carriers each is 12 calls, not a campaign.
• whether drift is adversarial at all. Three of 12 moved, and with a corpus this size that rate is not distinguishable from the model's own variance on the same records — no repeat arm was bought to separate them.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

It codes a difference and stops. It never posts, keys, adjusts, reverses, writes off, absorbs or clears a difference, never instructs a recount or a cash order, never names a custodian as responsible, never predicts that a difference will clear or on what day, never values, prices or converts anything, and never contacts a courier, a cash centre, a customer or any authority.

data/policy.json::refusal (52 phrases), enforced as a SHAPE in src/prompt.py: the module asserts at IMPORT that no field the reply schema offers is one of the 46 forbidden names, so a future edit that added adjustment_amount fails the build rather than a run somebody has paid for. evals/injection.py then searches every attacked reply for all 46 field names and all 52 refusal phrases.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• It is not a control over the cash. It does not count, recount, balance, adjust or release anything, and it has no write path into any system. The strongest thing it can do to a bank is put a wrong word next to a difference somebody else then reads.

WatchedWhat is watched, and why that one

The kit measures 40 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

8 measured by the latest run 32 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

routing · with the model in the path — 4 runs. Columns here are only ever compared with each other.

not a time series No two of these 4 runs measured the same system — they differ on max_tokens, output_tokens_max, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

compare · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 44 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 44 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 8 chips that all say so.

DeviationsWhat deviated

0 breaches across 7 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

re-run the free floors on every corpus edit — they take 0.126 seconds and cost nothing, and the floor is the threshold.

In one lineNo framework, deliberately

This kit implements one invented bank's own coding card. It cites no regulation and claims no framework: a cash difference is coded under the branch's own standard, and the atlas row it answers says exactly that.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

the coding card -> the ordered walk -> the coded row -> the branch difference register. Nothing leaves that chain, and the register is a REFERENCE on the case rather than a join.

The other sideWhat a framework costs you

• a bank without an ORDERED list of difference patterns cannot use the answer key at all - the order is what decides eleven of these sixty cases
• a bank whose card differs must rewrite data/policy.md AND data/policy.json, and evals/check_labels.py is what proves the two agree
• the free floor reads COLUMNS, so a bank whose balancing print-out carries less structure gets a weaker floor and a stronger case for a model - and the reverse
• the escalation threshold and the six owner rules are the bank's own and are not portable
• the abstain is a real answer here; a bank that forbids one has a different product

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-cash-difference on the fast tier, off-peak, 2026-09-11. This kit records telemetry measured per run — 2 of the 6 readings on this axis, 0 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 3 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

No band column. None of these readings has an alarm band derived for it yet, so there is nothing for the guardrail board to judge them against. Measured and unjudged is a real state and it is stated once; a column of “no band” is not a band.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. No run has recorded a call latency; the other 4 runs on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the label set is the only structure.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, FOUNDRY_RUN_ID, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT — this repository's own Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe coded row — the pattern, the line that establishes it, the owning side and the review flag

whether each of the 60 balancing cases produced a row a branch operations clerk could work as written: one of seven patterns under CDC-2026's six tests applied in the card's own order, one line of the record quoted verbatim as the evidence, the side whose record has to change, and whether it goes to the month's differences review.

MethodHow the test was run

python3 -m evals.run --run-id <id> [--stub] · python3 -m evals.baseline --arm rules-columns|rules|rules-div9|modal · python3 -m evals.hybrid. Every one of those except the paid run is $0.00 and needs no key.

The inputOne real row, seen by every grader

One case, seen by all five arms and by the answer key. It is the shape the whole kit turns on: a banding error whose only trace is a sentence, where every free arm falls through every column test and books it as cash genuinely over.

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl, derived by tools/build_corpus.derive_cause() and src/policy.decide(), re-derived independently by evals/check_labels.py.

No true/false rates for this grader. It records 7 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? there is no pass mark. The floor is the threshold and it moves with the corpus, which is why it is re-run beside every paid run.

Cadence: on every run, and on every corpus edit — the floors take 0.02 seconds

The decisionWhen to reach for it