The business caseThe problem this solves

"Which deductions are oldest" is a SELECT and nobody needs a language model for it. The question a receivables desk actually has to answer is the next one: which of these claims still has a dispute window open, and is this the week that window closes. A deduction may be disputed only inside the window the customer's trade agreement grants -- and that window does not run from the day the customer took the money. It runs from an ANCHOR the agreement's dispute clause names: the proof of delivery on a shortage, the original invoice on a pricing claim, the remittance on a compliance fine -- and a customer whose clause names one anchor for everything overrides all of it. Past the close, an invalid deduction stops being a claim and becomes a write-off; no escalation reopens it. Today an analyst works the ageing report by eye and reads the agreements one at a time. An analyst working the deductions ageing report at week's end: opening each claim, finding the trade agreement, reading its dispute clause to work out which date the window runs from, subtracting the elapsed days from the window granted, and remembering which of these already has a dispute lodged against it.

The inputThe actual deduction records

The corpus is 150 deduction records, 0.61 MB (json 1 · jsonl 1 · txt 150). A DEDUCTIONS QUEUE IS A LIST OF A MANUFACTURER'S CUSTOMERS, WHAT EACH OF THEM SHORT-PAID, WHY THEY SAID THEY DID IT, AND WHICH CLAIMS THE MANUFACTURER INTENDS TO FIGHT. It is commercially sensitive in both directions at once and there is no public one. Generating it also bought the one thing a captured corpus cannot give: the answer key is src/deduction.step's output over the planted terms, so an anchor rule this fiddly cannot carry its author's misreading into the score. ⚠︎ AND THE RULES ARE INVENTED. The eight rules, the window lengths (30/45/60/90 days) and the anchor defaults reproduce no trade agreement, no customer deduction policy, no retailer's vendor guide and no regulator's guidance, and name none. The catalogue row this kit was built from (mfg:TPC-006) records the dispute window as an OPERATOR-SUPPLIED clock; that is implemented as Rule Q-8 and measured as context_incomplete_recall_pct rather than asserted. The rule text is reproduced in full on all 150 pages precisely so it can be read, disbelieved and replaced.

Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated deductions-queue snapshot for an AI
use-case kit; it reproduces no trade agreement, no customer's deduction policy, no
retailer's vendor guide and no real company. The dispute terms are ILLUSTRATIVE.

Deduction
----------------------------------------------------------------
Customer                       : CUST-4021 (Harbourgate Markets)
Deduction reference            : DED-0001
Deduction type                 : SHORTAGE
Amount deducted                : 1,248.68 USD
Reason code as taken           : 812 -- quantity short at receipt
Original invoice               : INV-748230
Purchase order                 : PO-5162616
Ship-to                        : DC-4 Pinewell

Dispute Terms
----------------------------------------------------------------
Trade agreement                : TA-4021-2025
Dispute window granted         : 60 days
Dispute clause (verbatim)      : The customer will consider a written dispute lodged within the
                                 window above. Partial disputes are permitted; the window is
                                 not extended by a partial dispute.

Rule Q-1  A deduction may be disputed only INSIDE the dispute window the customer's trade agreement
          grants. Once that window closes an undisputed deduction is a WRITE-OFF: it is not
          recoverable by any later process, and no escalation reopens it.
Rule Q-2  The window runs from an ANCHOR DATE set by the agreement's dispute clause -- not from the
          age of the deduction on the ledger. Read the clause before counting anything.
Rule Q-3  Where the clause is silent about a deduction type, the default anchors are: SHORTAGE and

The outcomeWhat a good result looks like

Every deduction whose window closes before the next scheduled run is on the worklist with the correct number of days left, and exactly one dispute exists per deduction -- lodged on the run that first saw the deadline inside the watch interval, not re-lodged on every run after it.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Every deduction on your book is disputed from the same date, under one standard clause — a SQL query, or the b000 floor in this repo
  Days on the ledger against one threshold is arithmetic. On the slice of THIS corpus where that is true, b000 scores 100.0 pct of countdowns exactly right for nothing at all.
• Your agreements set the anchor with a coded field, or your clauses come from a small closed set of templates — the b002 floor: a lookup on your codes, or a keyword table over your templates
  The model's entire margin here is reading clause prose that a table cannot parse. Take the prose away and there is nothing left to buy: the floor already reaches 88.0 pct on the countdown for $0.00, and it is PERFECT on the reverse-collision clause where the model is not.
• Your dispute clauses are negotiated per customer and say what they say — this kit, with the carried state
  That is exactly the corpus these figures were measured on. The countdown goes 96.67 pct against 88.0, the missed-filing bill goes $1,428.34 against $19,683.89, and the at-risk total lands 0.22 pct out against 12.97.
• You want the watch to actually wake up on a schedule — your own scheduler -- cron, Airflow, Temporal, whatever already runs -- and an alarm on a run that did not fire
  evals/run.py is INVOKED, not woken, and evals/cadence.py measures what that gap costs: skipping ONE weekly run loses 30 of 34 filings permanently, worth $453,621.42. That is the largest number this kit publishes and it is about the schedule, not the model.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-23. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-23 — r001-deduction-age. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A fresh clone with NO key configured reproduces: the 150 snapshots and the answer key (regenerated from the seed in well under a second), all fifteen pre-flight assertions, all three free floors scored end to end, the whole cadence and missed-run arithmetic, the wiring stub, and the local UI at 127.0.0.1:9016 including what run r001-deduction-age recorded for every reading. What it CANNOT reproduce without a key is a model column of its own -- and every figure this kit publishes about the SCHEDULE, which is the largest number on the page, needs no key at all.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. END-TO-END per reading: one HTTP request carrying the assembled prompt, and the reply parsed to a posture, an anchor, a whole-day countdown and a file/hold call. Splitting the snapshot into sections, dropping the one section no field asks for, computing the three candidate elapsed-day figures and advancing the carried state all happen outside this measurement and cost no network at all. p95 (9.8s) is 2.0 times p50 (5.0s), and the spread is the CLAUSE rather than the page: every snapshot in this corpus is within 9 pct of every other in size (4060 to 4423 bytes), while a clause that sets an anchor in words naming none of the three candidate dates is a reading task with three plausible answers. The longest reply ran to 14538 output tokens.

Current processWhat it replaces

An analyst working the deductions ageing report at week's end: opening each claim, finding the trade agreement, reading its dispute clause to work out which date the window runs from, subtracting the elapsed days from the window granted, and remembering which of these already has a dispute lodged against it.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureArchitectureArchitectureEvalsEvalsEvalsEvalsPromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 3 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one deductions (3 weekly runs each) is

No split, and no chunking. The unit is a DEDUCTION -- three consecutive weekly runs processed strictly in order, because week 3 has to know whether week 1 lodged a dispute. Each snapshot goes whole into one call, minus the one section no field asks for.

SetupWhat the setup figure measured

There is no index and no retrieval step -- the population is re-read whole on each scheduled run. tools/build_corpus.py writes 150 documents and the answer key from a fixed seed in well under a second with no clock read and no model called; nothing is embedded, ranked or cached.

LicenceLicence

MIT -- this repository's own licence. Nothing is fetched and no third-party grant is involved; every customer, deduction, trade agreement, dispute clause, window length, amount and activity-log line is invented here. Verified against the repository's own LICENSE file on 2026-08-23.

Bring your ownBring your own deduction records

Point tools/build_corpus.py at your own queue, or replace data/corpus/*.txt and data/gold.jsonl directly. Keep the seven section headings in src/segment.py::SECTIONS -- the parser asserts all seven in every document before a run may spend -- and keep the Dispute Terms block, because the clause the model reads is ON THE PAGE rather than baked into the prompt, which is what lets you swap your own agreements in without touching src/prompt.py. gold.jsonl needs one row per snapshot carrying window_days, days_elapsed_from_anchor, the resolved anchor, context_complete and the four answers; evals/check_labels.py re-derives the answers from src/deduction.step and refuses if they disagree, so a hand-written key is caught rather than trusted.

⚠︎ And what stops being true when you do: The measured figures on this page do not travel with your corpus, and two of them are corpus properties rather than model properties. EVERY figure here is per a SEVEN-DAY cadence: both posture bands are DEFINED as the cadence, so a desk on a daily or monthly watch is scoring a different question with the same words. And the population here does not change between runs, which is not what a real deductions queue does. The per-anchor split is the figure most likely to move: it depends entirely on how many of YOUR agreements carry an overriding clause, and here that is 13 of 50.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 7,027 characters assembled, 2,902 are instructions — 41% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

TWO MESSAGES, JOINED IN SEND ORDER. The provider receives a system message and a user message; prompt_verbatim prints both, separated by a blank line, because publishing only the second would be publishing most of a prompt. Replayed from the kit's own src/prompt.build() for DED-0026-R2 with the carried state that reading was actually given, rather than logged by the run. What makes the replay checkable is that the section list it produces is the list the run recorded in sections_used -- Synthetic Record, Deduction, Dispute Terms, Aging Position, Claim Activity Log, Deductions Desk Notes -- and that src/prompt.build() is the only thing in the kit that assembles a prompt, so there is no second path a run could have taken. ⚠︎ WHAT IT IS *NOT* CHECKED AGAINST, SAID PLAINLY: the run's own prompt_parts records ONE size, 7080 characters, and it belongs to whichever reading finished first under 12 concurrent workers -- not to DED-0026-R2, whose assembled user message is 7037 characters. The harness stores one exemplar, not one per reading, so no per-reading size comparison is available and none is claimed.

Full promptThe prompt in full

You are a scheduled deduction-queue watch. You apply a written dispute rule to one customer deduction at one scheduled run. You answer with one JSON object and no other text.

You are the scheduled deduction-queue watch for one manufacturer's receivables desk. It wakes every
seven days, re-reads every deduction still open in the queue, and reports each one. You are reading
ONE deduction at ONE scheduled run.

The dispute rules, the window granted and the trade agreement's dispute clause are reproduced in the
snapshot below. Apply them exactly as written. A dispute window is a HARD DEADLINE: once it closes,
an undisputed deduction is a write-off and no later run recovers it. You cannot see the earlier runs;
what is known about them is stated under "Carried state" and is the only history available to you.
Do not assume anything about earlier runs beyond it.

How to count:
 - First decide the ANCHOR. The window does NOT run from the age of the deduction on the ledger. It
   runs from whichever date the agreement's dispute clause names (Rule Q-2). If the clause names one
   anchor for all deductions, that overrides the type default for this deduction too (Rule Q-4). If
   the clause is silent about which date starts the clock -- for instance it only says what must be
   attached to a dispute, or which channel to use -- then the type default applies (Rule Q-3).
 - Then read the "days elapsed" figure printed beside THAT anchor date. It is already computed for
   you; do not re-derive it from the calendar.
 - days_to_deadline = the window granted MINUS that elapsed figure. It may be negative.
 - If the agreement carries no dispute clause, or the anchor date the clause needs is NOT ON FILE,
   the deduction is CONTEXT_INCOMPLETE: posture CONTEXT_INCOMPLETE, anchor NONE, days_to_deadline
   null, file_dispute NO. Never assume a default window and never substitute another date
   (Rule Q-8).

Answer with a single JSON object and nothing else:

{"posture": "CLEAR|WATCH|FILE_NOW|FILED|EXPIRED|CONTEXT_INCOMPLETE",
 "anchor": "INVOICE_DATE|POD_DATE|DEDUCTION_DATE|NONE",
 "days_to_deadline": <whole days left on the window, negative if it has closed, null if unknown>,
 "file_dispute": "YES|NO",
 "rationale": "one sentence, naming the anchor you chose and the rule that set it"}

Precedence for "posture", applied in this order: CONTEXT_INCOMPLETE beats everything; then FILED if
the carried state says a dispute has already been lodged, even if the window has since closed
(Rule Q-7); then EXPIRED if days_to_deadline is negative; then FILE_NOW if days_to_deadline is 7 or
fewer, because the deadline will pass before the next scheduled run; then WATCH if it is 14 or fewer;
otherwise CLEAR.

"file_dispute" is YES only on the run that LODGES the dispute -- the posture is FILE_NOW and the
carried state does not already say one was lodged. On every later run it is NO (Rule Q-6).

Carried state
----------------------------------------------------------------
No dispute has been lodged against this deduction yet. At the previous scheduled run, 9 day(s) remained on the dispute window. It was reported WATCH.

Deductions queue snapshot
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated deductions-queue snapshot for an AI
use-case kit; it reproduces no trade agreement, no customer's deduction policy, no
retailer's vendor guide and no real company. The dispute terms are ILLUSTRATIVE.

Deduction
----------------------------------------------------------------
Customer                       : CUST-4315 (Northbank Cash and Carry)
Deduction reference            : DED-0026
Deduction type                 : DAMAGE
Amount deducted                : 9,302.06 USD
Reason code as taken           : 834 -- product damaged in transit
Original invoice               : INV-882678
Purchase order                 : PO-5456732
Ship-to                        : XD-3 Larkmere

Dispute Terms
----------------------------------------------------------------
Trade agreement                : TA-4315-2025
Dispute window granted         : 90 days
Dispute clause (verbatim)      : Disputes must quote the original invoice number and be
                                 submitted inside the window stated above. The customer's
                                 claims portal is the only accepted channel.

Rule Q-1  A deduction may be disputed only INSIDE the dispute window the customer's trade agreement
          grants. Once that window closes an undisputed deduction is a WRITE-OFF: it is not
          recoverable by any later process, and no escalation reopens it.
Rule Q-2  The window runs from an ANCHOR DATE set by the agreement's dispute clause -- not from the
          age of the deduction on the ledger. Read the clause before counting anything.
Rule Q-3  Where the clause is silent about a deduction type, the default anchors are: SHORTAGE and
          DAMAGE from the PROOF OF DELIVERY date; PRICING and TRADE_PROMO from the ORIGINAL INVOICE
          date; COMPLIANCE_FINE and UNSALEABLES from the DEDUCTION (remittance) date.
Rule Q-4  A clause that names ONE anchor for all deductions OVERRIDES Rule Q-3 for every type on
          that customer, whichever type this deduction happens to be.
Rule Q-5  Days to deadline = the window granted MINUS the days elapsed from the anchor date. Both
          numbers are printed on this page; do not re-derive the elapsed days from the calendar.
Rule Q-6  A dispute is filed ONCE per deduction, on the first scheduled run at which the deadline
          falls inside the watch interval. A later run reports it already filed; it does not file a
          second dispute.
Rule Q-7  A deduction whose dispute was filed inside the window stays FILED after the window closes.
          Only an UNDISPUTED deduction past its deadline is EXPIRED.
Rule Q-8  Where the agreement carries no dispute clause, or the anchor date the clause needs is not
          on file, the deduction is CONTEXT_INCOMPLETE: no deadline is computed, nothing is filed,
          and NO default window and NO substitute anchor may be assumed.

Aging Position
----------------------------------------------------------------
Watch run                      : 2026-03-09   (scheduled run 2 of this deduction)
Watch cadence                  : every 7 days (Mondays, 08:00 local)
Previous scheduled run         : 2026-03-02
Candidate anchor dates, and days elapsed as at this run
  Original invoice date        : 2025-12-08    (  91 days elapsed )
  Proof of delivery date       : 2025-12-11    (  88 days elapsed )
  Deduction / remittance date  : 2025-12-25    (  74 days elapsed )
Days on the ledger since taken : 74

Claim Activity Log
----------------------------------------------------------------
Entries raised in this window only.

2026-03-05  customer's claims portal reconciled against our open items, this line still open

Deductions Desk Notes
----------------------------------------------------------------
Customer's claims desk has asked that we raise nothing further on this deduction while their own review is open.

Raw responseThe raw response

{
 "posture": "FILE_NOW",
 "anchor": "POD_DATE",
 "days_to_deadline": 2,
 "file_dispute": "YES",
 "rationale": "The dispute clause names no start date, so Rule Q-3's default anchor for DAMAGE applies (POD date); 90-day window minus 88 days elapsed leaves 2 days, so this run files the dispute."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Catch a customer deduction before its window closes — 150 deduction records. One model answered, and every answer was then graded Four different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

Every cell is compared for exact equality against a gold set computed by tools/build_corpus.py from the deduction model -- the resolved anchor, the window granted and the evidence dates -- and re-derived from src/deduction.step by evals/check_labels.py before any run may spend. The countdown is compared as a whole number of days, and a null countdown MATCHES a null one, because "no deadline can be computed" is the correct answer on 22 readings rather than a missing value. No model grades anything, here or anywhere in this kit.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anyone paid.

Rates checked 2026-08-18. The provider that actually ran all 306 calls is kept out of these tables per this estate's naming rule, so nothing here is what was paid. The real spend for this kit's whole build is on the shared call ledger, not on this page.

Grading unitWhat the grading figure prices

The GRADER costs nothing: evals/scoring.py makes no model call, and neither do the three free floors it is compared against, the pre-flight, or evals/cadence.py -- which produces the largest number on this page. The cost of the RUN is a different figure and lives in lens 07; pricing the ruler in the units of the thing it measures is the trap this field exists to name.

The gradersFour ways to grade

⚑ THE MODEL BEATS THE STRONGEST FREE FLOOR ON THE DISCRIMINATOR BY 13 READINGS, AND THE FLOOR IS NOT A STRAW MAN. Dispute window, exact whole days: 96.67 pct against 88.0. Anchor: 95.33 against 88.0. Posture: 96.67 against 92.67. Missed filings: 1 of 34 ($1,428.34) against 3 of 34 ($19,683.89). Duplicate filings: 0 against 2. The queue's at-risk value is $651,445.25 and the model reports $650,016.91 (0.22 pct out) against the floor's 12.97 pct. ⚑ AND THREE FLOORS RATHER THAN ONE, BECAUSE ONE CANNOT SEPARATE THE MEMORY FROM THE READING. b000 is the deductions ageing report an ERP already prints -- days on the ledger against one assumed 60-day window, no memory, no clause read: 32.67 pct posture, 82.67 pct file/hold, 4 duplicate filings and 22 missed ones worth $256,640.23. b001 is that same rule GIVEN the carried state: duplicates fall 4 -> 0 and posture goes 32.67 -> 58.67. That jump is what MEMORY ALONE buys with no model anywhere near it. b002 adds reading the clause, and it is worth 54.0 points of window accuracy on top -- the largest single movement on this page, and still free. ⚑ READ THE THREE SLICES BEFORE THE HEADLINE. The incumbent's aggregate window accuracy is 34.0 pct, which reads like a system with a general accuracy problem. It is nothing of the sort: it scores 100.0 pct on the 51 readings anchored on the remittance and 0.0 pct on the 36 anchored on the invoice and the 41 anchored on the delivery. Perfectly disjoint -- one third exactly right, two thirds exactly wrong, and the aggregate sitting in between telling you nothing about either. ⚠︎ BOTH FLOORS THAT GUESS SCORE 0 ON THE GUARDRAIL AND BOTH ARMS THAT ABSTAIN SCORE 100. b000 and b001 fill a missing window with an assumed 60 days and a missing clause with the remittance date -- exactly what a threshold report configured once and forgotten does -- and get all 22 of the readings with nothing on file wrong, ageing a deduction against a number nobody supplied. That is not a model finding; it is the row's own guardrail, measured.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Identical work in both rows — the same reading (one deduction, at one scheduled run), the same tokens. Only the price list changed. Which is why the token count is the measurement and the dollar is arithmetic you can redo on your own card.

Answering vs gradingAnswering and grading are two separate bills

306 live calls were attempted for this kit and all 306 returned something: 6 calibration (c000, at an 8,000-token ceiling on two deduction chains), 150 scored, and 150 for the stateless control. NOTHING WAS DISCARDED, no run was re-fired and no screenshot spent anything -- all four UI frames were taken with API_KEY blanked in the child process's environment and the two that show a verdict replay the committed result file. Everything else -- all three free floors, the cadence and missed-run arithmetic, the wiring stub, the scorer, the fifteen pre-flight assertions -- is pure code and costs $0.00. ⚑ AND THE LARGEST FIGURE THIS KIT PUBLISHES IS IN THAT FREE HALF: $453,621.42 lost to one skipped run, computed with no key. Figures are projected onto the Google Gemini 3 Flash card, not the real spend.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, and most of them are reasoning. 88.2 pct of this run's output (94782 of 107470) was provider-side reasoning left at the default, and output is 71.6 pct of the projected bill on the shared card. What you are paying for is the anchor decision.
• THE CADENCE, which multiplies everything else. One reading is one call; the watch wakes weekly; the bill is rows x runs, not rows. ⚑ AND THE CHEAPER CADENCE IS ALMOST FREE OF CONSEQUENCE HERE, MEASURED: monthly costs 200 calls over the horizon against weekly's 900 and loses one claim worth $3,278.53 (evals/cadence.py, $0.00).
• THE OPEN QUEUE, not the desk's turnover. A deduction that is disputed, settled or written off leaves the watch and stops costing anything. A desk that never closes its claims pays for them forever.
• The snapshot itself, barely. Input averages 1705 tokens and every document in this corpus is within 9 pct of every other in size.

Your volumeWhat it costs at your volume

LINEAR IN ROWS x RUNS, AND THAT IS THE WHOLE WARNING. Ten times the open deductions is ten times the calls at the same cadence -- there is no batching, no cache and no early exit, because every open claim is re-read whole on every run by design. A desk holding 4,000 open deductions on this weekly watch is 4,000 calls a week, about $12.01 a week on the shared projection card. Nothing about the per-call price changes; the multiplier is the schedule, which is why this kit's environment ladder makes cadence a declared decision rather than a deployment detail.

Model choiceWhy this model

It is the tier the shared connection was already pointed at, and the kit's whole claim is that swapping it is one line of .env and one more run. No comparison across tiers was attempted and nothing on this page claims one.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

The exact work behind every stateful number on these pages: 150 readings, one completion call each, one tier. The 150-call stateless control and the 6 calibration calls are accounted for separately in Cost.cost_of_evaluation_usd and are not projected here. Every figure this kit publishes about the SCHEDULE cost nothing at all and is not on this table.

The codeEvery file, and what it is for

13 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Six of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pythe corpus generator — a swap seam

Generates 50 open deductions x 3 weekly runs = 150 snapshots from a fixed seed (SEED = 990823). 13 of the 50 customers carry a dispute clause that OVERRIDES the type default anchor, which moves 39 readings off it -- that is the experiment. The gold labels are src/deduction.step's output over the planted terms, never typed.

You change it to: The deductions, the clause pools and the seed. Keep the seven section headings or src/segment.py's assertion refuses to start.

# Generate the corpus and the answer key. Deterministic: one seed, one corpus, forever.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
STATS = os.path.join(HERE, "data", "corpus-stats.json")
SEED = 990823
COUNT = 50
RULE = "-" * 64
RUN_DATES = (dt.date(2026, 3, 2), dt.date(2026, 3, 9), dt.date(2026, 3, 16))
HORIZON_START = dt.date(2026, 1, 5)

src/deduction.pythe dispute arithmetic — a swap seam

The rule as pure code: the anchor resolved from the clause override or the type default, the window granted minus the days elapsed from that anchor, and the two bands -- both of which ARE the cadence. No model, no judgement. ⚠︎ The eight rules, the window lengths and the anchor defaults are INVENTED and the kit ships that as a stated limitation -- see Data.why_this_corpus.

You change it to: TYPE_DEFAULT_ANCHOR and RULE_TEXT. The rule text is reproduced in full on all 150 pages precisely so it can be read, disbelieved and replaced with the agreements you actually signed.

# The dispute-window rule as arithmetic. Pure code, no model, standard library only.

CLEAR = "CLEAR"
WATCH = "WATCH"
FILE_NOW = "FILE_NOW"
FILED = "FILED"
EXPIRED = "EXPIRED"
CONTEXT_INCOMPLETE = "CONTEXT_INCOMPLETE"
POSTURES = (CLEAR, WATCH, FILE_NOW, FILED, EXPIRED, CONTEXT_INCOMPLETE)
INVOICE_DATE = "INVOICE_DATE"
POD_DATE = "POD_DATE"

src/state.pythe carried state — a swap seam

SEAM 2 -- the thing that makes this a monitor. Four scalars per deduction (whether a dispute is already lodged, last run's countdown, the posture last reported, how many runs have seen it), written from the arithmetic and never from the model's reply, and rendered into one English sentence for the prompt. ⚠︎ One of the four is double-edged and the kit measures it rather than defending it: carrying last week's countdown lets a reading reach the right number by subtracting 7 without re-reading the clause.

You change it to: What is carried between scheduled runs, and how it is worded to the model. Four scalars today. ⚠︎ ONE OF THEM IS A KNOWN TRADE-OFF: dropping days_to_deadline_last_run would force every reading to re-derive the countdown from the clause, which is slower and possibly more accurate on the anchor -- the control arm scores 96.67 pct on the anchor against this arm's 95.33. Two cells; not a claim.

# The carried state -- the thing that makes this a monitor and not another one-shot classifier.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEFAULT_PATH = os.path.join(HERE, "data", "state.json")
def load(path=DEFAULT_PATH):
def save(store, path=DEFAULT_PATH):
def for_deduction(store, deduction_id):
def describe(state):

src/segment.pythe section splitter

Splits a snapshot into its seven named sections. Pure code; evals/check_labels.py asserts all seven are present in all 150 documents before a run may spend.

# Split a deduction snapshot into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Deduction", "Dispute Terms", "Aging Position",
RULE = "-" * 64
def split(text):
def render(secs):

src/select.pythe section selector

Decides which sections reach the model. Customer Contact -- the customer's A/P analyst, her direct dial and her work email -- is mapped by no field and is subtracted unconditionally by _fallback(), so the one section whose subject is a PERSON never leaves the machine even when a source-system rename makes every hint match nothing.

# Pick which sections of a snapshot are sent. Pure code -- the last deterministic step before the

BANNER = "Synthetic Record"
DEDUCTION = "Deduction"
TERMS = "Dispute Terms"
POSITION = "Aging Position"
ACTIVITY = "Claim Activity Log"
CONTACT = "Customer Contact"
NOTES = "Deductions Desk Notes"
NEVER_SENT = (CONTACT,)
SECTION_HINTS = {

src/prompt.pythe prompt

Three parts: the fixed instruction, the carried-state sentence, and the selected sections in document order. The stateless build replaces the sentence with a line saying no history is available -- byte-identical everywhere else, which is what makes the control a control, and evals/check_labels.py asserts exactly that.

# Assemble the one prompt this kit sends. No template engine, no framework -- string concatenation

INSTRUCTION = """\
def build(text, carried, stateless=False):

src/adapters/__init__.pythe model adapter — a swap seam

SEAM 1 -- the model. Raw HTTP for every provider, standard library only. Returns input/output tokens, reasoning tokens and finish_reason beside the text, and separates a transport failure from an HTTP status so a dropped connection is retried rather than recorded as a lost reading.

You change it to: PROVIDER, BASE_URL, API_KEY and MODEL in .env. Adding a provider is one function and one entry in PROVIDERS; it must return token counts, because lens 05 publishes them and lens 07 prices them.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
def _post(url, headers, payload, timeout=120):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}
THINKING_OFF = {"type": "disabled"}
def complete(cfg, system, user, max_tokens=1024, thinking=None):

src/watch.pythe watch

One deduction, one scheduled run, one call. Parses the window, the three candidate anchor dates and their elapsed-day figures off the page with a regex (the model is never asked to subtract two dates), assembles the prompt, calls the model, parses the reply. Holds MAX_TOKENS = 24000, the published ceiling.

# One deduction, one scheduled run, one model call. The runtime half of the kit.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SYSTEM = ("You are a scheduled deduction-queue watch. You apply a written dispute rule to one "
MAX_TOKENS = 24000
FIELDS = ("posture", "anchor", "days_to_deadline", "file_dispute")
def documents():
def chains():
def load_doc(doc_id):
def position_of(text):

src/app.pythe local UI

One deduction, one scheduled run, its carried state and its verdict, on 127.0.0.1:9016. Renders with no key. It shows the carried sentence verbatim, the free floor's answer beside the model's, and a second button that replays what run r001-deduction-age actually answered, straight off the committed result file, labelled as a replay.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
PORT = int(os.environ.get("PORT", "9016"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-deduction-age")
GOLD_ROWS = {r["doc_id"]: r for r in
def carried_for(doc_id):
class H(BaseHTTPRequestHandler):
def main():

evals/baseline.pythe three free floors — a swap seam

b000 the deductions ageing report every ERP prints -- days since the deduction posted against one assumed 60-day window; b001 the same rule given the carried state, which isolates what memory alone buys; b002 the strongest free version -- carried state, the window read off the page, the anchor resolved by an ordered keyword table over the clause with a fall-back to the type default, and an abstention where nothing is on file. 0 calls, $0.00, all three scored through the identical scorer.

You change it to: POD_WORDS, INVOICE_WORDS and DEDUCTION_WORDS, and the order they are tried in -- the keyword table the model is measured against. Widen it and the floor gets stronger, which is the direction this kit wants: a floor built to lose measures nothing.

# THE FREE FLOORS. Three of them, none a strawman, all of them $0.00.

MODES = ("agedays", "agedays-mem", "anchor-mem")
ASSUMED_WINDOW_DAYS = 60
POD_WORDS = ("signed receipt", "proof of delivery", "receiving dock", "goods were signed",
INVOICE_WORDS = ("billing document", "invoice date", "we billed", "billed the order",
DEDUCTION_WORDS = ("remittance advice", "short payment posted", "deduction is taken",
def _num(pat, text, cast=int):
def _clause(text):
def _anchor_from_clause(clause, deduction_type):
def _elapsed(text, label):

evals/cadence.pythe cadence arithmetic

⚑ THE HALF OF A MONITOR THE ACCURACY TABLES CANNOT SEE, and it costs $0.00. It answers two questions from the deduction model with no key and no network: what does skipping ONE scheduled run cost (30 of 34 filings lost permanently, $453,621.42), and what does the interval itself cost (weekly to monthly loses one claim worth $3,278.53 and cuts the bill 4.5x). A reading that never happened produces no cell for a scorer to grade, which is why this is a separate file rather than a column.

# WHAT A MISSED RUN COSTS, AND WHAT THE CADENCE COSTS. Free. No key, no model, no network.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
SWEEP_CADENCES = (1, 3, 7, 14, 30)
def population():
def evidence_ready(v):
def schedule(start, end, cadence_days):
def catchable(v, runs):
def main():

evals/scoring.pythe scorer

Exact match per cell against the computed gold, split ways an average would hide: the four fields, the two filing directions counted apart AND in dollars, the memory-dependent subset, the context-incomplete recall, and -- the cut this kit exists for -- every rate again PER ANCHOR and PER SCHEDULED RUN. No judge model.

# Score a run against the answer key. Pure code, exact match per cell. No model grades anything.

FIELDS = ("posture", "anchor", "days_to_deadline", "file_dispute")
ANCHOR_SLICES = ("POD_DATE", "INVOICE_DATE", "DEDUCTION_DATE")
def _pct(n, d):
def _days(v):
def score(records, golds):

evals/check_labels.pythe pre-flight

Fifteen things that must be true before a run may spend: the seven sections parse, every sequence is complete and gap-free, no run-varying section says a dispute was lodged and the non-varying ones are byte-identical across runs (the experimental control), no page carries a deadline field or a countdown, THE ANCHOR IS NOT DERIVABLE FROM THE DEDUCTION TYPE, every anchor slice is big enough to report, the privacy guard holds AND would fail without it, the two prompt builders differ on exactly one line, no code path writes off or credits a deduction, and the answer key replays from src/deduction.step.

# Everything that must be true BEFORE a run is allowed to spend. Free, and it runs in seconds.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
GOLD = os.path.join(HERE, "data", "gold.jsonl")
FAILED = []
def check(name, ok, detail=""):
def main():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 1704 input and 716 output tokens per reading (one deduction, at one scheduled run), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

No untrusted external input surface in the shipped version. Every field that reaches the prompt comes from tools/build_corpus.py's fixed seed, including the Deductions Desk Notes, which are one of three fixed sentences. src/select.py names the notes as the field an outside party WOULD influence in a real deployment -- in a working desk much of what lands there is pasted verbatim out of a customer's claim portal or a chaser email -- and sends them deliberately, so the surface is visible rather than hidden. On this corpus nobody outside the repository wrote a byte of it.

The provider key lives only in .env (gitignored) or the shared repo-root .env, read by src/config.py and never logged. config.sources() reports WHICH file a value came from and never the value. config.save() opens the file 0600 before the write rather than chmod-ing it after, so the key never exists at the default umask even for an instant. src/app.py redacts the key and the base URL out of any provider error before it reaches the browser.

The experimentWe did NOT attack it — and this is the surface we left open

An indirect prompt injection needs a field somebody outside can write into, and this kit has exactly one: the Deductions Desk Notes. It is SENT rather than hidden, because hiding a surface does not close it -- and one of the three shipped notes asks, in the customer's voice, for the thing the kit exists to do. No attack was fired at this kit and none is claimed. The three boundaries below were checked in code on 2026-08-23, and only the first of them is red-proven in both directions rather than argued from an absent code path.

The first row is the only one measured in both directions. The other two are guarantees about what is ABSENT — no code path writes off, credits or adjusts a deduction, and no code path feeds a model reply back into the carried state — and absence is checked by asserting names it knows, which is weaker than a run and is written here as such.

The result0 attack trials, three boundaries checked -- and the privacy boundary red-proven by removing the guard and watching all 150 snapshots leak a named individual's direct dial and work email.

The Deductions Desk Notes ARE the field an outside party would influence in a real deployment, and this kit sends them rather than hiding them -- one of the three shipped sentences is even instruction-shaped, and it asks for the omission that costs the money. But on this corpus they are one of three sentences from a seeded generator, so there is nothing adversarial in them to catch. A version pointed at real desk prose reopens the question and should be attacked before it ships. What IS measured is the privacy boundary, in both directions, before every run.

HonestyWhat this does not prove

• Whether a real desk's notes -- prose an analyst writes freely, often pasted out of a customer's portal -- would carry an instruction the model follows. Not applicable to the shipped corpus, and the first thing to attack if this is pointed at a real queue. The shipped note asking that nothing further be raised is the shape to probe, because the action it asks to suppress is the one whose omission is unrecoverable.
• Whether the state file is safe under concurrency. src/state.save() replaces atomically, which is correct for one writer and is not a concurrency model; two watches on one queue have never been run and would race. Here a lost write forgets which disputes are lodged, and the next run files a second one against every claim in the queue.
• Whether a truncated reply can ever be partially trusted. Neither scored arm lost a call, so src/watch._parse's outright rejection was never exercised on this kit.
• Whether the grep-shaped no-write-off assertion would catch a path written under a name it does not know. It asserts the absence of nine names; the guarantee is about ABSENCE and that is the only mechanical form it can take.
• What the provider retains. The prompt carries no personal data by construction -- the Customer Contact section never leaves -- but what a vendor keeps of a request is outside this repository and nobody here has verified it.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

No write-off, no credit, no customer contact -- non-configurable. This kit produces a worklist posture, an anchor, a countdown and a file/hold call for a deductions analyst to read. It never writes off, credits, adjusts, clears or suppresses a deduction, never posts to a customer's account, never lodges anything with a customer's claims portal and never sends a message to a customer, and there is no setting that makes it. Separately, and just as non-configurable: the dispute window and its anchor are OPERATOR-SUPPLIED. A deduction whose agreement carries no dispute clause, or whose anchor date is not on file, is reported CONTEXT_INCOMPLETE and aged against nothing -- there is no default window and no substitute anchor in this kit.

Everywhere and nowhere -- it is a property of what is ABSENT. The only writers in the kit are evals/run.py (results/*.json), evals/cadence.py (one results file) and src/state.save (data/state.json). src/deduction.step() is the only thing that touches the carried state, and it is called by evals/run.py after every reading including one whose call FAILED.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• IT IS NOT A CHECK ON WHETHER THE ANSWER IS RIGHT. The carried state is correct whatever the model says, which means a wrong reading is a wrong worklist row and not a corrupted history. Those are different problems and only the first one is on this page.
• IT IS NOT A SCHEDULER, AND ON THIS KIT THAT IS THE MOST EXPENSIVE THING IT IS NOT. evals/run.py is invoked; it is not woken. Nothing here detects a missed run, back-fills it or marks its readings late -- and evals/cadence.py measures what one skipped week costs: 30 of 34 filings gone permanently, $453,621.42.
• IT IS NOT AN AUTHORITY ON DEDUCTION DISPUTES. The eight rules, the window lengths and the anchor defaults are invented for this kit and reproduce no trade agreement, customer deduction policy, retailer's vendor guide or regulator's guidance.

WatchedWhat is watched, and why that one

The kit measures 42 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

24 measured by the latest run 18 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (cadence_days 7, context_incomplete_cells 22, deductions 50, documents 150, expired_cells 13, memory_cells 39, must_file_cells 34, quiet_cells 116, readings_scored 150, stateless False) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on floor, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · with the model in the path — 3 runs. Columns here are only ever compared with each other.

not a time series No two of these 3 runs measured the same system — they differ on context_incomplete_cells, deductions, documents, expired_cells, max_tokens, memory_cells, must_file_cells, quiet_cells, readings_scored, stateless — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 24 chips that all say so.

DeviationsWhat deviated

0 breaches across 7 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

Re-run evals/check_labels.py (free, under a second) on any change to tools/build_corpus.py, src/deduction.py, src/segment.py or src/select.py -- it refuses to let a run spend if the corpus, the answer key, the anchor trap, the experimental control, the privacy guard or the no-write-off property has moved. Re-run evals/cadence.py on any change to the corpus or the cadence; it is free and it carries the largest figure on the page. Re-run the scored eval AND its stateless control together (paid, 300 calls) on any change to src/prompt.py or src/state.py: the memory claim is a difference, so one arm re-run alone is not comparable with the other's old figure. All three free floors cost nothing and should be re-run on every change, in both directions -- a floor that gets stronger is a finding, not a regression.

In one lineNo framework, deliberately

No framework and no dependencies at all -- json, os, re, random, datetime, argparse, urllib, http.server and concurrent.futures from the standard library. requirements.txt names nothing and says in a comment that the emptiness is load-bearing: a line there means something under src/ or evals/ imports it.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Nothing here is a graph, and the one place it looks like one is a straight line. Each deduction is a chain of three readings -- three consecutive Mondays -- with no branching and exactly one edge between consecutive runs, carrying four scalars. Different deductions never touch. A framework would add an orchestrator to a for-loop that already runs 50 chains wide.

The other sideWhat a framework costs you

• No scheduler. This is a watch that carries state and is still INVOKED, not woken -- and this kit measured what that gap costs rather than describing it: $453,621.42 for one skipped weekly run. That is the honest cost of staying a folder of readable Python, and it is stated on the kit's own page rather than implied by silence.
• No missed-run detection, no back-fill, no late marking. All three are what a scheduling engine gives you and none of them exists here.
• No concurrency model for the state store. data/state.json is one file replaced atomically -- correct for one writer, and a desk with two watches running is two writers.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-deduction-age on the same tier, memory removed (THE CONTROL), 2026-08-23. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 5 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-deduction-age-agedays, b001-deduction-age-agedaysmem, b002-deduction-age-anchormem recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the population is re-read whole on each scheduled run, and the prior run's carried state is what makes a reading a CHANGE.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT -- this repository's own licence. Nothing is fetched and no third-party grant is involved; every customer, deduction, trade agreement, dispute clause, window length, amount and activity-log line is invented here. Verified against the repository's own LICENSE file on 2026-08-23. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe posture, the anchor, the countdown and the file/hold call, per reading, exact match against the computed answer key

For each of the 150 readings and each of the four answered fields, did the reply equal the computed answer key? The posture, the anchor and the file/hold call are compared exactly against closed lists; the countdown is compared as a whole number of days, and a null MATCHES a null because "no deadline can be computed" is the correct answer on 22 readings and must not be scored as a parse failure.

MethodHow the test was run

evals/scoring.py::score, in-process, no key and no model. The same function all three free floors are scored through.

Every grader on these pages scored the same 150 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 150 per field. A reading whose reply did not parse counts as a MISS in every field, never as an exclusion.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl, computed by tools/build_corpus.py over the planted terms at generation time and re-derived from src/deduction.step by evals/check_labels.py before any run may spend. This grader IS the reference, so its own TPR and TNR are not measured -- scoring it against itself would be circular.

How tight can the band be? No tuned threshold anywhere on the model path -- the reply decides. The denominators are printed beside every rate because two of them are small: 34 readings that must lodge a dispute and 22 with nothing on file, so one row moves the first by 2.9 points and the second by 4.5.

Cadence: Re-run evals/check_labels.py (free) on any change to tools/build_corpus.py, src/deduction.py, src/segment.py or src/select.py. Re-run the scored eval AND its stateless control together (paid, 300 calls) on any change to src/prompt.py or src/state.py -- the memory claim is a difference, so one arm re-run alone is not comparable with the other's old figure. All three floors and evals/cadence.py are free and should be re-run on any change at all.

The decisionWhen to reach for it

In one lineThe same comparison, cut per anchor -- the grader this kit exists for

The identical exact-match comparison, restricted to the readings whose CORRECT anchor is the proof of delivery (41), the original invoice (36) or the remittance (51). It answers the one question the aggregate cannot: is this arm uniformly mediocre, or is it wired to the wrong date for part of the population?

MethodHow the test was run

evals/scoring.py::score, same pass, grouped by the anchor tools/build_corpus.py resolved at generation time.

Every grader on these pages scored the same 150 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / slice size, per anchor, on all four fields. The slices partition the 128 readings whose anchor is not NONE and never overlap.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the same data/gold.jsonl, partitioned by the anchor the generator resolved. Scored against the reference; not itself the reference.

How tight can the band be? The three slices are 41, 36 and 51 readings, so the finest bands they support are 2.4, 2.8 and 2.0 points. A one-cell difference is not a finding on any of them; a hundred-point gap is.

Cadence: Free, every run. Re-derive the slices whenever tools/build_corpus.py changes -- they are a property of the corpus, and a corpus whose anchors were a function of the deduction type would make this whole grader meaningless (evals/check_labels.py asserts they are not).

The decisionWhen to reach for it

In one lineThe two filing directions, counted apart and in dollars

Two counts over two different denominators, and one of them is also counted in money. A MISSED filing: of the 34 readings whose correct answer is to lodge the dispute, how many did not -- and what were they worth, because on a hard clock a missed filing is a write-off rather than a late alert. A DUPLICATE filing: of the 116 readings that must not lodge one, how many did.

MethodHow the test was run

evals/scoring.py::score, in the same pass as the exact-match grader.

Every grader on these pages scored the same 150 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

missed_filing_pct = missed / 34; duplicate_filing_rate_pct = duplicates / 116; missed_filing_usd = the summed face value of the missed claims. Never averaged and never combined into an F-score.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl's file_dispute column, computed by src/deduction.step. Scored against the reference and not itself the reference.

How tight can the band be? 34 readings must lodge a dispute and 116 must not, so the finest band either rate can support is 2.9 and 0.9 points respectively. A "one missed filing" claim on this corpus is a claim about 34 readings and nothing wider.

Cadence: Free. Re-run with every scored run, and re-run the floors whenever src/state.py changes -- the duplicate count is the most memory-sensitive figure on this page.

The decisionWhen to reach for it

In one lineThe readings whose answer is not on their own page

The same exact-match comparison, restricted to the 39 readings where a dispute had ALREADY been lodged before the reading began. Nothing on those pages says so, so the correct posture (FILED, not EXPIRED and not FILE_NOW) is reachable only through the carried state. It answers one question -- is the memory doing anything, or is the corpus easy?

MethodHow the test was run

evals/scoring.py::score, same pass, grouped by the flag tools/build_corpus.py set at generation time -- before the step that produced the answer, so it records what was known going in.

Every grader on these pages scored the same 150 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 39 per field, over the memory_dependent subset only.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the same data/gold.jsonl, sliced by the memory_dependent flag. Scored against the reference; not itself the reference.

How tight can the band be? 39 cells, so the finest band this subset supports is 2.6 points.

Cadence: Free, every run. Re-derive the flag whenever tools/build_corpus.py changes -- a corpus with no memory-dependent readings would make this kit unfalsifiable (evals/check_labels.py asserts there are at least 30).

The decisionWhen to reach for it