The business caseThe problem this solves

A card network's daily settlement file is reconciled against the processor's own transaction record. The totals usually tie. When they do not, the difference is almost never a missing transaction -- it is a fee or rate rule, and WHICH rule decides who, if anyone, does anything about it. An interchange downgrade is a defect at source that re-earns the same gap on every settlement date until a clearing message changes. A cross-border assessment on genuinely cross-border volume is explained AND correct and needs nobody. A cut-off or a chargeback reversal clears on its own. All of them are one number, over the materiality floor, on one component. a settlement difference report that flags every gap over a materiality floor -- which on this corpus means routing 71 of 100 quiet readings to somebody, including every one of the 13 correctly-priced cross-border assessments, while catching 0 of the 13 interchange downgrades that are the only gaps anybody can actually stop.

The inputThe actual settlement reconciliation extracts

The corpus is 120 settlement reconciliation extracts, 1.04 MB (txt 120). A processor's settlement reconciliation extract gives merchant-level daily volume, the acquiring BIN, the interchange programme each MID is rated under and exactly which MIDs are downgrading. There is no public one. A scrubbed export is worse rather than better: scrubbing removes the recon notes, which is exactly the layer this kit measures.

Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated card-settlement reconciliation extract
for an AI use-case kit; it reproduces no card network, no processor, no acquirer, no bank,
no merchant and no real person. The expected settlement windows, the adjustment-filing
window, the materiality floor, the interchange tier names and the assessment rates are
ILLUSTRATIVE DEFAULTS AND INVENTED LABELS -- no real scheme's interchange table is
reproduced here and no rate on this page is a real published rate.

Settlement Batch
----------------------------------------------------------------
Batch reference                    : NS-0001
Merchant MID                       : 919628558  Northaven Supplies
Card network                       : MERIDIAN-SCHEME
Acquiring BIN                      : 4066 71
Interchange programme on file      : PREMIUM DEBIT
Settlement date                    : 2026-08-18 (Tuesday)
Recon analyst of record            : Noor Haddad

Two-Way Position
----------------------------------------------------------------
The processor's own transaction record for this settlement date against what the network
actually settled, per component, in GBP. Fee components are shown as negatives because they
are deducted from the merchant's settlement. A negative gap means the network settled LESS
than the processor's own record accounts for.

  Component      Processor record    Network settled            Gap   Expected settlement
  PRINCIPAL            844,125.75         844,125.75           0.00   T+1 banking day
  INTERCHANGE          -16,145.68         -16,145.68           0.00   T+1 banking day
  ASSESSMENT            -4,521.68         -20,516.43     -15,994.75   T+2 banking days

The outcomeWhat a good result looks like

every settlement date's verdict with the rate rule that explains it, the recon runs open, and -- the part free code cannot reach -- WHICH TEAM it belongs to: 13 of 13 downgrades caught, 13 of 13 correctly-priced assessments named and routed to nobody, 1 false routes across 100 quiet readings.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Every network advice arrives as a coded RC: line, keyed the same day — the free floor -- fee-code-strong, $0.00
  It reads the codes, carries runs open, and BEATS the model on the route-now call (100.0 against 99.17) and on cause over the memory-dependent slice (90.91 against 86.36).
• Downgrades and assessments are established on calls and typed as sentences — the model, with the carried state
  100.0 pct of downgrades caught against the strong floor's 84.62, 100.0 pct of correctly-priced assessments NAMED against 92.31, and 100.0 pct on cause on the run the evidence lands in prose against 0.0.

And where nothing here is good enough:

• You want the reading but cannot carry state between recon runs — neither, and on this kit the reason is unusual
  The stateless control catches 0.0 pct of the downgrades that are not readable off their own page, answers 3 of 20 batches still needing somebody at the last run against 20 of 20, misses 86.41 pct of the value at risk, AND spends 2.2x the output tokens.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-25. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-25 — r002-network-settle. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Every corpus extract, the answer key, all three free floors, the injection probe, the discarded 16,000-ceiling run and what r002-network-settle actually answered ship in the repo. python3 -m evals.check_labels and python3 -m src.app both run with no key and no network.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one reading -- one settlement batch, at one recon run, end to end including provider-side reasoning tokens, on the shared connection. Not a cold-start figure and not a per-batch SLA: 120 readings ran in 407.1 wall seconds because different batches run concurrently while a single batch's runs are strictly serial.

Current processWhat it replaces

a settlement difference report that flags every gap over a materiality floor -- which on this corpus means routing 71 of 100 quiet readings to somebody, including every one of the 13 correctly-priced cross-border assessments, while catching 0 of the 13 interchange downgrades that are the only gaps anybody can actually stop.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureArchitectureArchitectureEvalsEvalsEvalsEvalsPromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 3 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one reading is

40 settlement batches x 3 daily recon runs, walked strictly in order per batch so the carried state moves forward the way it would in a real deployment. Different batches run concurrently; a single batch's runs never do.

SetupWhat the setup figure measured

There is no index to build -- each reading's extract goes whole into the prompt. The 0.6s and $0.00 are the corpus generation itself.

LicenceLicence

MIT

Bring your ownBring your own settlement reconciliation extracts

tools/build_corpus.py writes data/corpus/*.txt and data/gold.jsonl. Keep the 8 section headings and the field labels src/recon.position_of parses. Set your own settlement windows, filing window and materiality floor in src/settle.py first.

⚠︎ And what stops being true when you do: The measured figures on this kit's page do not transfer to your own settlement feed. The downgrade-not-on-the-page share (7 of 13) and the prose-versus-coded-advice evidence split are properties of this generator's declared distribution, not facts about card settlement reconciliation. Re-run the evals on your own data; that is what the harness is for.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — they were measured against the provider’s own counter, not apportioned by character share.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 4,182 tokens assembled, 2,129 are contexts — 51% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Reproduced from src/prompt.build with that reading's real carried state, not retyped -- byte-identical to what evals/run.py sent.

Full promptThe prompt in full

You are a daily card-settlement reconciliation. You read one settlement batch's two-way position -- the processor's own transaction record against what the network settled, per component -- at one recon run, against the DEFAULT fee and rate rules, and you answer with one JSON object and no other text.

You are the daily settlement reconciliation for one card processor. It runs every banking morning
and re-reads EVERY settlement date that is not yet reconciled. You are reading ONE settlement batch
at ONE recon run.

The batch, its two-way position (the processor's own transaction record against what the network
actually settled, per component), the DEFAULT fee and rate rules, and everything recorded since the
previous recon run are reproduced in the extract below. Apply them exactly as written. You cannot see
the earlier recon runs; what is known about them is stated under "Carried state" and is the only
history available to you. Do not assume anything about earlier runs beyond it.

⚠︎ THE EXPECTED SETTLEMENT WINDOWS, THE ADJUSTMENT-FILING WINDOW, THE MATERIALITY FLOOR, THE
INTERCHANGE TIER NAMES AND THE ASSESSMENT RATES ARE ILLUSTRATIVE DEFAULTS AND INVENTED LABELS, NOT
ANY REAL CARD NETWORK'S, PROCESSOR'S OR ACQUIRER'S TERMS. Apply them as printed regardless of whether
they look right for the MID in front of you.

THE POINT OF THIS JOB: the totals usually tie, and when they do not, the difference is almost never a
missing transaction. It is a FEE OR RATE RULE -- and which rule decides who, if anyone, does anything
about it:

  A gap caused by an INTERCHANGE DOWNGRADE is a defect AT SOURCE. Volume cleared at a worse tier
  because a qualifying field was absent from the clearing message. The money is not coming back and
  there is nothing to file with the network -- the volume was rated correctly for the message that
  was actually sent. It goes to the team that owns that message, and until they change it the same
  gap is re-earned on every settlement date.

  A gap caused by a CROSS-BORDER ASSESSMENT on genuinely cross-border volume is EXPLAINED AND
  CORRECT. It is a price, not a defect. It goes to NOBODY. Routing it is a morning spent proving that
  a price is a price, and it is how a settlement exception queue stops being read.

  A gap caused by the network's CUT-OFF or by a CHARGEBACK REVERSAL value-dating in a later cycle is
  TIMING, and it will clear on its own.

  A gap that is none of those, past its component's window and not moving, is genuinely UNMATCHED
  money.

Those four look IDENTICAL on the page: one number, over the materiality floor, on one component.

How to work it out:
 - FIRST decide whether both feeds arrived for the settlement date. If the processor record or the
   network settlement file is missing, the verdict is CONTEXT_INCOMPLETE: no gap is judged, no cause
   is determined, nothing is routed, and the runs-open count does NOT advance. THERE IS NO DEFAULT
   SETTLEMENT WINDOW for a feed that never arrived (Rule N-9).
 - THEN decide whether this settlement date is already CLEARED. If the carried state says an earlier
   run recorded the gap as settled, the verdict is CLEARED and stays CLEARED (Rule N-8).
 - THEN read the two-way gap off the printed per-component table. A gap under the printed materiality
   floor is not a gap.
 - If the position agrees: the verdict is TIES if this settlement date has NEVER had a gap open
   against it, and CLEARED if it had one and it has now settled. Those two are the same zero on the
   page and only the carried state tells them apart (Rule N-8). The cause is NONE for TIES. Nothing
   is routed either way.
 - If there IS a gap, note which component carries it -- that component selects the EXPECTED
   SETTLEMENT WINDOW from the printed table (Rule N-1).
 - If the record explains the gap as an INTERCHANGE DOWNGRADE, the verdict is RATE_RULE_FIXABLE at
   any age, inside the window or not, and it NEVER ages into anything else: nothing is going to
   arrive, and it is not a claim, so the filing clock is not its clock (Rules N-4, N-7).
 - If the record explains the gap as a CROSS-BORDER ASSESSMENT correctly applied to genuinely
   cross-border volume, the verdict is RATE_RULE_PRICED at any age, inside the window or not, and
   NOTHING IS ROUTED. It is explained and it is correct (Rules N-5, N-7).
 - THEN, if the settlement date is already OLDER than the printed adjustment-filing window, the
   verdict is AGED_PAST_FILING -- on sight, on its first recon run, without waiting for a second
   position to compare against. A filing window is a DEADLINE, not a settlement schedule (Rule N-7).
 - If the record says the network confirms transactions were NEVER RECEIVED in clearing and will not
   be settled, the verdict is UNMATCHED at any age, inside the window or not (Rule N-6).
 - Otherwise, compare the settlement date's age in BANKING DAYS SINCE THE SETTLEMENT DATE against
   that component's expected settlement window. INSIDE the window: TIMING_WINDOW. Do not route it
   (Rule N-2).
 - PAST the window, it is UNMATCHED only if the carried state shows the SAME gap at the previous
   recon run AND it has not moved. A gap that has SHRUNK since the previous run is settlement
   arriving and stays TIMING_WINDOW however old it is. If there is no previous position on record at
   all -- this is the settlement date's first recon run -- nothing convicts it and it stays
   TIMING_WINDOW (Rule N-3). A network advice promising an adjustment is NOT an adjustment: convict
   on the position, never on the promise.
 - THEN decide the CAUSE. Cause evidence ACCUMULATES from earlier runs (see "Carried state") and from
   anything recorded THIS run -- in the structured advice log or in an ordinary recon note. A note
   that describes a cause in plain prose counts exactly as much as a coded advice. Use UNDETERMINED
   only when the gap is real and nothing anywhere says why, and NONE only when there is nothing to
   explain.
 - THEN give "runs_open": the number of RECON RUNS on which this reconciliation has reported this gap
   open, INCLUDING this one. It is stated in the carried state and it is NOT the settlement date's
   age in banking days -- a settlement date can reach this queue already weeks old with a runs-open
   count of zero. It is 0 when nothing is open, and it FREEZES rather than resets once the gap
   settles.
 - THEN decide the OWNER. Start from the recon analyst on file, but a note recording a handover to
   someone else, in this run or an earlier one (see "Carried state"), supersedes it.
 - FINALLY decide "route_now": YES only if the verdict is RATE_RULE_FIXABLE, UNMATCHED or
   AGED_PAST_FILING AND the carried state does not already show a route raised at that same level.
   Those three go to three different teams, so a route already raised at one level does not satisfy
   another. RATE_RULE_PRICED and TIMING_WINDOW are NEVER routed. Otherwise NO.

Answer with a single JSON object and nothing else:

{"verdict": "TIES|TIMING_WINDOW|RATE_RULE_FIXABLE|RATE_RULE_PRICED|UNMATCHED|AGED_PAST_FILING|CLEARED|CONTEXT_INCOMPLETE",
 "cause": "INTERCHANGE_DOWNGRADE|CROSSBORDER_ASSESSMENT|CHARGEBACK_WINDOW|CUTOFF_LAG|UNMATCHED_TRANSACTION|UNDETERMINED|NONE",
 "runs_open": <integer>,
 "owner": "<the current recon analyst's name>",
 "route_now": "YES|NO",
 "rationale": "one sentence, naming the gap, the component, the window or rate rule you applied and
   the evidence -- carried or recorded -- you relied on"}

Precedence, applied in this order: CONTEXT_INCOMPLETE if a feed is missing (Rule N-9); then CLEARED
if the carried state already says so (Rule N-8); then TIES or CLEARED if the position agrees; then
RATE_RULE_FIXABLE on an interchange downgrade (Rule N-4); then RATE_RULE_PRICED on a correctly
applied cross-border assessment (Rule N-5); then AGED_PAST_FILING (Rule N-7); then UNMATCHED on
volume the network never received (Rule N-6); then UNMATCHED on an unmoved gap past its window
(Rule N-3); otherwise TIMING_WINDOW (Rule N-2). "cause" is UNDETERMINED whenever the verdict is
CONTEXT_INCOMPLETE.

Carried state
----------------------------------------------------------------
At the previous recon run this settlement date was short 4,881.14 GBP on the INTERCHANGE leg. This reconciliation has reported that gap open on 1 earlier recon run(s) -- so if it is still open now, its runs-open count is 2. No cause has been established for this settlement date on any earlier run. This reconciliation has not routed this settlement date anywhere. The recon analyst as last settled is Noor Haddad -- carried forward unless this run's record hands the settlement date to someone else. It was last reported TIMING_WINDOW.

Settlement reconciliation extract
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated card-settlement reconciliation extract
for an AI use-case kit; it reproduces no card network, no processor, no acquirer, no bank,
no merchant and no real person. The expected settlement windows, the adjustment-filing
window, the materiality floor, the interchange tier names and the assessment rates are
ILLUSTRATIVE DEFAULTS AND INVENTED LABELS -- no real scheme's interchange table is
reproduced here and no rate on this page is a real published rate.

Settlement Batch
----------------------------------------------------------------
Batch reference                    : NS-0020
Merchant MID                       : 438764905  Castleraine Events
Card network                       : MERIDIAN-SCHEME
Acquiring BIN                      : 5219 84
Interchange programme on file      : PREMIUM DEBIT
Settlement date                    : 2026-08-18 (Tuesday)
Recon analyst of record            : Noor Haddad

Two-Way Position
----------------------------------------------------------------
The processor's own transaction record for this settlement date against what the network
actually settled, per component, in GBP. Fee components are shown as negatives because they
are deducted from the merchant's settlement. A negative gap means the network settled LESS
than the processor's own record accounts for.

  Component      Processor record    Network settled            Gap   Expected settlement
  PRINCIPAL          1,642,154.56       1,642,154.56           0.00   T+1 banking day
  INTERCHANGE          -25,219.82         -30,100.96      -4,881.14   T+1 banking day
  ASSESSMENT            -7,382.82          -7,382.82           0.00   T+2 banking days
  CHARGEBACK           -12,612.76         -12,612.76           0.00   T+3 banking days

  Processor record, net                       1,596,939.16
  Network settled, net                        1,592,058.02
  Two-way gap                                    -4,881.14
  Materiality floor (default)                        25.00

Component carrying it              : INTERCHANGE

Fee And Rate Rules (Default)
----------------------------------------------------------------
The windows below are an OPERATOR-TUNABLE DEFAULT, not any real card network's, processor's
or acquirer's terms -- no documented settlement windows, filing deadline or materiality
floor exist for this row. Applied exactly as printed. A gap's age is counted in BANKING DAYS
FROM THE SETTLEMENT DATE, never in calendar days.

  Component    Expected settlement window
  PRINCIPAL    1 banking day from the settlement date
  INTERCHANGE  1 banking day from the settlement date
  ASSESSMENT   2 banking days from the settlement date
  CHARGEBACK   3 banking days from the settlement date

Adjustment-filing window           : 10 banking days from the settlement date
Materiality floor                  : 25.00 GBP
Re-rating authority                : NOT DEFINED. Nothing in this kit posts an
                                     adjustment, files a network claim, re-rates a transaction,
                                     releases a settlement or amends a merchant statement.

Rule N-1   The position is TWO-WAY and it is read PER COMPONENT off the printed table: what the
           processor's own transaction record says for this settlement date, against what the network
           actually settled, for the PRINCIPAL, the INTERCHANGE, the ASSESSMENT and the CHARGEBACK
           legs. The component carrying the gap selects which EXPECTED SETTLEMENT WINDOW applies.
           The windows, the adjustment-filing window and the materiality floor are operator-tunable
           DEFAULTS, not any real scheme's or acquirer's terms, and are applied exactly as printed.
Rule N-2   A gap INSIDE its component's expected settlement window -- counted in banking days from
           the settlement date -- is TIMING_WINDOW. It MUST NOT be routed. Most differences on a
           daily settlement recon are this, and routing them is the entire reason the exception
           queue gets ignored.
Rule N-3   PAST that window, a gap is UNMATCHED only if the PREVIOUS recon run's position shows the
           SAME gap and it has not moved. A gap that is still shrinking is settlement arriving, and
           it stays TIMING_WINDOW however old it is. If there is no previous position on record at
           all, nothing convicts it: it stays TIMING_WINDOW. A network advice PROMISING an adjustment
           is not an adjustment -- this rule convicts on the POSITION, never on the promise.
Rule N-4   A gap the record explains as an INTERCHANGE DOWNGRADE -- volume that cleared at a worse
           tier because a qualifying field was absent from the clearing message -- is
           RATE_RULE_FIXABLE at ANY age, inside the window or not. Nothing is going to arrive: the
           volume was rated, correctly, under the rule that applied to the message actually sent.
           IT IS NOT A CLAIM AND IT NEVER AGES INTO ONE. It is a defect at source that recurs on
           every settlement date until the message is changed, which is why it is routed to the team
           that owns that message and not to the network.
Rule N-5   A gap the record explains as a CROSS-BORDER ASSESSMENT correctly applied to genuinely
           cross-border volume is RATE_RULE_PRICED at ANY age, inside the window or not, and
           NOTHING IS ROUTED. It is explained AND it is correct. This is the one verdict on this
           board where the right answer is that a real, material, fully-explained gap needs no
           person at all -- and it looks identical on the page to Rule N-4's, which needs one
           urgently.
Rule N-6   A network confirmation that transactions were NEVER RECEIVED in clearing and will not be
           settled for this date is UNMATCHED at ANY age, inside the window or not. Volume that did
           not reach the network is not going to settle.
Rule N-7   A gap on a settlement date already OLDER than the printed adjustment-filing window is
           AGED_PAST_FILING on sight -- Rule N-3 does not get to hold it back. A filing window is a
           DEADLINE, not a settlement schedule: once it has passed, "it might still be settling" no
           longer makes the money recoverable, and the reading has to say so the first time it is
           seen. RATE_RULE_FIXABLE and RATE_RULE_PRICED do NOT age into it -- neither is a claim, so
           the filing clock is not theirs.
Rule N-8   Once a gap settles, the batch is CLEARED and stays CLEARED on every later run. CLEARED is
           not TIES. A settlement date that never disagreed is TIES; one that disagreed and then
           settled is CLEARED, and on a day when both show a zero gap the only thing that tells them
           apart is this reconciliation's own history.
Rule N-9   If either feed is missing for the settlement date, the reading is CONTEXT_INCOMPLETE: no
           gap is judged, no cause is determined and nothing is routed. THERE IS NO DEFAULT
           SETTLEMENT WINDOW for a feed that did not arrive, and the runs-open count does not
           advance -- a reconciliation that could not look cannot claim the item stayed open.
Rule N-10  "route_now" is YES only where the verdict owes somebody work AND this reconciliation has
           not already routed at that level. The three routable verdicts go to three different
           teams, so a route already raised at one level does not satisfy another. RATE_RULE_PRICED
           and TIMING_WINDOW are never routed at all.

Recon Position
----------------------------------------------------------------
Recon date                         : 2026-08-20   (recon run 2 of this settlement date)
Recon cadence                      : daily -- one settlement reconciliation run every banking morning
Banking days since settlement date : 2
Previous recon run                 : 2026-08-19
Next recon run                     : 2026-08-21
Adjustment-filing window remaining : 8 banking days

Events In This Window
----------------------------------------------------------------
Every network advice and every note recorded against this settlement date between the
previous recon run and this one. THIS RUN ONLY -- an advice recorded on an earlier run is
not repeated here.

07:28  Spoke to the scheme rep this morning: the whole batch downgraded because our
       clearing message is missing the customer reference on the clearing message. They
       will not re-rate it -- the fix is on our side, in the auth, and until it goes in
       we will lose the same amount every settlement date.

Recon Notes
----------------------------------------------------------------
This MID sits on a shared descriptor with two sister sites, so refunds against the
group's online orders land in the same settlement file.

Raw responseThe raw response

{"verdict": "RATE_RULE_FIXABLE", "cause": "INTERCHANGE_DOWNGRADE", "runs_open": 2, "owner": "Noor Haddad", "route_now": "YES", "rationale": "The 4,881.14 GBP gap on the INTERCHANGE leg is an interchange downgrade caused by the missing customer reference in the clearing message, so Rule N-4 makes it RATE_RULE_FIXABLE at any age, and the carried state shows no prior route."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Route a card network's settlement gaps to the right team — 120 settlement extracts drawn from 120 real settlement reconciliation extracts. One model answered, and every answer was then graded Three different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection, never a bill and never a vendor's claim about this workload.

Rates checked 2026-08-18. The provider that actually ran the calls is kept out of these tables per this estate's naming rule, so no figure here is a bill.

The gradersThree ways to grade

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

The three free floors, the stub, the cadence arithmetic and every pre-run check cost $0.00 and no calls. ⚠︎ A WHOLE LAP WAS DISCARDED: the 16,000-ceiling scored run, its stateless control, its injection probe and its calibration -- see results/discarded/README.md. Those calls are counted here rather than quietly dropped.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND ALMOST ALL OF THEM ARE REASONING. 94.06 pct of r002-network-settle's output (199347 of 211928 tokens) was provider-side reasoning left at the default. The answer is five short fields and a sentence; the bill is the thinking in front of them.
• THE PROMPT IS MOSTLY FIXED TEXT. 4182 input tokens per reading, and the ten rules, the fee and rate policy block and the instruction are the same on every call -- so a provider with prompt caching would price this workload very differently, and that was not measured.

Your volumeWhat it costs at your volume

LINEAR IN BATCHES x RUNS. Nothing amortises. The sublinear lever NOT implemented is skipping settlement dates that tied exactly.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-query average, this run

The codeEvery file, and what it is for

11 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Six of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/settle.pythe reconciliation rule and state machine — a swap seam

The DEFAULT settlement windows per component, the adjustment-filing window, the runs-open arithmetic and step() -- which decides the next carried state from the answer key's inputs, never from a model reply. It is also the answer key.

You change it to: Reading less often than the settlement window makes every gap look like timing.

# The network-settlement reconciliation rule as arithmetic. Pure code, no model, stdlib only.

PRINCIPAL = "PRINCIPAL"
INTERCHANGE = "INTERCHANGE"
ASSESSMENT = "ASSESSMENT"
CHARGEBACK = "CHARGEBACK"
COMPONENT_NONE = "NONE"
COMPONENTS = (PRINCIPAL, INTERCHANGE, ASSESSMENT, CHARGEBACK)
DEFAULT_SETTLEMENT_WINDOWS = {PRINCIPAL: 1, INTERCHANGE: 1, ASSESSMENT: 2, CHARGEBACK: 3}
DEFAULT_FILING_WINDOW_DAYS = 10
DEFAULT_MATERIALITY_GBP = 25.00

src/state.pythe carried state

The gap as at the previous run, which component carried it, and WHICH RATE RULE was established. A downgrade advice arrives once; on every later run the page shows the same gap with nothing on it.

# The carried state -- the thing that makes this a monitor and not another difference report.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEFAULT_PATH = os.path.join(HERE, "data", "state.json")
def load(path=DEFAULT_PATH):
def save(store, path=DEFAULT_PATH):
def for_batch(store, batch_id):
CAUSE_WORDS = {
def describe(state):

src/segment.pythe section splitter

Splits the page into its 8 named sections; asserted across all 120 documents.

# Split a settlement-reconciliation extract into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Settlement Batch", "Two-Way Position",
RULE = "-" * 64
def split(text):
def render(secs):

src/select.pythe send filter — a swap seam

Merchant Contact is mapped by no field and therefore never sent.

You change it to: SECTION_HINTS and NEVER_SENT.

# Pick which sections of a settlement extract are sent. Pure code -- the last deterministic step

BANNER = "Synthetic Record"
BATCH = "Settlement Batch"
POSITION = "Two-Way Position"
POLICY = "Fee And Rate Rules (Default)"
RECONPOS = "Recon Position"
EVENTS = "Events In This Window"
MERCHANT = "Merchant Contact"
NOTES = "Recon Notes"
NEVER_SENT = (MERCHANT,)

src/prompt.pythe prompt

Three parts: instruction and JSON shape, carried state, extract. The stateless control replaces exactly one line.

# Assemble the one prompt this kit sends. No template engine, no framework -- string concatenation

INSTRUCTION = """\
def build(text, carried, stateless=False):

src/adapters/__init__.pythe model call

Raw HTTP, stdlib only, bounded retry; the shared daily call cap is checked here.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
def _post(url, headers, payload, timeout=120):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}
THINKING_OFF = {"type": "disabled"}
def complete(cfg, system, user, max_tokens=1024, thinking=None):

src/recon.pythe reader

One settlement batch, one recon run, one call. Holds the published token ceiling.

# One settlement batch, one daily recon run, one model call. The runtime half of the kit.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SYSTEM = ("You are a daily card-settlement reconciliation. You read one settlement batch's two-way "
MAX_TOKENS = 32000
FIELDS = ("verdict", "cause", "runs_open", "owner", "route_now")
def documents():
def batches():
def load_doc(doc_id):
def position_of(text):

evals/scoring.pythe scorer

Exact match per cell. The downgrade discriminator is a separate binary with both denominators printed, and the correctly-priced look-alike is scored AGAINST it rather than with it.

# Score a run against the answer key. Pure code, exact match per cell. No model grades anything.

NEEDS_ATTENTION = ("UNMATCHED", "AGED_PAST_FILING", "RATE_RULE_FIXABLE")
FIELDS = ("verdict", "cause", "runs_open", "owner", "route_now")
def _pct(n, d):
def _same(got, want):
def score(records, golds):
def _protection(records, golds):

evals/baseline.pythe three free floors

tolerance-only, window-arithmetic and fee-code-strong -- $0.00 each, and the strongest beats the model on cause over the memory-dependent slice.

# THE FREE FLOORS. Three of them, none a strawman, all of them GBP 0.00.

MODES = ("tolerance-only", "window-arithmetic", "fee-code-strong")
def _section(text, name, nxt, flat=False):
def _facts(text):
def _finish(verdict, cause, runs_open, owner, route_now, why):
RC_CAUSE = (
RC_SETTLED = re.compile(r"RC: SETTLED ")
def review(text, carried=None, mode="fee-code-strong"):

src/app.pythe local UI server

http.server, stdlib. Renders with no key. Port 9020.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
PORT = int(os.environ.get("PORT", "9020"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r002-network-settle")
GOLD_ROWS = {r["doc_id"]: r for r in
def carried_for(doc_id):
class H(BaseHTTPRequestHandler):
def main():

ui/app.jsthe local UI client

Hand-written JS, no framework. Renders the routing DESK for both columns, because two arms can both say 'route it' and send it to different teams.

# 

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 4182 input and 1766 output tokens per reading (one settlement batch, at one daily recon run), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ MEASURED IN BOTH DIRECTIONS, AND THE TWO MEASUREMENTS DISAGREE. Recon Notes is this kit's injection surface. The forced, paired probe found no suppression; the discarded run of the identical 120 readings contains one. Both are published.

API_KEY is read from .env (gitignored) or the real environment; never logged, never placed in a prompt, and redacted from any provider error message the local UI passes to the browser.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. Both were run for real on 2026-08-25.

Two boundaries are measured in both directions here: the injection probe forces the condition rather than waiting for it AND pairs each reading against its own control, and the privacy guard is red-proven by reproducing the platform-migration condition that reaches its fallback.

The result0 of 20 forced, paired trials suppressed -- and 1 confirmed suppression in a discarded run of the same readings. Measured, not assumed, and not resolved.

One phrasing, one model, one corpus, 20 trials -- every reading where suppression was even possible. Fired ONCE. That is the limitation, and it is why the two runs disagree.

HonestyWhat this does not prove

• Whether the suppression is rare or merely under-sampled. The probe fires each reading once.
• Any phrasing but the one shipped.
• Whether a note placed in a section this kit does NOT send would have any effect -- by construction it cannot, and that was not probed either.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never post an adjustment, file a network claim, re-rate a transaction, release a settlement or amend a merchant statement -- and never present the settlement windows, the filing window, the interchange tier names or the assessment rates as any real scheme's, processor's or acquirer's terms.

Stated to the model on every call in src/prompt.py's INSTRUCTION block, and enforced by evals/check_labels.py's banned-code-path scan before any run may spend.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is a PROMPT RULE PLUS A STATIC SCAN, not a runtime enforcement layer. Nothing stops a forker adding such a path tomorrow; the scan catches it only the next time somebody runs check_labels.py, which is a manual step.
• The injection result is ONE SENTENCE against ONE model on ONE corpus -- and this kit's own two runs of it disagree. It is not a resistance rate for prompt injection in general and it is not a resistance rate for this sentence either.

WatchedWhat is watched, and why that one

The kit measures 102 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

4 measured by the latest run 98 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on false_routes, interchange_gap_caught, missed_routes, priced_left_alone, priced_routed, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · with the model in the path — 2 runs. Columns here are only ever compared with each other.

not a time series No two of these 2 runs measured the same system — they differ on batches_caught, batches_missed, false_routes, interchange_gap_caught, interchange_gap_false, missed_routes, stateless — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 12 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 4 chips that all say so.

DeviationsWhat deviated

0 breaches across 7 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

daily -- one settlement reconciliation run every banking morning. The guardrail's banned-code-path scan is a MANUAL step, run before each spend rather than a hook; it ran before every paid run on this kit and passed at 0 each time. The injection probe is a ONE-OFF and it was fired twice, at two different token ceilings, and the two runs disagree.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, no framework dependency, and a prompt anyone can read end to end. A framework abstraction would own the retrieval step -- there is none here -- and the memory layer, which is already the entire surface of src/state.py.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: the reader -> src/prompt.py -> src/adapters -> evals/scoring.py. No branching, no tool-calling and no agent loop.

The other sideWhat a framework costs you

• Swapping providers means editing the PROVIDERS dict by hand. In exchange the prompt has no hidden templating layer and requirements.txt stays empty.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r002-network-settle on the fast tier, with the carried state, 2026-08-25. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 4 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-network-settle-toleranceonly, b001-network-settle-windowarith, b002-network-settle-feecodestrong recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the population is re-read whole on each scheduled run, and the prior run's carried state is what makes a reading a CHANGE.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe verdict, cause, recon runs open, analyst and route-now call, per reading, exact match against the computed answer key

whether each of the 5 answered fields equals the answer key, per reading

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings. No model grades anything.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 120 per field. interchange_gap_accuracy_pct is scored separately over its own denominators.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/settle.step() from the same planted inputs the corpus builder wrote onto the page.

No true/false rates for this grader. It records 5 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold was swept: exact match has no tunable. Denominators are stated beside every rate because 120 readings makes each one wide.

Cadence: once per run; free

The decisionWhen to reach for it

In one lineAn interchange downgrade -- a rate gap fixable AT SOURCE -- as against everything else a settlement gap can be

whether the arm called a gap a fixable downgrade when it was, and did not when it was not

MethodHow the test was run

evals/scoring.py, same run, no extra call.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

caught over the 13 readings whose truth IS one; false positives over the 107 that are not; and a BAND figure over all 120, printed only beside the other two.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl's interchange_gap column, computed by src/settle.step().

No true/false rates for this grader. It records 2 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold. Exact verdict match.

Cadence: once per run; free

The decisionWhen to reach for it

In one lineA correctly-priced cross-border assessment -- explained by a rate rule AND correct

whether the arm named the look-alike correctly and routed it to nobody

MethodHow the test was run

evals/scoring.py, same run, no extra call.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

named correctly over the 13 readings whose truth is RATE_RULE_PRICED; and routed-to-nobody over the same denominator.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl's priced_lookalike column.

No true/false rates for this grader. It records 2 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold.

Cadence: once per run; free

The decisionWhen to reach for it