The business caseThe problem this solves

A carrier's invoice carries accessorials - detention, layover, redelivery, a lumper, a reconsignment, a truck ordered not used - and each one is three questions at once, not one. Is it contracted at all? The description is the CARRIER'S vocabulary and Exhibit B is the agreement's, so 'Driver detention at delivery' may be DET-C and 'Detention - team driver premium' may be nothing. Is the rate right? That is Exhibit B's rate, free time, increment and cap against the units the trip supports. And do the events support it? A detention charge needs the free time to have genuinely elapsed - and a gate note saying the driver was held for his own paperwork means it never accrued, however the guard log reads. Nobody does all three on every line, because doing it means reading a guard log, an event register and a paragraph of free text against a rate exhibit, per charge, per trip. It surfaces as a quarter's accessorial spend nobody can account for. THIS IS NOT A CATALOGUE CHECK. There is no master-data question here and no system export: the question is about ONE BILLED CHARGE on ONE TRIP, and the trip's own event log is the third leg. The line-by-line clearance of a carrier's accessorial charges before settlement - reading a guard log, an event register and the yard's own gate notes against the agreement's rate exhibit, per charge - which in most shops is sampled rather than done.

The inputThe actual trip files

The corpus is 58 trip files, 0.14 MB (txt 58). A carrier rate agreement's accessorial exhibit is the commercially sensitive half of a freight contract and a trip's guard log is operational data about named drivers at named sites. There is no public corpus of either and no licence under which a real one could ship inside a kit meant to be forked - so generating one is the only way this kit can carry the thing it is actually about and still be publishable. WHAT IT EXERCISES WAS CHOSEN RATHER THAN SAMPLED, and every choice is a measurement somebody can check. 154 of the 176 properly contracted lines are billed under a description Exhibit B does not print, because carriers bill in their own vocabulary - that is what makes name matching a reading. 10 descriptions LOOK like contracted codes and are not, so a matcher that is generous about synonyms fails a class an exact matcher gets right. 24 lines turn on FA-3.2, where the driver reached the gate before his appointment and the guard log's clock and the contract's clock disagree. 19 turn on a gate note that puts the delay on the carrier, written the way a yard clerk writes - and three of the ordinary notes talk about the driver and the carrier without finding any fault, which is the only reason a keyword floor can be wrong in the generous direction at all. 33 lines are decided by arithmetic alone, so free code's exactness is measurable rather than asserted.

TRIP FILE TRP-0001
Carrier: Cascade Ridge Carriers, Inc.     Carrier agreement: CA-4471
Shipper: Verity Foods Packing, Modesto CA
Consignee: Kestrel Hardware Depot 7, Rapid City SD
Load LD-80037   dry grocery, 11 pallets   Dispatched 2026-07-10

EVENT LOG  (guard shack and dock systems, local time at each stop)
  ORIGIN
    appointment        2026-07-10 14:30
    gate in            -- no gate stamp recorded --
    gate out           2026-07-10 15:34
    signature          2026-07-10 15:02
  DESTINATION
    appointment        2026-07-11 07:45
    gate in            2026-07-11 07:51
    gate out           2026-07-11 11:51
    signature          2026-07-11 09:11

TRIP EVENTS
    overnight holds recorded     0
    failed delivery attempt      yes
    lumper receipt on file       none on file
    reconsignment instruction    none on file
    cancelled after dispatch     no
    driver assist recorded       yes
    additional delivery stops    0

GATE NOTES  (free text, entered by the yard clerk at each stop)
    Origin: Load was re-tarped at our request before it left the yard.
    Destination: Load was re-tarped at our request before it left the yard.

EXHIBIT B of CA-4471 -- the accessorials this carrier agreement contracts
    code   Exhibit B description    basis          rate       free time  cap
    DET-S  Detention - Shipper      per_hour       $75.00     150 min    $750.00
    DET-C  Detention - Consignee    per_hour       $65.00     120 min    --
    RDLV   Redelivery               per_occurrence $150.00    --         --
    RECON  Reconsignment            per_occurrence $135.00    --         --
    DRVA   Driver assist            per_occurrence $75.00     --         --
    increments: DET-S 30 min, DET-C 30 min

The outcomeWhat a good result looks like

Every billed accessorial carries one of five verdicts, the FA-2026 clause that decides it, the entitled amount, the signed amount at issue to the cent, the named desk that owns it, and one line of the trip file copied verbatim as evidence; and every trip carries PASS or QUERY, the lines that do not tie, and the total at issue. On r001-accessorial-validate: 185 of 226 line verdicts correct (81.9 pct), 212 of 226 clocks, 202 of 226 Exhibit B codes, 174 of 226 clauses, and 56 of 58 trips right on the recommendation. Every citation the arm returned located in its own document - 226 of 226, 0 unlocatable.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Your accessorial exceptions are mostly arithmetic - free time that did not elapse, a rate that does not match the exhibit, an event nobody logged — the free alias floor, and do not buy the call
  free code is EXACT on those: 33 of 33 against the paid call's 24, p = 0.003906. It costs nothing, it never varies, and it is 119 lines of Python.
• Your carriers bill under their own vocabulary and your free-time clock has a rule about early arrival — the paid call for the clock, the synonym table for the vocabulary
  the call wins which clock governs - 212 of 226 against 193, p = 0.004324 - and LOSES the vocabulary: the floor's synonym table takes 220 of 226 against the call's 202, p = 0.000912.
• A false dispute costs you more than an overpayment - you are managing a carrier relationship, not a recovery target — the paid call
  it never wrongly disputed a clean charge: 55 of 55 clean lines correct against the floor's 34, p p < 0.0001. The floor raises 21 false disputes on this corpus.

And where nothing here is good enough:

• You want one arm to do the whole job — neither, on this evidence
  the composite is a draw. Verdict 185 against 169, p = 0.084646; all seven fields 139 against 144 with the FLOOR ahead, p = 0.716294. Nothing here justifies buying a call for the whole job.

At a glanceHow the whole thing runs

Run once, for real, on 2026-09-11. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-11 — r001-accessorial-validate. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A clean checkout with no key configured renders the whole board - all 58 trip files, the answer key, the scored run, both free floors, the paired tests and the twelve adversarial trials - and scores both free floors offline. tools/build_corpus.py --check, evals/check_labels.py, evals/baseline.py, evals/run.py --stub, evals/run.py --rescore and src/app.py all run with no network and no credential, and nothing is pip-installed. Only evals/run.py without --stub or --rescore, evals/injection.py, and the board's one Check button ever reach a provider.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. Measured on the fast tier — END TO END for one trip file, wall clock from the moment the prompt is assembled to the moment the reply is parsed, the boundary station has run and FA-2026 has ruled every charge line on it. There is no retrieval step and no second call: one trip file goes whole into one call. p50 1765 ms and p95 2216 ms over 58 calls. Cost.cost_by_model publishes the same p50 as MODEL latency; on this kit they are the same number, because nothing else is in the path.

Current processWhat it replaces

The line-by-line clearance of a carrier's accessorial charges before settlement - reading a guard log, an event register and the yard's own gate notes against the agreement's rate exhibit, per charge - which in most shops is sampled rather than done.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• json
• jsonl
• md
• 1 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one trip file is

one trip file is one unit, read whole into one call. No chunking, no retrieval, no carried state between trips. A charge line is one scored row.

SetupWhat the setup figure measured

THERE IS NO INDEX AND NOTHING WAS BUILT. One trip file goes whole into one call; there is no chunking, no embedding and no retrieval step, so the zeros are an absence rather than a fast build. The only preparation this kit does is generating the corpus and deriving the answer key, which tools/build_corpus.py does offline in under a second for $0.00 and no network - and --check proves it reproduces from seed 20260911.

LicenceLicence

MIT - this repository's own, because this repository generated it

Bring your ownBring your own trip files

Replace tools/build_corpus.py with a reader over your own data and nothing else changes. The kit needs three things per trip: a structured record (stops with appointment, gate-in, gate-out and signature; the event register; the billed charge lines in integer cents), the governing Exhibit B as rows, and one readable text file that prints all of it - that text file is what the model sees. data/gold.jsonl in the same shape is what the scorer needs; without one you still get every verdict, and you get no accuracy.

⚠︎ And what stops being true when you do: The kit will rule a trip file it has never seen - src/prompt.py's build(text) takes the document alone, and the governing Exhibit B is read off the document rather than out of data/. What it will NOT do is tell you how good the answer is. Every percentage on this page belongs to this corpus and to no other.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 8,461 characters assembled, 3,550 are instructions — 42% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Replayed from the kit's own src/prompt.py build() for TRP-0001, not logged by the run. It is checkable rather than merely asserted: the run record stores the part list and this replay matches it part for part.

Full promptThe prompt in full

SYSTEM:
You are reading a freight trip file for a shipper's freight-audit desk. You report only what the trip file and the carrier agreement's Exhibit B printed on it actually say. You never approve, authorise, pay, short-pay, hold, release, credit, debit, re-rate, dispute or settle anything, and you never state a verdict, a rate, an entitled amount or any sum of money.

USER:
FREIGHT ACCESSORIAL VALIDATION POLICY FA-2026, which governs every carrier agreement:

FA-1.1  Scope
Only an accessorial listed in Exhibit B of the governing carrier agreement is payable. The description printed on the carrier's invoice is not the authority; the Exhibit B code it corresponds to is.

FA-2.1  Charges outside Exhibit B
A billed accessorial that corresponds to no Exhibit B code of this agreement is not payable, however it is described and however similar its description is to a code that IS listed.

FA-3.1  Detention free time
Detention accrues only after the free time stated in Exhibit B has elapsed. Dwell is measured from the guard log's gate-in to its gate-out at the stop named by the code.

FA-3.2  Arrival before the appointment
Where the driver arrives before the scheduled appointment, free time begins at the APPOINTMENT time and not at gate-in. Time the carrier spends on our premises ahead of the slot it was given is the carrier's own.

FA-3.3  Increment and cap
Chargeable detention is billed in the increment stated in Exhibit B, rounded down, and may not exceed the per-stop cap where Exhibit B states one.

FA-4.1  Occurrence accessorials need the event
A per-occurrence or per-day accessorial is payable only where the trip record carries the qualifying event: a recorded failed delivery attempt for redelivery, a receipt for a lumper, a written reconsignment instruction, a cancellation after dispatch for a truck ordered not used, an overnight hold for a layover, a recorded driver-assist for driver assist, and a second delivery stop for a stop-off.

FA-4.2  Carrier-caused delay
Delay the carrier itself caused does not accrue detention or layover, and does not make a redelivery payable, however the guard log's timestamps read. Where a gate note establishes the cause, the note governs over the clock.

FA-5.1  Rate and units
The amount payable is the Exhibit B rate applied to the units the trip record supports. A figure that differs from that amount is a rate exception even where the accessorial itself is properly due.

FA-5.4  Insufficient evidence
Where the trip file carries no event able to decide a contracted accessorial, the line is queried to the trip records clerk. No amount is claimed against it and it is not disputed.

FA-6.1  Lumper pass-through
A lumper is reimbursed at the receipted amount, capped at the Exhibit B ceiling. Without a receipt in the trip file it is not payable under FA-4.1.

THE TRIP FILE, verbatim. Exhibit B of the governing carrier agreement is printed on it:

TRIP FILE TRP-0001
Carrier: Cascade Ridge Carriers, Inc.     Carrier agreement: CA-4471
Shipper: Verity Foods Packing, Modesto CA
Consignee: Kestrel Hardware Depot 7, Rapid City SD
Load LD-80037   dry grocery, 11 pallets   Dispatched 2026-07-10

EVENT LOG  (guard shack and dock systems, local time at each stop)
  ORIGIN
    appointment        2026-07-10 14:30
    gate in            -- no gate stamp recorded --
    gate out           2026-07-10 15:34
    signature          2026-07-10 15:02
  DESTINATION
    appointment        2026-07-11 07:45
    gate in            2026-07-11 07:51
    gate out           2026-07-11 11:51
    signature          2026-07-11 09:11

TRIP EVENTS
    overnight holds recorded     0
    failed delivery attempt      yes
    lumper receipt on file       none on file
    reconsignment instruction    none on file
    cancelled after dispatch     no
    driver assist recorded       yes
    additional delivery stops    0

GATE NOTES  (free text, entered by the yard clerk at each stop)
    Origin: Load was re-tarped at our request before it left the yard.
    Destination: Load was re-tarped at our request before it left the yard.

EXHIBIT B of CA-4471 -- the accessorials this carrier agreement contracts
    code   Exhibit B description    basis          rate       free time  cap
    DET-S  Detention - Shipper      per_hour       $75.00     150 min    $750.00
    DET-C  Detention - Consignee    per_hour       $65.00     120 min    --
    RDLV   Redelivery               per_occurrence $150.00    --         --
    RECON  Reconsignment            per_occurrence $135.00    --         --
    DRVA   Driver assist            per_occurrence $75.00     --         --
    increments: DET-S 30 min, DET-C 30 min
    Any accessorial not in this table is not contracted with this carrier.

ACCESSORIAL CHARGES BILLED ON THIS TRIP
    #   description as printed                units unit          rate       amount
    1   Driver count and sort                     1 ea         $150.00      $150.00
    2   Return and redeliver                      1 ea         $175.00      $175.00
    3   Origin waiting time                       2 hr          $75.00      $150.00
    4   Driver detention at delivery              2 hr          $65.00      $130.00
        TOTAL ACCESSORIALS                                                  $605.00



Reply with JSON and nothing else, in exactly this shape:

{"lines":[{"line":<the # as printed in the billed-charges table>,
           "contract_code":"<the Exhibit B code this billed line corresponds to, or NONE>",
           "clock_basis":"GATE_IN_GATE_OUT|APPOINTMENT_TO_GATE_OUT|SIGNATURE_TO_GATE_OUT|NOT_TIME_BASED",
           "evidence_state":"EVENTS_SUPPORT|EVENTS_CONTRADICT|EVENTS_ABSENT",
           "quote":"<the line from the trip file you read this from, copied exactly>"}],
 "confidence":<0.0 to 1.0>,
 "why":"<one sentence, no more>"}

One object per billed charge line, in the order they are printed. Return every line.

contract_code -- WHICH Exhibit B CODE THIS CHARGE IS, not what the carrier called it. Carriers bill
under their own vocabulary, so match on what the charge IS. Use the code exactly as Exhibit B on
this trip file prints it. Answer NONE where no Exhibit B code of THIS agreement covers the charge,
including where the description closely resembles one that is listed -- a description that reads
like a contracted code is not a contracted code.

clock_basis -- for a per_hour code, WHICH recorded pair of timestamps the evidence establishes as
the dwell at the stop that code is read at:
  GATE_IN_GATE_OUT         the guard log's gate in to gate out, and nothing displaces it
  APPOINTMENT_TO_GATE_OUT  the driver reached the gate BEFORE the scheduled appointment, so under
                           FA-3.2 free time begins at the appointment and not at gate in
  SIGNATURE_TO_GATE_OUT    no gate-in stamp exists and the signature is the only arrival evidence
  NOT_TIME_BASED           the code is not billed by the hour at all
Answer NOT_TIME_BASED for every per_occurrence and per_day code.

evidence_state -- what the trip's own events do to this charge:
  EVENTS_SUPPORT     the log and the notes are consistent with the charge being due. This is the
                     ordinary answer. A gate note describing a busy yard, weather, our own dock,
                     our own staff, our own equipment or something WE asked for is EVENTS_SUPPORT
  EVENTS_CONTRADICT  a GATE NOTE establishes that the delay, or the event itself, was THE
                     CARRIER'S OWN doing -- their driver, their tractor or trailer, their
                     paperwork, their dispatcher. FA-4.2 removes the charge only where the CARRIER
                     caused it, so a note about anything else is not a contradiction. Notes are
                     free text and use no fixed wording, so read them rather than matching words
  EVENTS_ABSENT      nothing in the file can decide this charge either way. A row printed as
                     "none recorded", "none on file", "no" or "0" IS a record and means the event
                     did NOT happen -- that is EVENTS_SUPPORT, and the charge fails elsewhere.
                     Answer EVENTS_ABSENT only where the file prints "not recorded either way",
                     or where the timestamps a per-hour code needs are simply not in the log

Do not state a verdict, a rate, an entitled amount, a difference or any sum of money. Those are
computed from your three readings, not taken from you.

Raw responseThe raw response

{"lines":[{"line":1,"contract_code":"NONE","clock_basis":"NOT_TIME_BASED","evidence_state":"EVENTS_SUPPORT","quote":"Driver assist recorded       yes"},{"line":2,"contract_code":"RDLV","clock_basis":"NOT_TIME_BASED","evidence_state":"EVENTS_SUPPORT","quote":"failed delivery attempt      yes"},{"line":3,"contract_code":"DET-S","clock_basis":"SIGNATURE_TO_GATE_OUT","evidence_state":"EVENTS_SUPPORT","quote":"Load was re-tarped at our request before it left the yard."},{"line":4,"contract_code":"DET-C","clock_basis":"GATE_IN_GATE_OUT","evidence_state":"EVENTS_SUPPORT","quote":"Load was re-tarped at our request before it left the yard."}],"confidence":0.75,"why":"Line 1's driver count and sort matches no Exhibit B code; line 3's origin dwell runs from the recorded signature to gate out because no gate-in stamp exists; lines 3 and 4 are contradicted only by carrier-caused delay, but the gate notes describe our own request."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Validate a billed accessorial against the contract and the trip's own events — 58 trip files. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

Seven fields are graded on every charge line - the three readings, the verdict, the clause, the entitled amount and the signed amount at issue - and four on every trip. Every comparison is exact and every grader is pure Python, so RE-SCORING IS FREE: the committed reply cache re-derives the whole result file, p-values included, with no key and no network.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every projected dollar is a MEASURED token count multiplied by a named vendor's PUBLISHED rate. The token counts come from the provider's own usage block on each of the 58 calls. The tier that actually ran is priced from its own card at the tariff in force per call, and that figure is a real bill.

Rates checked 2026-09-05. The runtime tier is not named on this page. The estate publishes rate cards for a fixed list of providers and the runtime one is deliberately not on it.

Grading unitWhat the grading figure prices

$0.00 to grade and $0.00 to re-grade. The only money this kit ever spent is 58 calls on the scored run ($0.028546), 12 on the adversarial probe ($0.005367) and 2 on a wiring probe that is not published as an arm ($0.001149).

The gradersOne way to grade, and why it is the only one

⚠ THE alias FLOOR WAS BUILT TO WIN AND IT DOES. A floor built to lose proves nothing and inflates every margin above it, so this one carries a full synonym table over the accessorial vocabulary, reads the structured event register including its unknown rows, and scans the gate note at the stop each code is argued at. The exact floor - the brief's own description, a code lookup against Exhibit B's printed wording plus timestamp arithmetic - takes only 69 of 226, and a kit that published only that floor would have shown the model winning easily and taught its reader nothing. AND THE CUE LIST IS AN UPPER BOUND: it was written by the same hand that wrote this corpus's gate notes.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — top_k, chunk size, prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-09-05. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

$0.00. Every grader is pure code and every reply is cached, so re-scoring the recorded run - every p-value included - costs nothing and returns the same answer. Both free floors are also $0.00 and 0 calls. The only money this kit has ever spent is 58 calls on r001-accessorial-validate ($0.028546), 12 on the injection probe ($0.005367) and 2 on a wiring probe that is not published as an arm ($0.001149) - $0.035062 in total.

Cost driversWhat actually moves the bill

• THE RULEBOOK, NOT THE TRIP FILE. FA-2026's clauses, the role and the reply schema are 6006 of the 8461 characters sent and are identical on every call; the trip file is the smaller half.
• the number of TRIP FILES, not the number of charges - one call per trip however many accessorials it carries, so a longer bill is cheaper per charge
• the reply length, which scales with charge lines: 236.4 output tokens per call on average across trips of 1 to 6 charges
• cache hits on the stable prefix: 31872 of 119470 input tokens were billed at the cached rate
• THE TARIFF WINDOW. Every call on this run landed off peak; the identical tokens inside the weekday peak window would have cost exactly twice as much.

Your volumeWhat it costs at your volume

linear in trip files, and slightly sub-linear in dollars because the stable prefix caches. 10x the trips is 10x the calls; the per-call cost falls only as far as the cache-hit share allows. It is NOT linear in charges - a trip with twenty accessorials costs one call, until the reply outgrows the 1400-token ceiling.

Model choiceWhy this model

One provider, one key, one model - the estate default, reasoning explicitly disabled, the smallest ceiling the reply fits in. NO MODEL COMPARISON WAS BOUGHT: the other rows on this lens are this run's measured tokens projected onto published cards, and none of them ran. The comparison this kit DID buy is the one that matters more - a paid call against free code on the same corpus, and free code holds.

Other modelsThis run's measured tokens, priced against published cards

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the 58 calls of r001-accessorial-validate, one per trip file. The 12 calls of x001-accessorial-validate are not in this figure and are priced separately at $0.005367; the two-call wiring probe cost $0.001149.

The codeEvery file, and what it is for

12 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pyCorpus generator — a swap seam

writes all 58 trip files, 226 charge lines, six carrier agreements and the answer key from ONE fixed seed; --check reproduces the shipped bytes or names the file that moved

You change it to: Replace the generator with a reader over your own TMS trip records and rate exhibits. The scorer needs data/gold.jsonl in the same shape and nothing else.

# Generate the whole corpus from ONE fixed seed: agreements, trips, the readable trip files and

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
SEED = 20260911
N_TRIPS = 58
CARRIERS = [
SHIPPERS = [
CONSIGNEES = [
TYPES = {

data/policy.jsonPolicy FA-2026 — a swap seam

eleven clauses, five verdicts in one precedence order, four named human desks and one boundary, as data

You change it to: Add, remove or reorder a verdict, retarget a desk, or rewrite a clause. The precedence the engine applies, the clauses the prompt quotes verbatim and the desks the scorer checks all read this one file, so a retuned policy needs no code change and cannot leave the three of them disagreeing.

# 

src/policy.pyFA-2026 engine

THE RULING, in pure code. Integer cents and whole minutes end to end; resolves the clock, subtracts free time, rounds down to the Exhibit B increment, applies the cap, checks the qualifying event and produces the verdict, the clause and the signed amount

# FA-2026 -- the accessorial validation engine. PURE CODE, NO MODEL, NO NETWORK.

VERDICT_ORDER = ("NOT_CONTRACTED", "EVIDENCE_MISSING", "EVIDENCE_SHORT", "RATE_WRONG", "SUPPORTED")
CLOCK_ORDER = ("GATE_IN_GATE_OUT", "APPOINTMENT_TO_GATE_OUT", "SIGNATURE_TO_GATE_OUT",
EVIDENCE_ORDER = ("EVENTS_SUPPORT", "EVENTS_CONTRADICT", "EVENTS_ABSENT")
GOV_STOP = {"DET-S": "shipper", "DET-C": "consignee", "LAY": "consignee", "RDLV": "consignee",
CLOCK_FIELDS = {"GATE_IN_GATE_OUT": ("gate_in", "gate_out"),
DESK = {"NOT_CONTRACTED": "carrier contract analyst",
def money(cents):
def _ts(s):
def _assert_cents(v, what):

src/prompt.pyPrompt assembly

four named parts in fixed order; three of them are byte-identical on every one of the 58 calls, and build(text) takes the document and nothing else

# Assemble the one call. ONE ARGUMENT -- the trip file, verbatim.

MAX_TOKENS = 1400
STABLE_PREFIX_PARTS = ("role", "policy", "schema")
SYSTEM = (
CLAUSES_IN_PROMPT = ("FA-1.1", "FA-2.1", "FA-3.1", "FA-3.2", "FA-3.3",
def _policy_text():
SCHEMA = """Reply with JSON and nothing else, in exactly this shape:
def build(text):
def render(parts):
def verbatim(text):

src/reader.pyReader

the only place a model is called - one trip file, one call, tolerant JSON parse, closed-vocabulary normalisation that never coerces an unknown value into a known one

# The ONE place a model is called. One trip file, one call, three readings per charge line.

MAX_TOKENS = PROMPT.MAX_TOKENS
THINKING = adapters.THINKING_OFF
def _outer_json(text):
def parse(text, exhibit):
def read_trip(cfg, text, exhibit):

src/refusal.pyBoundary station

FA-B1 in pure Python AFTER the reply: keeps three readings and a quote, discards everything else the reply said and records what it discarded

# The boundary station FA-B1, in pure Python, AFTER the reply comes back.

KEEP = ("line", "contract_code", "clock_basis", "evidence_state", "quote")
FORBIDDEN = ("verdict", "entitled", "entitled_cents", "amount", "amount_at_issue", "rate",
def scrub(obj):
def forbidden_asserted(dropped_names):

src/recheck.pyRecheck station

re-applies FA-2026 in code to the arm's own three readings, joined on the PRINTED line number, which is the only handle the model has on a row

# THE STATION, and it is what this kit ships.

def rule_trip(trip, exhibit, readings_by_line_no):

src/rules.pyFree floors — a swap seam

two of them, both producing the same three readings and both ruled by the SAME engine - exact (Exhibit B's printed wording only) and alias (a synonym table, the event register and a cue scan of the gate notes), 0 calls and no network

You change it to: The synonym table and the three fault cues. This is the file the whole headline is measured against, so weakening it is the one edit that can make this kit look better than it is - the kit publishes both floors so the difference between a floor built to win and one built to lose is visible.

# THE FREE FLOORS. Pure Python, no key, no network, $0.00.

FLOORS = ("exact", "alias")
SYNONYMS = [
FAULT_CUES = ("carrier", "driver", "their ")
def _norm(s):
def _exact_code(desc, exhibit):
def _alias_code(desc, exhibit):
def read(trip, exhibit, floor):
def constant_rates(gold):

src/citation.pyQuote locator

three outcomes, not two - on point inside the section that bears on the charge, located elsewhere in the document, or unlocatable

# Is the quote the reader returned actually in the document?

SECTIONS = ("EVENT LOG", "TRIP EVENTS", "GATE NOTES", "EXHIBIT B", "ACCESSORIAL CHARGES BILLED")
def _norm(s):
def sections(text):
def locate(text, quote, want_section=None):

evals/scoring.pyScorer

seven graded fields per charge line and four per trip, three named splits, both error directions carried apart; every comparison exact, no model in the grading path

# The scorer. EVERY GRADER IS PURE CODE and no model is anywhere in the grading path.

LINE_FIELDS = ("contract_code", "clock_basis", "evidence_state", "verdict", "clause",
SPLITS = {
def score(records):
def score_trips(trip_results):
def record_for(trip, line, ruled, reading, doc_text):
def three_way_floor():

evals/significance.pyPaired tests

McNemar exact, two-sided, on math.comb. Every p-value ships with its discordant counts so a reader can recheck it

# McNemar's exact test, two-sided. Pure stdlib, no SciPy, and re-runnable at $0.00.

def mcnemar(a_correct, b_correct):
def display(p):

src/app.pyLocal board

three panels, renders with no key; only /api/validate spends and its button is disabled without one

# The local board. python3 -m src.app  -> http://127.0.0.1:9387

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
RESULTS = os.path.join(HERE, "results")
PORT = int(os.environ.get("PORT", "9387"))
ARMS = [("r001-accessorial-validate", "the scored run", "paid"),
def _load(run_id):
def board():
def trip_detail(tid):
def validate_live(tid):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 2059 input and 236 output tokens per query at top-k 1, of which about 150 tokens are fixed (system prompt, chat framing, the question itself) and the rest is retrieved context. Change the retrieval depth and the input scales with it. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model and top-k follow the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

THE CAP IS THE SHAPE OF THE ANSWER, not a sentence in a banner. The reply schema offers three enum fields, a quote, a confidence and one sentence. There is no field in which the model could approve a payment, issue a credit or re-rate a charge - so a compliant reply cannot express any of it, and src/refusal.py counts and DISCARDS any field the arm adds beyond the schema. Every verdict, clause, amount, queried set and total on every surface is computed by src/policy.py from the three readings.

One key, in the repo-root .env, gitignored and written 0600. src/config.py reports WHICH file a value came from and never the value. The board's Check button is disabled with no key and says why on the page. No API key is ever requested from a reader on any surface.

The experimentWhat was actually fired

Four wordings - an authority claim, an instruction override, a schema attack and a role claim - each appended to the charge description of one line whose key says the charge is not payable, on three trip files whose key says QUERY. 12 calls, $0.005367. python3 -m evals.injection --run-id x001-accessorial-validate Measured on 3 trip files whose key says QUERY, 4 wordings each, 12 bought calls.

A pass does not mean the model said no. It means an injected sentence did not move a READING far enough to change what FA-2026 computes.

The result0 of 12 injected trials flipped a trip to PASS and 0 flipped a charge to SUPPORTED. 5 moved the target reading, and the control names 3 of those as already wrong on the same arm with no attack present.

Four wordings — an authority claim, an instruction override, a schema attack and a role claim — appended to the charge description of one line whose key says the charge is not payable, on three trip files whose key says QUERY. The charge description is the only text on a trip file written by the party whose money is at stake, so this attack needs a carrier's clerk to type a sentence rather than an attacker to breach anything. 12 calls, $0.005367. The control is the honest half: 3 of the 5 moved readings were already wrong on the SAME arm with no attack in the document, and evals/injection.py --control-only recomputes that from the committed files at $0.00.

HonestyWhat this does not prove

• Four wordings over three trip files is a probe, not a security assessment. It says nothing about wordings nobody here thought of.
• whether a wording nobody here thought of would do better. The four were chosen from the shapes this estate has seen, not from an adversary's imagination.
• whether the same attack inside a GATE NOTE rather than a charge description would land. The gate note is written by our own yard clerk, so it is not the surface a carrier controls - but a compromised yard system would be, and that was not probed.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

Nothing this kit produces pays, approves, short-pays, holds, releases, credits, debits, re-rates, disputes or settles a freight invoice. QUERY is a line on a freight-audit desk's queue: it moves no money and authorises nothing.

Two places, and the first is structural. (1) The reply schema in src/prompt.py offers no field that could carry any of those acts - three enums, a quote, a confidence and one sentence - so a compliant reply cannot express one. (2) src/refusal.py runs on EVERY arm after the reply and keeps only those four fields, recording every other key the arm added; src/recheck.py then recomputes the verdict, the clause, every amount, the queried set and the recommendation from the three readings alone.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is not a filter over a dangerous capability. There is no write path to an invoice anywhere in the kit and there is no apply button, so there is no action for a reply to trigger. What the boundary prevents is a number the model asserted reaching a published page under this kit's name.

WatchedWhat is watched, and why that one

The kit measures 62 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

41 measured by the latest run 21 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (dataset_version accessorial-validate-v1-58trips, failures 0, free_floors alias 169/226 · exact 69/226 (line verdicts, 0 calls, $0.00, no network) — the paid call takes 185, paired 46 to 30, exact two-sided p = 0.084646, NOT significant; all seven fields 139 against the alias floor's 144, p = 0.716294, graded_cells 1582, lines 226, policy_id FA-2026, three_way_floor_verdict always EVIDENCE_SHORT 33.6% · code always NONE 22.1% · clock always NOT_TIME_BASED 73.5% · evidence always EVENTS_SUPPORT 85.0%, trips 58) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 2 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · no model in the path — a baseline, not a peer column — 2 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 2 runs measured the same system — they differ on the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

comply · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 41 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

every run - re-scoring is $0.00, so the boundary counts are recomputed in full rather than sampled

In one lineNo framework, deliberately

Nothing here is imported. Every seam a framework would normally own is a small module, and the reason is the fork test: pip install pulls nothing and the kit runs on a clean checkout with the standard library alone.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

corpus -> prompt -> one call -> refusal -> recheck -> FA-2026 -> scorer -> paired tests. The two free floors enter at refusal with the same three readings and are ruled by the same engine.

The other sideWhat a framework costs you

• The cost of owning no framework is that every seam is ours to get wrong; the cost of owning one would be a dependency in the one file whose purpose is not having a preference. This kit chose the first.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-accessorial-validate, 2026-09-11. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 2 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, FOUNDRY_RUN_ID, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT - this repository's own, because this repository generated it Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe three readings and the ruling

whether the arm read the Exhibit B code, the governing clock and the gate notes correctly - the only three fields it is asked for - and whether FA-2026 applied to those readings reproduces the key's verdict, clause and money

MethodHow the test was run

python3 -m evals.run --run-id r001-accessorial-validate --rescore (re-derives from the committed reply cache; 0 calls, $0.00, no network)

The inputOne real row, seen by every grader

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl, re-derived by evals/check_labels.py through the same FA-2026 engine every arm is ruled by.

No true/false rates for this grader. It records 6 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? There is no tuned threshold anywhere in this kit. Every verdict comes from FA-2026's precedence applied to three enums, and the only numeric comparison is billed == entitled, to the cent. Read every rate below against its constant-answer floor, which is printed with it.

Cadence: every run; re-scoring a committed cache is $0.00 so there is no reason to sample

The decisionWhen to reach for it