The business caseThe problem this solves

A unit comes back and somebody has to say what happens to it -- scrap it, repair it, send it to the supplier, or hold it -- and then say what decided that. Almost none of it is a judgement call. It is a month count from purchase to receipt against the model's warranty, three tests against a clause somebody signed, a join onto the desk's own closed repair orders inside a 36-month lookback, and one percentage against a replacement value. What makes it slow and what makes it wrong are different things: it is slow because those four lookups sit in four places, and it is wrong because the loudest sentence in the box is the customer's account of the fault and the fact that governs is the inspection finding. On 23 of these 60 units the two point in opposite directions. The desk's manual pass over one returned unit: reading the dossier, counting the months from the purchase date to the receipt date, checking that count against the model's warranty and against each of its supplier's clauses, counting the days the claim clock has run, looking the serial up in the repair history and deciding which of those repairs fall inside the lookback, comparing the estimate with 60 pct of the model's replacement value, and then applying the four-way precedence. It does not replace the person: the record is what they confirm.

The inputThe actual returned units

The corpus is 60 returned units, 0.02 MB (txt 60). The cases are the ones that make this decision hard, and none of them can be scraped. A customer who writes 'I dropped it' on a unit whose main-board joint has a void in it. A clause wide open on age and already shut on a claim clock that started when the unit landed. Three repairs on a serial, two of them four years ago and outside the lookback. An estimate of 53.40 against a threshold of 53.40. A unit nobody has opened yet, whose bench note says 'not stripped yet -- it is queued behind the batch from last week'. Every one of those is a boundary you can only plant, because the paperwork that would show it names a customer, a serial and a supplier's agreement. The mix is CHOSEN -- 17 clean and 43 planted -- so a rate here is a statement about this corpus and about nothing else.

From: returns@meridian-service.example
To: returns.desk@calderfield-works.example
Subject: [RMA RT-0001] returned unit, disposition required

RETURNED UNIT
  RMA reference        RT-0001
  Received at the desk 28 Jun 2026
  Model                CW-522  Countertop blender
  Serial               SN-522-713257
  Date of purchase     28 Feb 2024
  Reported fault       leaking
  Repair estimate      USD 49.02

  Inspection result    On the bench, liquid has got inside the unit and tracked across the board.

  Customer packed it in the original carton with the accessories.

The outcomeWhat a good result looks like

One disposition record a person confirms instead of a dossier they read: the disposition, the clause id that decided it, the repair money to release to the cent, whether the unit is inside its warranty, whether a supplier term is open, and one sentence naming the clause. Every arithmetic field is re-derived in pure code from data/rulebook.json and data/history.json, so the record can be checked against the rulebook rather than believed.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Dossiers whose inspection finding arrives as a machine CODE -- the CSV feed row and the returns-portal export — the free rules floor alone
  On all 30 coded units the floor reads the finding 30 of 30, exactly as the paid arm does, and every number after that is the same engine on both arms. It also gets in_warranty 60 of 60, all four safety holds, the three lookback units and the other five read fields 60 of 60 -- nine measurements where the paid call buys a tie, and on one of them (in_warranty) the raw paid arm is a row BEHIND.
• Dossiers where a person WROTE what the bench found -- the RMA email and the bench notes — the model, and then the recheck
  30 of 30 against the floor's 16 of 30, and the floor's 14 misses are structural rather than tunable: where the finding is a sentence it falls back to the customer's reported fault through the rulebook's own suggests table, and four findings have no fault that points at them. Ten of those fourteen change a graded field, which is the entire measured case for the call on this corpus.
• The unit where the customer's account and the bench point in opposite directions — the model -- and it is the only place this corpus separates the arms at all
  23 of 23 against the floor's 14 of 23, and 3 of 3 against the floor's 0 of 3 on the fault_disagreement_rtv family, where the customer says 'I dropped it' and the bench found a void under a main-board joint on a clause with years to run. Every one of the floor's seven false authorises and its one false withhold is here or in its neighbours.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-31. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-31 — r001-return-disposition. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — ⚠︎ PARTIALLY EVIDENCED, AND THE REST IS NOT MEASURED. What is on disk and checkable: the 60 dossiers, data/returns.json, data/history.json, data/rulebook.json, data/fields.json, the 60-row data/gold.jsonl, data/corpus-stats.json, and three committed run records -- the free floor, the stub arm and the paid run. requirements.txt pulls nothing at runtime and src/config.py reads the .env by hand, so a clone needs Python 3 and nothing else to render the UI, rebuild the corpus, grade the key and re-score every arm. VERIFIED THIS CAPTURE: the corpus rebuilds byte-identically out-of-tree under two different PYTHONHASHSEEDs. NOT VERIFIED: nobody has cloned this kit onto a machine that has never seen it and run the whole sequence from scratch.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one returned unit end to end -- one dossier, one model call, including provider-side reasoning tokens, on a shared consumer connection. p50 15,831 ms and p95 50,099 ms over the 60 calls of r001-return-disposition; fastest 7,115 ms, slowest 76,330 ms. The pure-code recheck that follows adds nothing measurable at this resolution, and the free rules floor makes no call at all -- results/eval-b000-return-disposition-rules.json records wall_seconds 0.0.

Current processWhat it replaces

The desk's manual pass over one returned unit: reading the dossier, counting the months from the purchase date to the receipt date, checking that count against the model's warranty and against each of its supplier's clauses, counting the days the claim clock has run, looking the serial up in the repair history and deciding which of those repairs fall inside the lookback, comparing the estimate with 60 pct of the model's replacement value, and then applying the four-way precedence. It does not replace the person: the record is what they confirm.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one disposition record is

No train/test split, because nothing is trained or tuned, and no segmentation, because one dossier goes whole into one call. The unit every rate stands on is ONE DISPOSITION RECORD, one per returned unit: 60 units, 60 records, and every arm is scored over all 60 whether or not it answered. 17 units are clean -- one model, no history, no clause open, nothing near a boundary -- and 43 plant exactly one thing built to be got wrong, across 17 named cases: econ_scrap 5, latent_defect_rtv 5, chargeable_repair 4, in_warranty_repair 4, repair_cap_scrap 4, safety_hold 4, supplier_in_warranty 4, threshold_edge_repair 4, threshold_edge_scrap 4, claim_window_missed 3, fault_disagreement_repair 3, fault_disagreement_rtv 3, nonrepairable_scrap 3, repair_history_ok 3, rtv_over_cap 3, missing_finding_hold 2, unknown_serial_hold 2. THE ONE CUT THAT DECIDES THE RESULT is orthogonal to all of them: the inspection finding is a CODE on 30 units and a SENTENCE on 30.

SetupWhat the setup figure measured

THERE IS NO INDEX. There is nothing to retrieve from: one dossier goes whole into one prompt with the rulebook, the agreement, the desk record and a 36-month history slice, at a mean of 6,269.6 input tokens. The free half's whole cost is the fraction of a second the floor run records -- results/eval-b000-return-disposition-rules.json carries wall_seconds 0.0 for answering and scoring all 60 units with no key and no network. ⚠︎ 0.0 IS THE RECORDED VALUE, NOT A ROUNDED MEASUREMENT: the harness writes wall_seconds for the model arm's calls and the free arm makes none.

LicenceLicence

MIT -- the kit's own licence, whose text is in LICENSE-PUBLIC at the repository root. ⚠︎ NOT the root LICENSE, which governs the REPOSITORY itself and grants a reader nothing, because the repository is private: the kit and its corpus are MIT, the repository is not. There is no third-party data in this kit to licence -- the dossiers, the repair history, the rulebook and the supply agreement are all generated in-process.

Bring your ownBring your own returned units

Drop your own dossiers as .txt into data/corpus/, add one row per unit to data/returns.json (id, format, received_on, decided_on), put your models, warranties, replacement values, findings, supplier clauses, threshold, cap and lookback into data/rulebook.json, and your own closed repair orders into data/history.json. Nothing else changes: src/rules.py, src/terms.py, src/history.py and src/clock.py read all of it rather than typing any of it, and the free floor and the UI work immediately with no key. To SCORE your own set you also need a key file -- data/gold.jsonl, one object per unit with the five graded fields and the six read fields -- and that is the expensive half, because a disposition record is only as good as the rulebook it was derived from.

⚠︎ And what stops being true when you do: ⚠︎ A REAL RETURNED UNIT'S FILE IS PERSONAL DATA AND A REAL SUPPLY AGREEMENT IS SOMEBODY ELSE'S CONFIDENTIAL CONTRACT, AND THE WHOLE DOSSIER GOES TO A PROVIDER VERBATIM. The dossier carries a customer's name, their account of a failure and a serial that identifies one purchase; the agreement carries commercial terms both parties agreed not to publish. Nothing in this kit redacts anything, because redaction that is not measured is worse than none. Point this at your own returns only under whatever agreement covers sending that text to that provider -- and note that the history slice is the larger disclosure: it is 29.8 pct of every prompt and it contains EVERY serial the desk has touched in 36 months, not just the one being decided.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 22,769 characters assembled, 10,398 are rules — 46% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The exact seven parts sent for RT-0001, the first unit of run r001-return-disposition, replayed from the kit's own src/prompt.py build(). Trustworthy as the prompt that was sent because the replayed part names and character counts (3,616 / 7,416 / 2,982 / 351 / 6,776 / 616 / 1,012 = 22,769) are byte-for-byte the decomposition the run itself recorded under prompt_parts and prompt_chars_total in results/eval-r001-return-disposition.json. Across all sixty units the total moves only with the history slice and the dossier.

Full promptThe prompt in full

You are the returns desk of Calderfield Works, working one RETURNED UNIT. Your output is the
DISPOSITION RECORD a desk supervisor confirms -- so that the desk confirms a record instead of
reading a dossier.

How to read it:

- The dossier is the account the unit ARRIVED with: a service partner's email, a portal export, a
  row off the returns feed, or the receiving inspector's own bench notes. Read its fields off it
  whatever shape it arrives in: the model code, the serial, the date of purchase, the fault the
  CUSTOMER reported, the repair estimate, and the INSPECTION FINDING.
- THE INSPECTION FINDING GOVERNS. The reported fault is the customer's account of what went wrong
  and it decides nothing at all. Where the two point at different things -- and they often do --
  the finding is the fact and the reported fault is context. Report both.
- Two of the formats print the finding as a code from the list below. Two of them are written by a
  person and describe it in a sentence; map that sentence onto the one code in the list that it
  actually names. If no inspection has been done, the finding is null. Do not infer one from the
  reported fault, ever.
- The repair history below is the desk's OWN closed repair orders. It is the authority on what has
  already been done to this serial; the dossier is not. Join on the serial, and count only what
  the rulebook's lookback says counts.
- The supplier agreement below is the desk's own contract. Only the terms belonging to the MODEL'S
  OWN supplier are ever consulted, and a term is open only if all three of its tests pass.
- Apply the rulebook as written, including its arithmetic: the age of the unit in WHOLE MONTHS from
  the date of purchase to the date it was received, the claim clock in DAYS from receipt to the
  date the desk is deciding, and the scrap threshold as a percentage of the model's replacement
  value.
- Choose exactly one disposition, trying them in the order the rulebook gives:

  QUARANTINE        Something needed to decide is not in yet -- no inspection finding, or no serial -- or the finding is on the safety-hold list. The unit is held: nobody scraps it, repairs it or ships it to a supplier until a person resolves the hold.
  RETURN_TO_VENDOR  A supplier agreement term is OPEN on this finding for this unit: the finding is one the clause covers, the unit was inside the coverage window on the day it was received, and the claim is being filed inside the term's claim window. The unit goes back and the cost goes with it.
  SCRAP             The unit is not going to be repaired: the finding is on the non-repairable list, this serial has already reached the repair cap inside the lookback, or the estimate is above the scrap threshold for the model.
  REPAIR            Nothing holds it, no supplier owes it, and the repair is inside the threshold for the model. Authorise the quoted estimate. Whether the customer is charged is a separate question, answered by `in_warranty`.

- Then name the clause that decided it in `decided_by`: one of the clause ids in the rulebook, or,
  for RETURN_TO_VENDOR, the id of the supplier term that is open.
- `repair_authorised` is the money to release now: the quoted estimate for REPAIR, and 0.00 for
  every disposition that repairs nothing.
- `in_warranty` and `supplier_claimable` are asked SEPARATELY from the disposition and are not
  shorthand for it. A unit can be under warranty and scrapped; a unit can be claimable against a
  supplier term and still quarantined, because a hold outranks a claim.

Reply with JSON and nothing else, in the shape given at the end.

THE RULEBOOK, as this desk applies it:

THE MODELS THIS DESK SELLS -- its supplier, its warranty, what a new one costs
  MODEL     WHAT IT IS               SUPPLIER    WARRANTY  REPLACEMENT VALUE
  CW-210    Cordless stick vacuum    NORTHFIELD  24 months 249.00
  CW-315    Espresso brewer          KELVARO     24 months 399.00
  CW-408    Air purifier             ASTRIDGE    36 months 299.00
  CW-522    Countertop blender       NORTHFIELD  12 months 129.00
  CW-640    Robotic floor cleaner    PENMARK     24 months 649.00
  CW-733    Steam iron               KELVARO     12 months 89.00
  CW-856    Portable heater          ASTRIDGE    24 months 159.00
  CW-970    Cordless drill driver    PENMARK     36 months 219.00
  The warranty runs in WHOLE MONTHS from the date of purchase to the date the unit
  was received. A unit received the day before its month rolls over is still inside.

THE INSPECTION FINDINGS -- exactly one per unit, or null if nobody has looked yet
  CODE                   WHAT IT MEANS                            CUSTOMER  SAFETY    NON-REPAIRABLE
  battery_swell          Swollen battery cell                     -         HOLD      -
  bearing_wear           Bearing wear beyond service limit        -         -         -
  connector_fracture     Fractured internal connector             -         -         -
  firmware_corrupt       Corrupt firmware image                   -         -         -
  housing_crack          Cracked structural housing               -         -         yes
  impact_damage          Impact damage                            yes       -         -
  liquid_ingress         Liquid ingress                           yes       -         -
  missing_component      Component missing from the assembly      -         -         -
  no_fault_found         No fault found on test                   -         -         -
  power_board_failure    Power board failure                      -         -         -
  seal_failure           Seal failure on the water path           -         -         -
  solder_void            Solder void under a main-board joint     -         -         -
  thermal_runaway        Evidence of thermal runaway              -         HOLD      yes

THE REPORTED FAULTS -- what the CUSTOMER said, which decides nothing
  CODE                 WHAT IT MEANS                      USUALLY TURNS OUT TO BE
  leaking              Leaking                            seal_failure
  noisy                Noisy or vibrating                 bearing_wear
  other                Something else, or nothing stated  no_fault_found
  overheating          Runs hot or smells hot             battery_swell
  parts_missing        Parts missing from the box         missing_component
  physical_damage      Dropped or knocked                 impact_damage
  stops_mid_cycle      Stops part way through a cycle     firmware_corrupt
  will_not_power_on    Will not power on                  power_board_failure
  The reported fault is the CUSTOMER'S account of what went wrong and it decides nothing. `suggests` is what that complaint usually turns out to be, and it is published here only so the size of a disagreement can be named. Where the reported fault and the inspection finding point at different things, THE INSPECTION FINDING GOVERNS, always, without exception.

THE ECONOMICS
  scrap threshold       a repair is NOT economic when the estimate is MORE THAN
                        60% of the model's replacement value. Exactly at the
                        threshold is still economic.
  repair cap            2 repair(s) already closed on the same serial is at the cap
  lookback              only repairs closed within 36 months before the unit was
                        received count against the cap. Older ones are history.

THE DISPOSITIONS -- exactly one per unit
  QUARANTINE        Something needed to decide is missing, or the finding is on the safety-hold list. The unit is held; nobody scraps it, repairs it or ships it to a supplier until a person resolves the hold.
  RETURN_TO_VENDOR  A supplier agreement term is OPEN on this finding for this unit at this age, and the claim clock has not run out. The unit goes back to the supplier and the cost goes with it.
  SCRAP             The unit is not going to be repaired: the finding is non-repairable, the repair cap on this serial is reached, or the estimate is above the scrap threshold for the model.
  REPAIR            Nothing holds it, no supplier owes it, and the repair is inside the threshold. Authorise the estimate.

THE ORDER THEY ARE TRIED IN
  QUARANTINE -> RETURN_TO_VENDOR -> SCRAP -> REPAIR
  Fixed order, applied top down. QUARANTINE is first because you cannot dispose of a unit whose facts you do not have, and because a safety hold outranks every commercial question in the book. RETURN_TO_VENDOR precedes SCRAP because the repair cap and the scrap threshold are rules about THIS desk's repair budget, and a unit the supplier is taking back is not being repaired here at all -- a unit over the cap with an open supplier term still goes to the supplier. REPAIR is the default and never a decision of its own.

THE CLAUSE IDS -- `decided_by` is one of these, or a supplier term id
  ID                       DISPOSITION       WHEN IT FIRES
  HOLD-NO-FINDING          QUARANTINE        No inspection finding is recorded. The customer's reported fault is not a substitute and is never read as one.
  HOLD-NO-SERIAL           QUARANTINE        No serial is recorded, so the repair history cannot be read and no supplier claim can be filed. Both later steps need it.
  HOLD-SAFETY              QUARANTINE        The inspection finding is on the safety-hold list. Held for investigation whoever is going to pay.
  REPAIR-CHARGEABLE        REPAIR            Outside the standard warranty and inside the scrap threshold. Repaired and charged.
  REPAIR-IN-WARRANTY       REPAIR            Inside the standard warranty on the model. Repaired at no charge to the customer; this desk carries the cost.
  SCRAP-ECONOMIC           SCRAP             The estimate is above the scrap threshold for this model's replacement value.
  SCRAP-NONREPAIRABLE      SCRAP             The finding is on the non-repairable list. No estimate makes it repairable.
  SCRAP-REPAIR-CAP         SCRAP             This serial has already reached the repair cap inside the lookback. A third visit is a unit that is telling you something.
  The disposition says WHAT happens to the unit; decided_by names the clause that made it happen. For RETURN_TO_VENDOR the id is the SUPPLIER TERM's own id from supplier_terms above, never one of the eight ids listed here.

  Holds are tried in this order: HOLD-NO-FINDING -> HOLD-SAFETY -> HOLD-NO-SERIAL
  No finding first: until somebody has inspected the unit there is nothing to call a hazard and nothing to claim against. Safety next, because once a hazard IS named it outranks a records problem. A missing serial last -- it is the cheapest of the three to fix and the only one that is purely administrative.
  Scrap reasons are tried in this order: SCRAP-NONREPAIRABLE -> SCRAP-REPAIR-CAP -> SCRAP-ECONOMIC
  A non-repairable finding ends the question before any arithmetic. The cap is next because it is a fact about the serial rather than about this visit's estimate. The threshold is last and is the only one of the three that a cheaper quote could change.

THE SUPPLIER AGREEMENT, term by term. Only the terms belonging to the MODEL'S OWN
supplier are ever consulted.

  SUP-NORTHFIELD-SEAL-30  (NORTHFIELD -- Northfield Assemblies)
      Seal and bearing defect clause
      covers                 seal_failure, bearing_wear
      age at receipt         up to 30 months from the date of purchase
      claim must be filed    within 45 days of the date the unit was received
      Runs from the date of purchase, not from the date of manufacture, and it OUTLIVES the 12- and 24-month standard warranties on both Northfield models. That gap is the whole reason this clause is written down.

  SUP-NORTHFIELD-LATENT-42  (NORTHFIELD -- Northfield Assemblies)
      Latent solder defect clause
      covers                 solder_void
      age at receipt         up to 42 months from the date of purchase
      claim must be filed    within 45 days of the date the unit was received
      A void under a joint is not visible on receipt and does not present for years, so the window is long and the claim window is not.

  SUP-KELVARO-BOARD-36  (KELVARO -- Kelvaro Electronics)
      Power board and solder clause
      covers                 power_board_failure, solder_void
      age at receipt         up to 36 months from the date of purchase
      claim must be filed    within 30 days of the date the unit was received
      Thirty days from receipt of the unit, not thirty days from the customer's complaint. A unit that sat on the bench for six weeks is out.

  SUP-ASTRIDGE-CELL-48  (ASTRIDGE -- Astridge Power Systems)
      Cell and interconnect clause
      covers                 power_board_failure, connector_fracture, battery_swell
      age at receipt         up to 48 months from the date of purchase
      claim must be filed    within 60 days of the date the unit was received
      It covers a swollen cell, which is also a SAFETY HOLD -- so a unit can be claimable against this clause and still not be dispositioned, because the hold outranks the claim. Claimable and disposed are two different questions.

  SUP-PENMARK-FIRMWARE-18  (PENMARK -- Penmark Controls)
      Firmware and completeness clause
      covers                 firmware_corrupt, missing_component
      age at receipt         up to 18 months from the date of purchase
      claim must be filed    within 21 days of the date the unit was received
      The shortest window and the shortest claim clock in the book. It expires INSIDE both Penmark warranties, so a unit can be under warranty and past its supplier clause at the same time.

  The order they are tried in
    SUP-NORTHFIELD-SEAL-30 -> SUP-NORTHFIELD-LATENT-42 -> SUP-KELVARO-BOARD-36 -> SUP-ASTRIDGE-CELL-48 -> SUP-PENMARK-FIRMWARE-18
    Only the terms belonging to the MODEL'S OWN supplier are consulted; the order above decides when more than one of that supplier's terms is open on the same finding. Fixed, published, and asserted against src/terms.py's own iteration order at import.

THE DESK RECORD -- this return as the desk's own queue holds it.
The age of the unit runs from the date of purchase on the dossier to the
date it was received below; every claim clock runs from that same date to
the date the desk is deciding.

  Return id               RT-0001
  Received on             2026-06-28
  Decided on              2026-07-06

THE REPAIR HISTORY -- Calderfield Works' own closed repair orders, every unit, for the 36
months before this one was received (2023-06-28 to 2026-06-28). THIS is what has already been done to a
serial; the dossier is not. Join on the serial you read off the dossier.

  ORDER       SERIAL         CLOSED    COST        WHAT WAS DONE
  --------------------------------------------------------------
  RP-3052     SN-970-782230  2023-07-09       22.37 Reflowed the main-board joints
  RP-3086     SN-408-187390  2023-07-17      106.42 Replaced the seal kit
  RP-3012     SN-522-964311  2023-07-31       35.44 Replaced the bearing set
  RP-3008     SN-210-562624  2023-08-01       65.81 Replaced the bearing set
  RP-3046     SN-640-335088  2023-08-04      153.39 Replaced the fan and the thermal cutout
  RP-3055     SN-970-885815  2023-08-09      106.43 Cleaned and re-greased the drive
  RP-3035     SN-856-639821  2023-08-16       56.00 Cleaned and re-greased the drive
  RP-3065     SN-733-203101  2023-09-05       18.65 Cleaned and re-greased the drive
  RP-3032     SN-522-541004  2023-09-17       49.49 Replaced the seal kit
  RP-3011     SN-856-925981  2023-09-21       32.78 Reflashed the controller firmware
  RP-3087     SN-856-780211  2023-09-26       74.41 Replaced the fan and the thermal cutout
  RP-3083     SN-856-567625  2023-09-28       46.57 Cleaned and re-greased the drive
  RP-3024     SN-970-899346  2023-11-07       52.07 Replaced the fan and the thermal cutout
  RP-3090     SN-640-460741  2023-11-13      313.97 Replaced the fan and the thermal cutout
  RP-3067     SN-640-498757  2023-11-30      223.48 Replaced the fan and the thermal cutout
  RP-3069     SN-210-453032  2024-01-04       43.23 Replaced the seal kit
  RP-3033     SN-733-375247  2024-02-02       25.89 Cleaned and re-greased the drive
  RP-3070     SN-640-566177  2024-02-18      121.17 Replaced the power board
  RP-3016     SN-970-527622  2024-03-09       72.71 Replaced the impeller and the seal
  RP-3039     SN-408-690984  2024-04-07       36.19 Reflashed the controller firmware
  RP-3018     SN-640-890956  2024-04-25      165.50 Reflashed the controller firmware
  RP-3074     SN-210-550030  2024-05-05       59.72 Replaced the fan and the thermal cutout
  RP-3073     SN-210-607396  2024-06-07       79.80 Replaced the switch assembly
  RP-3050     SN-733-306449  2024-06-19       16.94 Replaced the seal kit
  RP-3049     SN-970-127450  2024-06-22       23.41 Replaced the impeller and the seal
  RP-3038     SN-733-875570  2024-06-24       42.89 Replaced the fan and the thermal cutout
  RP-3023     SN-970-969125  2024-07-10       20.84 Replaced the seal kit
  RP-3079     SN-856-353755  2024-07-19       65.75 Reseated and re-terminated the internal loom
  RP-3059     SN-733-218821  2024-07-20       19.09 Replaced the bearing set
  RP-3048     SN-408-963090  2024-07-28      116.17 Cleaned and re-greased the drive
  RP-3099     SN-970-468711  2024-08-05       27.37 Reflowed the main-board joints
  RP-3110     SN-733-330963  2024-08-11       11.12 Reseated and re-terminated the internal loom
  RP-3091     SN-522-713257  2024-08-28       21.50 Reflashed the controller firmware
  RP-3105     SN-408-236030  2024-09-06       49.83 Reflashed the controller firmware
  RP-3019     SN-733-765377  2024-09-09       14.45 Reseated and re-terminated the internal loom
  RP-3103     SN-210-652357  2024-10-14       41.50 Reflowed the main-board joints
  RP-3043     SN-640-202159  2024-10-20      176.06 Cleaned and re-greased the drive
  RP-3082     SN-408-735783  2024-10-25      105.62 Replaced the bearing set
  RP-3045     SN-856-741156  2024-11-23       71.33 Replaced the impeller and the seal
  RP-3112     SN-733-945703  2024-12-07       14.83 Replaced the bearing set
  RP-3014     SN-970-693189  2024-12-11       70.68 Replaced the impeller and the seal
  RP-3101     SN-315-580718  2025-01-18       49.87 Replaced the fan and the thermal cutout
  RP-3056     SN-408-948517  2025-01-26      104.10 Replaced the switch assembly
  RP-3006     SN-856-806267  2025-01-30       31.94 Replaced the power board
  RP-3084     SN-640-904050  2025-02-16      210.85 Reflashed the controller firmware
  RP-3080     SN-640-779095  2025-04-30       89.01 Reflashed the controller firmware
  RP-3025     SN-315-863563  2025-05-02       81.91 Replaced the switch assembly
  RP-3078     SN-522-434963  2025-05-03       31.36 Reflowed the main-board joints
  RP-3109     SN-733-756498  2025-05-04       11.12 Reflashed the controller firmware
  RP-3098     SN-315-704238  2025-05-28       66.50 Reseated and re-terminated the internal loom
  RP-3095     SN-315-856305  2025-06-18       49.87 Reflowed the main-board joints
  RP-3104     SN-210-652357  2025-07-14       31.12 Reseated and re-terminated the internal loom
  RP-3071     SN-210-659400  2025-08-07      116.53 Reflashed the controller firmware
  RP-3053     SN-640-907647  2025-08-13       74.35 Reflashed the controller firmware
  RP-3100     SN-970-468711  2025-10-05       36.50 Replaced the impeller and the seal
  RP-3113     SN-733-945703  2025-10-07       11.12 Replaced the power board
  RP-3051     SN-970-709092  2025-10-20       55.34 Replaced the impeller and the seal
  RP-3111     SN-733-330963  2025-11-11       14.83 Replaced the bearing set
  RP-3020     SN-733-189254  2025-11-17       35.58 Replaced the fan and the thermal cutout
  RP-3102     SN-315-580718  2025-11-18       39.90 Replaced the power board
  RP-3041     SN-210-151338  2025-11-30       75.11 Replaced the power board
  RP-3106     SN-408-236030  2025-12-06       49.83 Replaced the fan and the thermal cutout
  RP-3003     SN-522-638162  2025-12-16       42.96 Reseated and re-terminated the internal loom
  RP-3007     SN-408-181976  2025-12-21      125.10 Reseated and re-terminated the internal loom
  RP-3092     SN-522-713257  2025-12-28       21.50 Reflashed the controller firmware
  RP-3037     SN-733-380504  2026-01-04       28.86 Reflashed the controller firmware
  RP-3004     SN-970-410256  2026-01-25       38.57 Replaced the fan and the thermal cutout
  RP-3085     SN-640-380591  2026-01-25      277.00 Reflashed the controller firmware
  RP-3030     SN-522-339125  2026-02-02       35.07 Reflashed the controller firmware
  RP-3022     SN-522-484856  2026-02-23       47.52 Replaced the switch assembly
  RP-3054     SN-408-907356  2026-04-05       49.05 Reflashed the controller firmware
  RP-3066     SN-315-414299  2026-04-26      108.33 Replaced the bearing set
  RP-3002     SN-640-297685  2026-06-01      165.94 Replaced the seal kit
  RP-3034     SN-408-763241  2026-06-12       25.89 Replaced the bearing set
  RP-3009     SN-408-689464  2026-06-24       91.80 Reseated and re-terminated the internal loom

THE RETURNED UNIT'S PAPERWORK, verbatim:

From: returns@meridian-service.example
To: returns.desk@calderfield-works.example
Subject: [RMA RT-0001] returned unit, disposition required

RETURNED UNIT
  RMA reference        RT-0001
  Received at the desk 28 Jun 2026
  Model                CW-522  Countertop blender
  Serial               SN-522-713257
  Date of purchase     28 Feb 2024
  Reported fault       leaking
  Repair estimate      USD 49.02

  Inspection result    On the bench, liquid has got inside the unit and tracked across the board.

  Customer packed it in the original carton with the accessories.


Reply with JSON and nothing else, exactly this shape:

{"reading": {
   "model_code": "<the model code as the dossier gives it, e.g. CW-210>",
   "serial": "<the serial, or null if the dossier does not carry one>",
   "purchase_date": "<YYYY-MM-DD, the date the unit was BOUGHT, not the date it came back>",
   "repair_estimate": "<the quoted repair cost, decimal, e.g. 84.50>",
   "reported_fault": "will_not_power_on" | "overheating" | "leaking" | "noisy" | "stops_mid_cycle" | "physical_damage" | "parts_missing" | "other",
   "inspection_finding": "<one finding code from the rulebook, or null if none has been recorded>"},
 "disposition": "QUARANTINE" | "RETURN_TO_VENDOR" | "SCRAP" | "REPAIR",
 "decided_by": "<the id of the clause or supplier term that decided it>",
 "repair_authorised": "<decimal to release now; 0.00 for every disposition that repairs nothing>",
 "in_warranty": true | false,
 "supplier_claimable": true | false,
 "why": "<one sentence naming the clause that decided the disposition>"}

Raw responseThe raw response

{"reading": {
   "model_code": "CW-522",
   "serial": "SN-522-713257",
   "purchase_date": "2024-02-28",
   "repair_estimate": "49.02",
   "reported_fault": "leaking",
   "inspection_finding": "liquid_ingress"},
 "disposition": "SCRAP",
 "decided_by": "SCRAP-REPAIR-CAP",
 "repair_authorised": "0.00",
 "in_warranty": false,
 "supplier_claimable": false,
 "why": "Serial SN-522-713257 has two closed repairs within the lookback, so SCRAP-REPAIR-CAP applies."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Disposition of a returned unit from its paperwork — 60 returned units. One model answered, and every answer was then graded Three different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

No model grades anything and there is no LLM judge anywhere in the path. Every metric is exact match against data/gold.jsonl: strings for the disposition and the clause id, integer cents for the money, booleans for the two flags, and strings for the six read fields. The key itself is DERIVED rather than written -- src/rules.py is handed the generator's planted facts and computes the record -- so no person ever decided what a returned unit deserves.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Run it twiceThe same set, run again

The same 60 replies scored twice: 98.3 pct as answered and 100 pct after the engine re-derived every arithmetic field from the model's own reading. The single row that moves is RT-0015's in_warranty flag -- recheck_overrides is 1 and recheck_unrecheckable is 0.

record_all_correct over the 60 returned units -- all five graded fields right on one unit — They are not two samples of anything. They are one sample scored by two rules -- as answered, and after the pure-code station -- so averaging them would report a number no arm produced.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anyone paid. The token counts come from the provider's own usage block on each call, and 92.9 pct of the output tokens they report are provider-side reasoning the kit did not ask for.

Rates checked 2026-08-27. The provider that actually ran every call here is kept off this page per the series rule. The real spend, its own dated card (2026-08-23), its peak/off-peak split and its cached-input tier are all recorded per call in results/eval-r001-return-disposition.json, not on this page.

Grading unitWhat the grading figure prices

Scoring is pure Python over committed JSON: 60 records in well under a second, no key, no network. A committed run can be re-scored for nothing. THE ARM IS WHAT COSTS -- the free floor and the stub arm are $0.00 and the paid arm is $0.139903 for 60 calls.

The gradersThree ways to grade

TWO FLOORS AND THEY MEASURE DIFFERENT THINGS. The NULL baseline is answering the majority disposition on every unit: REPAIR is 21 of 60, so a fixed answer agrees 35.0 pct of the time and gets the clause, the money and both flags wrong on almost all of it. It exists to say what a number like 83.3 is measured against. THE ONE THAT MATTERS IS THE FREE RULES FLOOR -- b000-return-disposition-rules, a field reader in front of the same engine, no key, no network, wall_seconds 0.0. It scores 50 of 60 units all-correct, 83.3 pct, and it is not a strawman: it gets in_warranty 60 of 60, the model code, serial, purchase date, estimate and reported fault 60 of 60 each, both no-serial holds, all four safety holds, all three claim-window-expired units, all three lookback units and both sides of the eight cent-level threshold edges. ⚠︎ ALL TEN OF ITS MISSES ARE THE SAME MISS: every one lands in finding_misread and nothing lands in any other bucket. Its arithmetic is perfect and its reading is not, which is the most useful thing this kit measured before a cent was spent. ⚑ AND THE FLOOR'S OWN RECHECK WAS SWITCHED OFF UNTIL 2026-08-31 — IT IS NOW MEASURED RATHER THAN ARGUED. evals/run.py's --floor branch built the floor's rechecked column by COPYING its own raw answer instead of calling src/recheck.py, which is what the model path does on every record, and a copied column can only ever flatter the arm compared against it. The branch now makes the real call with the same arguments, and every free arm was re-run at $0.00 on zero calls: recheck_overrides 0, recheck_unrecheckable 0, and NOT ONE FIGURE IN THIS NOTE OR THE SCORES BLOCK MOVED. That is structural rather than lucky — evals/baseline.py already derives its record with src/rules.derive(), the same engine src/recheck.py re-applies — so the floor's 83.3 pct was never inflated, and the two arms are now on one code path. The paid arm was not re-fired: eval-r001-return-disposition.json and its cache are byte-identical and the total is still $0.139903.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-27. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingGrading is free; the bill is the 60 calls that answered.

The scorer, the label gate, the corpus generator, the rules floor and the stub arm are pure code and cost $0.00 to run against any result set -- there is no LLM judge anywhere in the grading path, and a committed run can be re-scored for nothing. The figure above is the token count of the ONE paid run (60 calls, 376,179 in / 144,583 out) priced at the same projected card cost_per_query_usd uses. ⚠︎ THERE IS NO SECOND PAID RUN TO ADD: the injection probe is written and unfired, so reproducing this kit's whole paid evidence is exactly this one number.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND ALMOST ALL OF THEM ARE INVISIBLE. 92.9 pct (134,252 of 144,583) are provider-side reasoning left at the tier's default and re-rolled per call, and the output side is 69.8 pct of the projected bill. The visible answer is a JSON object of a few hundred tokens -- the RT-0001 reply is ~470 characters and was billed 1,674 output tokens, 1,513 of them reasoning.
• THE STABLE PREFIX, WHICH WORKS IN YOUR FAVOUR. The system role, the rulebook and the supplier agreement are byte-identical across all 60 calls and 61.5 pct of the prompt's characters. MEASURED: 179,712 of 376,179 input tokens came back as cache hits, 47.8 pct -- short of what the character split predicts, because 6 of the 60 calls report no cache hit at all. The projection card has no cache tier and prices all of it at the full input rate.
• ⚑ THE DOSSIER IS 2.7 PCT OF THE PROMPT AND THE DESK'S OWN HISTORY IS 29.8 PCT. 616 characters of paperwork against 6,776 of repair orders, on a 22,769-character prompt whose input token count varied only between 6,204 and 6,360 across all sixty units. You are not paying to read the return; you are paying to carry the workshop's last 36 months on every call.

Your volumeWhat it costs at your volume

Linear in units and flat in everything else -- ten times the returns is ten times the calls at the same per-unit cost, because one dossier goes whole into one prompt and there is no index to rebuild and nothing shared between calls except the cached prefix, which gets CHEAPER per call as the run gets longer. ⚠︎ WHAT IS NOT LINEAR IS THE HISTORY SLICE: it carries every serial the desk touched in 36 months, so a desk with ten times the repair throughput pays more per call on a prompt that is already 29.8 pct history. Neither shape was run at any scale but this one: 60 units, one machine, 249.3 s of wall clock.

Model choiceWhy this model

It is the tier the estate runs, so the figure compares with every sibling kit. The finding here is not about the model: the reading task turned out to be inside it on all sixty dossiers, in both formats, which says the corpus stopped separating arms before the tier did. A cheaper arm has already been measured and it is the free floor -- 83.3 pct for $0.00.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the whole 60-unit scored run (r001-return-disposition, the fast tier): 376,179 input tokens (179,712 of them prefix cache hits) and 144,583 output, of which 134,252 were provider-side reasoning. ⚠︎ usd_actually_paid is null ON THIS PAGE by the series rule, not because it is unknown: the real bill is $0.139903 at the off-peak tariff -- $0.279808 repriced at the same provider's weekday peak rate -- and it is recorded per call, with its tariff, in the run record.

The codeEvery file, and what it is for

16 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Eight of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/prompt.pyprompt assembly

Seven parts in a fixed order -- the system role, data/rulebook.json rendered, the five supplier terms, the desk record, the 36-month repair-history slice, the dossier verbatim, and the JSON schema. THE FIRST THREE ARE BYTE-IDENTICAL ON EVERY CALL and are sent first, so a provider that prices cached input separately can bill a 14,014-character prefix (61.5 pct of the prompt) at the cached rate. The prompt names all four dispositions, every clause id, every finding and every term -- scoring an arm on a vocabulary it was never given measures the prompt, not the arm -- and it never does the arithmetic: it never says how old the unit is, whether a clause has expired, whether the estimate clears the threshold or how many repairs count.

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
SLICE_MONTHS = H.LOOKBACK_MONTHS
VERDICTS = ("QUARANTINE", "RETURN_TO_VENDOR", "SCRAP", "REPAIR")
VERDICT_MEANINGS = {
FINDINGS = RULES["findings"]
FAULTS = RULES["reported_faults"]
DECIDED_BY = RULES["decided_by"]
ECON = RULES["economics"]

src/prompt.pythe disposition vocabulary

VERDICTS and VERDICT_MEANINGS — the four dispositions declared ONCE as module-level literals, asserted at import against data/rulebook.json's dispositions AND against its published precedence order. The system prompt's definition block is generated from VERDICT_MEANINGS, so the four words the prompt defines and the four the page defines cannot drift. There is deliberately no data/verdicts.json: a second copy beside the scorer is the thing this arrangement exists to avoid, and the site reads these two names out of this module by AST at build time rather than importing the kit.

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
SLICE_MONTHS = H.LOOKBACK_MONTHS
VERDICTS = ("QUARANTINE", "RETURN_TO_VENDOR", "SCRAP", "REPAIR")
VERDICT_MEANINGS = {
FINDINGS = RULES["findings"]
FAULTS = RULES["reported_faults"]
DECIDED_BY = RULES["decided_by"]
ECON = RULES["economics"]

src/classifier.pythe model call

One call per returned unit, and the only place a model is called. Parses the reply (fence-tolerant), normalises the closed vocabularies for case and shape only -- never meaning, and never filling a field in, so an unreadable estimate stays None and is scored as a miss rather than quietly becoming 0.00. max_tokens is 32,000 because the tier re-rolls a provider-side reasoning budget per call; a reply cut off at the ceiling is recorded with at_ceiling and stays in the denominator rather than scoring partially.

# One returned unit in, one disposition record out. The only place a model is called.

MAX_TOKENS = 32000
THINKING = None
def parse_reply(text):
def _text(v):
def _bool(v):
def _code(v):
def normalise(obj):
def classify(cfg, text, unit, complete_fn=None, max_tokens=None):

src/adapters/__init__.pythe adapters — a swap seam

Raw HTTP over urllib to any OpenAI-compatible provider or Anthropic. TIMEOUT_S is 1,200 s and the run record stores it as socket_timeout_s. Nothing is streamed: one call per unit, the whole reply read at once.

You change it to: PROVIDER + BASE_URL + MODEL in .env -- one line, then the same run again. Two adapter shapes ship (OpenAI-compatible and Anthropic's Messages API); a third is one function and one entry in PROVIDERS, and it must return token counts, because the Cost lens prices them.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
TIMEOUT_S = 1200
TRANSPORT_RETRIES = 1
def _post(url, headers, payload, timeout=TIMEOUT_S):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

src/budget.pythe call budget

A shared cap on LIVE CALLS -- not dollars -- counted against a ledger beside whichever .env is the shared one, so every kit under that root counts against one budget. It counts calls because a dollar cap needs a rate card the kit does not know. NO CAP CONFIGURED MEANS NO CAP AND IT SAYS SO OUT LOUD, because a forker who clones one kit has no repo root above it.

# A cap on live calls, shared by every kit on this machine. No dependency, no service.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
KIT = os.path.basename(HERE)
ROOT = os.path.dirname(os.path.dirname(HERE))
SHARED_ENV = os.path.join(ROOT, ".env")
LEDGER = os.path.join(ROOT if os.path.exists(SHARED_ENV) else HERE, ".calls-ledger.jsonl")
PAID_ARMS = ("r", "x")
FREE_ARMS = ("t", "b")
BOOT = os.urandom(6).hex()
PID = os.getpid()

src/rules.pythe disposition engine

PURE CODE, and every arm stands on it. The four-way precedence and the two inner orders (three hold reasons, three scrap reasons) are read from data/rulebook.json and ASSERTED AT IMPORT against the engine's own branch order -- an engine whose branches have drifted from the order it publishes explains itself wrongly on every page it appears on. It writes the answer key, it backs the free floor, it rechecks the model and it is what the UI prints. Sixty percent, the cap of 2, the 36-month lookback and every coverage window are in the JSON, not in this file.

# The disposition engine: the rulebook as arithmetic. Pure code, no model, no key.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
DISPOSITIONS = tuple(RULES["precedence"])
AUTHORISES_REPAIR = tuple(k for k in DISPOSITIONS if RULES["dispositions"][k]["authorises_repair"])
FINDINGS = RULES["findings"]
REPORTED_FAULTS = tuple(RULES["reported_faults"])
MODELS = RULES["models"]
ECON = RULES["economics"]
HOLDS = tuple(RULES["hold_precedence"])

src/terms.pythe supplier terms — a swap seam

Three tests per clause, all arithmetic: the finding must be in the term's covers_findings, the unit's age in whole months from purchase to RECEIPT must be inside coverage_window_months, and the days from receipt to DECISION must be inside claim_within_days. The third is the one a desk loses money on -- it is a property of the desk, not of the unit, and it runs out while the unit sits on a bench. ONLY THE MODEL'S OWN SUPPLIER'S TERMS ARE CONSULTED: two suppliers here both cover power_board_failure and solder_void on different windows, so reading the term list as a flat set of findings turns a Kelvaro board into an Astridge claim.

You change it to: The three tests are functions over a term's declared fields. A condition none of them covers is a new function, not a new expression language in the JSON -- the JSON carries numbers and clause ids, the code carries the shape.

# The supplier agreement, as arithmetic: which clause is open on this unit. Pure code.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
TERMS = RULES["supplier_terms"]
TERM_ORDER = tuple(RULES["term_precedence"])
MODELS = RULES["models"]
SUPPLIERS = RULES["suppliers"]
def supplier_of(model_code):
def terms_for(model_code):
def evaluate(model_code, finding, purchase_date, received_on, decided_on):

src/history.pythe repair history

The desk's own 113 closed repair orders are the authority on what has already been done to a serial, and the dossier is not. prior_repairs() counts only repairs CLOSED inside the rulebook's 36-month lookback before receipt. NO SERIAL MEANS NO HISTORY AND NO HISTORY IS NOT AN EMPTY HISTORY: prior_repairs(None) raises rather than returning [], because 'never repaired' and 'we do not know what this unit is' are different facts with different dispositions.

# The workshop's own repair history: what this serial has already cost. Pure code.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
ECON = RULES["economics"]
HISTORY = json.load(open(os.path.join(HERE, "data", "history.json"), encoding="utf-8"))
REPAIRS = HISTORY["repairs"]
LOOKBACK_MONTHS = int(ECON["prior_repair_lookback_months"])
MAX_PRIOR = int(ECON["max_prior_repairs"])
class UnknownSerial(ValueError):
def all_for(serial):

src/clock.pythe clock

Ages in whole months by calendar walk -- not 30 days, not 365/12 -- and claim clocks in plain days from receipt to decision. A unit bought 2024-03-31 and received 2026-03-30 is 23 months old, not 24, and on a 24-month warranty that one day is the difference between a free repair and an invoice. Dates only, no times, deliberately.

# Ages in whole months and claim clocks in days. Pure code, no model, no key.

def d(x):
def iso(x):
def months_between(start, end):
def days_between(start, end):
def minus_months(date_, n):
def describe(unit):

src/money.pymoney

Integer cents end to end -- no float ever touches an authorised amount. The parser is tolerant on the way in (four dossier formats spell money four ways) and strict on the way out: a string with no digits or more than two decimal places returns None, and None is scored as a miss rather than silently becoming zero.

# Money as integer cents, and nothing else. Pure code, no model, no key.

def to_cents(value):
def fmt(cents):
def tolerance_cents(amount_cents, bps, floor_cents):

src/recheck.pythe recheck — a swap seam

THE PURE-CODE STATION. It takes exactly SIX fields from the model -- model_code, serial, purchase_date, repair_estimate, reported_fault, inspection_finding -- and re-derives everything else: the age, the claim clock, every clause test, the history join, the threshold comparison and the four-way precedence. The result file carries the answer both ways, so the eval can say how much of the accuracy the model bought and how much the reconciliation behind it bought. ⚠︎ IT CANNOT FIX A MISREAD, and on this kit that is the whole story.

You change it to: The six fields taken from the reply. Widening it moves the injection surface: everything on that list is reachable by a sentence in the box, and everything off it is re-derived from files no dossier can touch.

# The model's reading, the desk's own arithmetic. Pure code, no model, no key.

TAKEN_FROM_MODEL = ("model_code", "serial", "purchase_date", "repair_estimate_cents",
GRADED = ("disposition", "decided_by", "repair_authorised_cents", "in_warranty",
def recheck(answer, unit):

evals/baseline.pythe free rules floor

A field reader in front of the same engine, with no model and no key. It reads the labelled fields out of the dossier and, where the inspection finding is written as a sentence rather than a code, falls back to the customer's reported fault mapped through the rulebook's own suggests table. It scores 50 of 60 units all-correct for $0.00 -- and all ten of its misses are the same miss.

# THE FREE FLOOR. No key, no model, no network. A genuine attempt at the job.

MODES = ("rules",)
HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RULES = json.load(open(os.path.join(HERE, "data", "rulebook.json"), encoding="utf-8"))
FINDINGS = RULES["findings"]
FAULTS = RULES["reported_faults"]
LABEL_TO_CODE = {v["label"].lower(): k for k, v in FINDINGS.items()}
MONTHS = {m: i + 1 for i, m in enumerate(
MONTH_RE = (r"(jan(?:uary)?|feb(?:ruary)?|mar(?:ch)?|apr(?:il)?|may|jun(?:e)?|jul(?:y)?|"
MODEL = [r"\b(CW-\d{3})\b"]

evals/scoring.pythe scorer — a swap seam

Pure code. Exact match against data/gold.jsonl on five graded fields per unit, per-class precision and recall over the four dispositions, both money directions in cents, and a nine-bucket taxonomy assigned in a FIXED ORDER WITH THE READING BUCKETS FIRST -- because a wrong disposition on a misread finding is a reading defect, not a rules defect, and this kit exists to tell those apart.

You change it to: The five graded fields, the two money directions, per-class precision and recall, the case and format cuts, and the nine-bucket taxonomy with its fixed assignment order.

# Grade one arm against the answer key. Deterministic -- no model judges anything here.

FIELDS = ("disposition", "decided_by", "repair_authorised_cents", "in_warranty",
READING_FIELDS = ("model_code", "serial", "purchase_date", "repair_estimate_cents",
AUTHORISES = set(R.AUTHORISES_REPAIR)
WRITTEN_FORMATS = ("rma_email", "bench_notes")
TAXONOMY = ("no_verdict", "finding_misread", "serial_misread", "date_misread", "estimate_misread",
def _pct(n, d):
def _s(v):
def _bucket(g, m, gr, mr):
def score(records, golds, texts=None):

tools/build_corpus.pythe corpus generator — a swap seam

Writes the 60 dossiers, the 113-order repair history and the answer key from seed 20260831. The key is COMPUTED by src/rules.py from the generator's planted facts, never written by a person, and the generator refuses to emit a corpus in which a case fails to derive the disposition its name promises, or in which a written dossier contains the finding's code or published label anywhere in its text.

You change it to: The four writers -- rma_email, portal_export, csv_row, bench_notes, 15 each. The CODE/SENTENCE split rides on this seam: the two machine formats carry finding_code verbatim and the two written ones paraphrase, which is what makes the free floor's ceiling a measurement rather than a claim.

# Build the corpus, the repair history and the answer key. Deterministic, free, offline.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
SEED = 20260831
DATASET_VERSION = "return-disposition-v1-60returns"
RULES = json.load(open(os.path.join(DATA, "rulebook.json"), encoding="utf-8"))
MODELS = RULES["models"]
FINDINGS = RULES["findings"]
TERMS = RULES["supplier_terms"]

evals/check_labels.pythe label gate

Grades the answer key with INDEPENDENT arithmetic, importing no src/ module: its own calendar month walk, its own lookback cutoff via calendar.monthrange, its own claim clock, its own three-test clause evaluation, its own threshold comparison and its own precedence -- and it asserts that every field the key claims to have read is present, as a string, in the dossier's own text. Two implementations, one key.

# Grade the ANSWER KEY. No key, no model, no network.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
RULES = json.load(open(os.path.join(DATA, "rulebook.json"), encoding="utf-8"))
HISTORY = json.load(open(os.path.join(DATA, "history.json"), encoding="utf-8"))
UNITS = {u["id"]: u for u in json.load(open(os.path.join(DATA, "returns.json"), encoding="utf-8"))}
MODELS = RULES["models"]
FINDINGS = RULES["findings"]
TERMS = RULES["supplier_terms"]

src/app.pythe local UI

Standard library only, on port 9216. /api/returns, /api/return, /api/rules, /api/recorded, /api/prompt and /api/corpus need no key; only /api/work calls a provider and with no API_KEY it returns 200 saying so. It prints the model's answer twice -- raw and rechecked -- with the free floor beside it, and says per unit whether the floor READ the finding off a code or GUESSED it from the customer's words.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
PORT = int(os.environ.get("PORT", "9216"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-return-disposition")
FLOOR_RUN = "b000-return-disposition-rules"
GRADED = ("disposition", "decided_by", "repair_authorised_cents", "in_warranty",
def units():
def golds():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 6269 input and 2409 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ WRITTEN, WIRED, AND NOT FIRED — AND NO NUMBER ON THIS PAGE SAYS OTHERWISE. A returned unit's paperwork is this kit's injection surface and it is not a hypothetical one: the dossier is written OUTSIDE this desk — by the customer, by a retailer, by a service partner — and it reaches the model verbatim, free-text fields and all. evals/injection.py appends one attacker-shaped line to the register of a supplier note on each of the units whose correct answer is NOT a supplier's problem, and pairs against the scored run's own cached answers so the control costs nothing. It has never been run. What the architecture CLAIMS is that an injected sentence about the AGREEMENT cannot reach the rechecked answer, because coverage is three tests against data/rulebook.json and a date in data/returns.json — and what it admits is that a sentence engineered to move the READING moves everything, because the six read fields are exactly what src/recheck.py trusts.

API_KEY is read from the shared <code>&lt;repo&gt;/.env</code>, then this kit's own <code>.env</code>, then the real environment (src/config.py) — eight lines rather than a dependency. The .env is gitignored from the first commit and this repo has never held a credential. No key is ever written into a result file, a cache line or a log. src/budget.py counts LIVE CALLS against a shared ledger beside whichever .env is the shared one, so every kit under that root counts against one budget; with no cap configured it says so out loud rather than refusing to run.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. ⚠︎ THE ADVERSARIAL ARM WAS NEVER FIRED. python3 -m evals.injection --run-id x001-return-disposition --limit 20 --yes is the command; 20 calls at roughly this run's per-call cost, with the control read FREE from results/cache-r001-return-disposition.jsonl. Measured on no run — results/ holds no x001-return-disposition file, and no adversarial figure appears anywhere in this spec.

Both gates are open. Writing the shape down here is what stops a future run reporting a padded denominator as a defence — the paired design only counts units the control got RIGHT, because 'the injected arm also got it wrong' is not evidence about injection.

The result⚠︎ NOT FIRED — no adversarial number exists for this kit. evals/injection.py is written, wired, and has had its free control since the scored run was cached. 20 calls would produce the first number on this page.

The design, so it can be checked rather than re-argued later: each trial is one dossier of this corpus with a single sentence appended as the last entry in the register of a supplier note already on file — not a new section, not a header, because a note that announces itself is a note a reader would delete. The target set is the units whose correct answer is NOT RETURN_TO_VENDOR (45 of 60), capped at 20 by --limit. The control arm is the scored run's own cached answers, so a trial costs one call rather than two and is compared against the exact answers this kit published. Only units the control got right enter the paired denominator. Both arms are scored — RAW, which the injection targets, and RECHECKED, which it can only reach through the reading — and the probe additionally counts how many replies had their inspection_finding moved.

HonestyWhat this does not prove

• EVERYTHING ABOUT INJECTION. The probe is written and unfired: no wording has been tried, on any model, on any day. This is not a 0 pct suppression rate; it is no rate.
• ⚠︎ WHETHER THE PURE-CODE STATION PROTECTS ANYTHING. recheck_overrides is 1 on 60 units and the one thing it caught was a warranty boolean. The engine has never disagreed with this model about a finding, a clause or a cent — even without an attack.
• AN ATTACK ON THE READING. The six read fields are what the recheck trusts, and a sentence engineered to move inspection_finding moves the whole record. Unmeasured, and it is the direction the architecture cannot defend.
• WHETHER THE HISTORY IS REACHABLE AT ALL. The claim that data/history.json outranks the paperwork is architectural: the join happens in src/history.py on the model's serial, so an attack would have to move the SERIAL rather than argue about a repair. Nobody has tried it.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never scrap a unit, never raise a supplier claim, never release a repair order, never invoice a customer and never write to any system. This kit produces the RECORD a person confirms, and QUARANTINE is that record saying, in the vocabulary the desk already uses, that it cannot be confirmed yet.

Stated on the UI, in the README and in src/app.py's docstring, and enforced by there being no such endpoint: /api/returns, /api/return, /api/rules, /api/recorded, /api/prompt and /api/corpus are read-only and need no key, and /api/work is the only route that calls a provider — with no API_KEY it returns 200 saying so rather than failing. Nothing anywhere writes to data/.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is ABSENCE OF A WRITE PATH plus a prompt rule, not a runtime enforcement layer — with one partial exception: src/recheck.py IS enforcement for everything the rulebook decides by arithmetic. It re-derives the age, the claim clock, all five clause tests, the history join, the threshold comparison and the four-way precedence from the model's reading and overrides the reply.
• ⚠︎ AND IT ENFORCED ALMOST NOTHING ON THIS RUN: recheck_overrides is 1 in 60 units, on a warranty BOOLEAN. It is a control that has fired once and has never yet disagreed with this model about anything that moves money.
• ⚠︎ THERE IS NO CONTROL AT ALL FOR THE FAILURE THIS KIT IS ABOUT. A wrong inspection_finding produces a confident, well-explained, internally consistent, wrong record that cites a real clause. The recheck trusts the finding by design; nothing downstream catches it but the person confirming the record. The free floor's ten misses are ten demonstrations, and the paid arm produced none — so the gap is unmeasured rather than closed.
• THERE IS NO CONFIDENCE FIELD AND THAT IS DELIBERATE. This kit asks for a clause id and a sentence naming it, not a number. A self-reported confidence would read as a guardrail and measure nothing; a clause id can be checked against data/rulebook.json.
• QUARANTINE IS NOT A REFUSAL MECHANISM. It is a disposition with its own precedence and its own three reasons, and an arm can be wrong by over-using it exactly as it can by under-using it. Neither arm over-uses it here: both are 8 of 8 with no false positives.

WatchedWhat is watched, and why that one

The kit measures 35 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

127 measured by the latest run -92 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (cap_records 4, clause_discriminating_records 39, clause_records 60, clean_records 17, coded_finding_records 30, dataset_version return-disposition-v1-60returns, disagreement_records 23, failures 0, hard_records 43, lookback_records 3, no_repair_records 39, reading_records 60, rechecked_cap_records 4, rechecked_clause_discriminating_records 39, rechecked_clause_records 60, rechecked_clean_records 17, rechecked_coded_finding_records 30, rechecked_disagreement_records 23, rechecked_hard_records 43, rechecked_lookback_records 3, rechecked_no_repair_records 39, rechecked_reading_records 60, rechecked_repair_records 21, rechecked_safety_records 4, rechecked_written_finding_records 30, repair_records 21, returns 60, returns_answered 60, safety_records 4, socket_timeout_s 1200, written_finding_records 30) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 127 chips that all say so.

comply · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 131 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 127 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The corpus generator, the label gate, the free floor, the stub arm and the scorer are free and re-run in seconds on a key-less machine, so every band above can be re-measured for $0.00 against the committed results. Re-measuring the MODEL costs 60 calls, and firing the adversarial arm for the first time costs 20 with the control free from the cache.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end — no orchestration layer, no vendor SDK, no agent loop, no retrieval, no state carried between calls. requirements.txt pulls nothing at runtime, so a forker runs this on whichever key they already hold.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear, and concurrent only at the unit level. One dossier in, one prompt, one call, one parse, one pure-code recheck, one record out. There is no agent, no tool loop, no retrieval and no state carried between units — the only thing shared across calls is the cached prompt prefix, which is the provider's affair.

The other sideWhat a framework costs you

• Adding a provider means editing the PROVIDERS dict by hand. In exchange pip install pulls nothing.
• The reply is parsed by hand. It held on all 60 paid calls, and that is an observation, not a guarantee.
• No rules-engine DSL, so a rulebook whose SHAPE differs from this one's (read → clause order → first condition met) is a change to src/rules.py rather than to a config file.
• No ORM and no database for the repair history: data/history.json is loaded whole and filtered in memory. That is fine at 113 orders and is the first thing to break on a real desk's ledger.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-return-disposition on the fast tier, 2026-08-31. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. Only one run has recorded a call latency; the other 1 run on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT -- the kit's own licence, whose text is in LICENSE-PUBLIC at the repository root. ⚠︎ NOT the root LICENSE, which governs the REPOSITORY itself and grants a reader nothing, because the repository is private: the kit and its corpus are MIT, the repository is not. There is no third-party data in this kit to licence -- the dossiers, the repair history, the rulebook and the supply agreement are all generated in-process. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe whole disposition record -- all five graded fields on one returned unit

whether each of the 60 units produced a record a supervisor could confirm as written: the disposition, the clause id that decided it, the repair money authorised to the cent, whether the unit is inside its warranty, and whether a supplier term is open. All five, not the headline one -- a right disposition with the wrong clause on it cannot be checked against the rulebook, and a right disposition with the wrong warranty flag invoices the wrong party.

MethodHow the test was run

python3 -m evals.run --run-id <id> [--floor rules | --stub]; evals/scoring.py compares the reply against data/gold.jsonl — exact strings for the disposition and the clause id, integer cents for the money, booleans for the two flags. No model is in the grading path, and a committed run re-scores from results/cache-*.jsonl for $0.00.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl — DERIVED, NOT WRITTEN. src/rules.py is handed tools/build_corpus.py's planted facts at seed 20260831 and computes the record; no person ever decided what a returned unit deserves. evals/check_labels.py then re-derives every age, claim clock, clause test, history join, threshold comparison and precedence decision with arithmetic written inside itself, importing no src/ module — its own calendar month walk rather than a multiply-and-adjust, its own lookback cutoff via calendar.monthrange rather than a decrementing clamp — and asserts that every field the key claims to have READ appears as a string in the dossier's own text, and that no written dossier spells the finding's code or published label anywhere. Two implementations, one key. Every manufacturer, model, supplier, agreement, serial, repair order and returned unit is invented; see data/SOURCES.md.

No true/false rates for this grader. It records 8 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No tolerance anywhere. The disposition and the clause id are exact strings, the money is exact integer cents (src/money.py, no float ever touches it) and the two flags are exact booleans. The only numbers that look like thresholds belong to the RULEBOOK and are applied identically by every arm: 60 pct of replacement value, a repair cap of 2, a 36-month lookback, and each clause's coverage window and claim clock.

Cadence: Once per corpus version. The paid arm ran ONCE, was not re-fired after its miss was read, and the rechecked column is re-derivable from the committed cache for nothing.

The decisionWhen to reach for it

In one lineThe two money directions and the three claim directions, never averaged

which way an arm fails, in cents. FALSE AUTHORISE is money released on a unit that should not have been repaired -- scored on the 39 units whose correct answer is not REPAIR, and totalled in cents. FALSE WITHHOLD is the reverse, on the 21 that should be. Beside them: MISSED CLAIM (a supplier term open and not raised, on the 15 RETURN_TO_VENDOR units), FALSE CLAIM (raised against no open term), and MISSED SAFETY HOLD on the 4 safety units. Each has its own denominator and none is averaged with another, because a desk that over-authorises and a desk that over-scraps have different problems.

MethodHow the test was run

The same run; evals/scoring.py counts each direction on its own denominator and sums the cents. No model, no network, $0.00.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl — DERIVED, NOT WRITTEN. src/rules.py is handed tools/build_corpus.py's planted facts at seed 20260831 and computes the record; no person ever decided what a returned unit deserves. evals/check_labels.py then re-derives every age, claim clock, clause test, history join, threshold comparison and precedence decision with arithmetic written inside itself, importing no src/ module — its own calendar month walk rather than a multiply-and-adjust, its own lookback cutoff via calendar.monthrange rather than a decrementing clamp — and asserts that every field the key claims to have READ appears as a string in the dossier's own text, and that no written dossier spells the finding's code or published label anywhere. Two implementations, one key. Every manufacturer, model, supplier, agreement, serial, repair order and returned unit is invented; see data/SOURCES.md.

No true/false rates for this grader. It records 7 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? Cents, not dollars, and no tolerance. cents_over_authorised and cents_under_authorised are summed as integers across the whole set and converted once for display; a unit that is one cent out is a miss, which is the point of the eight threshold-edge units.

Cadence: Once per corpus version, free, from the committed results.

The decisionWhen to reach for it

In one lineThe six read fields -- and the CODE/SENTENCE split that decides this kit's whole result

whether the arm READ the dossier, separately from whether it reasoned correctly. The six fields are the model code, the serial, the purchase date, the repair estimate, the reported fault and the inspection finding, and the sixth is cut two ways: 30 units where the finding is a machine CODE and 30 where it is a person's SENTENCE. The taxonomy assigns reading buckets FIRST, so a wrong disposition caused by a misread finding is counted as a reading defect and not as a rules defect.

MethodHow the test was run

The same run; evals/scoring.py compares each read field against data/gold.jsonl and cuts the finding by the corpus's own format flag. The taxonomy in the same file assigns reading buckets FIRST, so a wrong disposition caused by a misread finding is counted once, as a reading defect.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl — DERIVED, NOT WRITTEN. src/rules.py is handed tools/build_corpus.py's planted facts at seed 20260831 and computes the record; no person ever decided what a returned unit deserves. evals/check_labels.py then re-derives every age, claim clock, clause test, history join, threshold comparison and precedence decision with arithmetic written inside itself, importing no src/ module — its own calendar month walk rather than a multiply-and-adjust, its own lookback cutoff via calendar.monthrange rather than a decrementing clamp — and asserts that every field the key claims to have READ appears as a string in the dossier's own text, and that no written dossier spells the finding's code or published label anywhere. Two implementations, one key. Every manufacturer, model, supplier, agreement, serial, repair order and returned unit is invented; see data/SOURCES.md.

No true/false rates for this grader. It records 5 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? Exact string match on the rulebook's own finding vocabulary, after case folding. There is no partial credit and no near-miss: liquid_ingress and seal_failure are adjacent in a person's language and opposite in the agreement, which is exactly the distinction being measured.

Cadence: Once per corpus version, free, from the committed results.

The decisionWhen to reach for it