The business caseThe problem this solves

A meter is not read every cycle. When it is not, the utility bills an ESTIMATE, and when an actual read finally arrives the two have to be reconciled: what did the meter really record over the gap, how does that differ from what was billed, in kWh and in money under a tiered seasonal tariff, and is the difference big enough to rebill. That is one question. The second is harder and nobody does it: across this meter's recent history, is the ESTIMATION RULE ITSELF wrong -- persistently high, persistently low, merely seasonal, or explained by something that actually changed at the premise. A single unlucky estimate is noise. A run of them in one direction is a defective rule, and it keeps producing rebills until somebody notices. The line-by-line true-up of an estimated read against the tariff -- registers, rollover, day-weighted allocation, tier ladder, season, threshold -- that a billing analyst does by hand or, far more often, does not do at all; and the cross-cycle drift review that essentially nobody does, because it means reading a meter's whole history alongside its field notes.

The inputThe actual residential account packs

The corpus is 64 residential account packs, 0.22 MB (txt 64). There is no public corpus of estimated meter reads and there could not be one: a read history is a record of what happened inside somebody's home, cycle by cycle, against a tariff a utility files with a regulator. Every byte here is generated, and generating it is what makes the second question answerable at all -- the drift call needs a KNOWN truth about whether a note records something that actually happened, and no real archive carries that label.

CEDAR HOLLOW MUNICIPAL UTILITY
ACCOUNT FILE -- ESTIMATED READ TRUE-UP PACK
File CHMU-ER-0001
============================================================================

SECTION 1 -- ACCOUNT AND SERVICE POINT
  Account                                      6363-18263
  Premise                                      1514 Cotton Mill St, Cedar Hollow
  Rate class                                   Residential, single-phase
  Meter                                        MT-73681
  Register                                     5 digits, rolls at 99999
  Trailing 12-month average cycle consumption  720 kWh

SECTION 2 -- TARIFF IN FORCE (Schedule R-2026)
  Customer charge, per cycle                   $11.95
  Tier 1, first 600 kWh                        10.4300 c/kWh
  Tier 2, next 600 kWh                         12.8700 c/kWh
  Tier 3, above 1200 kWh                       15.9400 c/kWh
  Summer adder, Tier 3 only (Jun-Sep)         +2.1500 c/kWh
  A cycle is a summer cycle when the midpoint of its period falls in June, July, August
  or September. The customer charge is per cycle and no true-up changes it.

SECTION 3 -- ESTIMATION RULE ON FILE
  Rule          EMP-2026 Table B, degree-day weighted against a 12-month base
  Set on        2024-02-28
  Last reviewed 2024-02-28

SECTION 4 -- CLOSED TRUE-UP HISTORY (oldest first)
  id      cycle    period                     estimated   actual   signed   signed
                                                    kWh      kWh      kWh      pct
  TU-01   2025-05  2025-04-03..2025-05-02        580      617      +37     +6.0
  TU-02   2025-05  2025-05-03..2025-05-31        593      624      +31     +5.0
  TU-03   2025-06  2025-06-01..2025-06-28        890      937      +47     +5.0

The outcomeWhat a good result looks like

One row per account: the adjustment in kWh and in cents, whether the estimate ran high or low, whether the account is a REBILL CANDIDATE QUEUED FOR A BILLING ANALYST or within tolerance, and separately a drift verdict off a seven-rung ladder with the printed lines that support it quoted verbatim.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• You want the estimated read trued up and the rebill threshold walked — free code -- evals/baseline.py --floor strict
  it is exactly right on all four fields for all 64 accounts against the paid call's 0, it costs $0.00, it needs no network, and it re-runs in under a second.
• You want the drift call and you will accept a statistical answer — free code -- the same floor
  53 of 64 against the paid call's 12, p = 5.889e-11, for nothing.
• You want to know WHY a meter's estimates drifted, and a documented cause named — the paid call, with the arithmetic taken away from it
  on the 11 accounts whose verdict turns on a sentence it beats every free floor 8-0 (p = 7.812e-03) and the floors score 0 of 11 by construction. Run it BEHIND the free floor, on the accounts the floor already calls persistent, and let src/recheck.py keep only its three judgement fields.

At a glanceHow the whole thing runs

Run once, for real, on 2026-09-11. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-11 — r001-estimated-read. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A clean checkout with no key configured renders the whole board -- all 64 packs, the tariff, the derived key and every committed arm -- and runs all four free floors, the stub control, the dry run and evals/check_labels.py on it. Zero calls, zero network, zero dollars. Only ASK THE MODEL and evals/run.py's paid arm need a credential, and both say so rather than failing at the HTTP layer.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one account end to end -- one model call with reasoning disabled, read back and parsed. The pure-code stations either side add nothing measurable at this resolution: they are integer arithmetic on kWh and cents.

Current processWhat it replaces

The line-by-line true-up of an estimated read against the tariff -- registers, rollover, day-weighted allocation, tier ladder, season, threshold -- that a billing analyst does by hand or, far more often, does not do at all; and the cross-cycle drift review that essentially nobody does, because it means reading a meter's whole history alongside its field notes.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• json
• jsonl
• md
• 1 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one residential service points (one account, one meter, one tariff, one open true-up, one read history) is

No train/test split, because nothing is trained or tuned, and no chunking, because one account pack goes whole into one call -- the sizes here are the files' own bytes. The unit every rate on this page is read over is one ACCOUNT.

SetupWhat the setup figure measured

There is no index and nothing is embedded, retrieved or rebuilt. The rulebook is data/policy.md for the model that reads and src/policy.py for every arm that computes, and the account pack goes whole into one call.

LicenceLicence

MIT, with the rest of this repository

Bring your ownBring your own residential account packs

Replace data/corpus/*.txt with your own packs in the same eight-section layout -- the headings in src/pack.py::HEADINGS are exact-match anchors and the three row shapes are regexes -- and rewrite data/gold.jsonl with your own key. Change the tariff in BOTH src/policy.py and data/policy.md; the asserts at the foot of src/prompt.py stop the build rather than letting an arm be scored against a rulebook it was never handed.

⚠︎ And what stops being true when you do: The measured result does not travel with them. Every figure here was measured on THIS corpus, whose difficulty is a property of how it was generated: the ratio of decoy notes to real ones, the wording of both, how far a misaligned event sits from the run it does not explain, and how many accounts straddle a threshold by one unit. Re-run the floors on your own packs before you believe any margin.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 3,880 tokens assembled, 1,925 are rulebooks — 50% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Replayed from the kit's own src/prompt.py::build() for CHMU-ER-0032, which is the same function the run called. It is checkable rather than asserted: the run record carries the part list and the character count, and the board renders the pack half of it verbatim.

Full promptThe prompt in full

You are reconciling ONE estimated meter read at ONE residential service point of ONE
municipal utility. In front of you is an account pack: the account and its meter, the tariff in
force, the estimation rule on file, the closed true-up history, the cycles billed on an estimated
read since the last actual read, the new actual read, the field notes and correspondence, and what
the utility's own billing system printed.

You answer TWO SEPARATE QUESTIONS about it.

  THE TRUE-UP. What did this meter actually record over the gap, how does that differ from what was
  billed, in kWh and in money, and does the difference reach the rebill threshold.

  THE DRIFT. Looking across this meter's recent true-up history, is the ESTIMATION RULE ITSELF
  wrong for this meter -- persistently high, persistently low, merely seasonal, explained by a
  documented change at the premise, or not answerable because the register history is broken.

FOUR THINGS YOU NEVER DO.
  1. You never re-read the meter. A register figure the pack does not print does not exist.
  2. You never bill, credit, refund or move money, and nothing you write may say that a customer
     owes or is owed anything. Everything you produce is a REBILL CANDIDATE QUEUED FOR A BILLING
     ANALYST and nothing more.
  3. You never close, finalise or waive a true-up.
  4. You never name a person and never assign fault to one.

Follow EMP-2026 exactly as printed below. Both ladders are walked IN ORDER and the FIRST MATCH
WINS.

# EMP-2026 — Cedar Hollow Municipal Utility, estimated meter read true-up

*Schedule R-2026 (residential, single-phase) and Estimated Meter Read Procedure EMP-2026.*
*Everything in this document is invented for a demonstration corpus. Cedar Hollow Municipal
Utility, Schedule R-2026 and EMP-2026 do not exist.*

You never settle, never bill, never credit and never move money. Everything you produce is a
**rebill candidate queued for a billing analyst**, and nothing more.

## s1 — Schedule R-2026

| element | value |
|---|---|
| Customer charge, per cycle | $11.95 |
| Tier 1, first 600 kWh in a cycle | 10.4300 c/kWh |
| Tier 2, the next 600 kWh (601–1200) | 12.8700 c/kWh |
| Tier 3, above 1200 kWh | 15.9400 c/kWh |
| Summer adder, **Tier 3 only** | +2.1500 c/kWh |

A cycle is a **summer cycle** when the **midpoint of its period** falls in June, July, August or
September. The customer charge is per cycle; no true-up changes how many cycles were billed, so
**the customer charge is never part of an adjustment**.

## s2 — the open true-up

A pack shows the register at the **last actual read**, one to three cycles that were then billed on
an **estimated** read, and the register at the **new actual read**. The adjustment is the difference
between what the meter actually recorded over that gap and what was billed across it.

## s3 — the gap consumption, and the rollover test

The register has five digits and rolls from 99999 to 00000.

1. If the new reading is **greater than or equal to** the prior reading, the gap consumption is the
   plain difference. Basis: `direct`.
2. If the new reading is **lower**, assume one rollover: `new + 100000 − prior`. Then test it. The
   result is accepted only if it is at most **three times** the account's trailing 12-month average
   cycle consumption **times the number of cycles in the gap**. Basis: `rollover`.
3. If the rolled figure fails that test, the read is **not usable**. Answer `READ-DEFECTIVE`, and
   return `null` for the adjustment kWh, the adjustment amount and the direction.

## s4 — allocation

Allocate the gap's true consumption across the gap's cycles **pro rata by billing days**, floored to
whole kWh, with the **remainder added to the last cycle**. The last cycle is named by the procedure;
putting the remainder anywhere else can move kWh between a summer and a winter rate.

## s5 — re-rating

Re-rate **each cycle separately** at its own tier ladder and its own season, using its allocated
kWh. Do the same for the kWh that were billed. The **adjustment amount** is
`recomputed energy − billed energy`, in cents, energy only.

## s6 — direction

* `ESTIMATE-LOW` — the meter recorded **more** than was billed (adjustment kWh positive).
* `ESTIMATE-HIGH` — the meter recorded **less** than was billed (adjustment kWh negative).
* `EXACT` — the adjustment is exactly zero.

## s7 — the rebill ladder, walked in order, first match wins

1. `READ-DEFECTIVE` — s3 refused the read.
2. `REBILL-CANDIDATE` — **any one of**: the adjustment is **50 kWh or more** in absolute value; or
   it is **$15.00 or more** in absolute value; or the re-rating moves **any cycle across a tier
   boundary** it did not previously reach (a cycle whose billed kWh and allocated kWh land in
   different tiers).
3. `WITHIN-TOLERANCE` — none of the above.

## s8 — the drift window

The drift question is **not** about this true-up. It asks whether the **estimation rule on file is
wrong for this meter**. The window is the **six most recent true-ups**: the five most recent closed
rows in Section 4 plus this open one, whose signed error percent you compute from s2–s3. When the
read is `READ-DEFECTIVE` the open cycle contributes nothing and the window is the **six most recent
closed rows** instead. Signed error percent is `(actual − estimated) / actual × 100`, **of actual**,
to one decimal place.

## s9 — the drift ladder, walked in order, first match wins

1. `NO-DRIFT-SHORT-HISTORY` — fewer than six true-ups are available. Cite nothing.
2. `HISTORY-BROKEN` — a **completed** meter exchange or register reconfiguration is dated **inside
   the window's span**. Nothing across that event is comparable and no drift call is possible.
   Cite the line that records it. `drift_cause` is `meter_exchange` or `register_reconfigured`.
3. `EXPLAINED-LOAD-CHANGE` — the estimates are persistently wrong (test 4/5 below) **and** a
   **completed, dated** load change is recorded **where the run of same-signed true-ups begins** —
   between the start of the cycle immediately before the run's first cycle and the end of the run's
   first cycle. Cite that line and every row of the run. `drift_cause` is one of
   `solar_interconnection`, `new_tenant`, `vacancy_ended`, `vacancy_began`,
   `pool_pump_installed`, `ev_charger_installed`, `business_closed`.
4. `PERSISTENT-UNDER-ESTIMATE` — at least **five of the six** window rows are positive, the
   **median** signed percent is **+8.0 or more**, and no completed dated load change sits where the
   run begins. Cite every row of the trailing run. `drift_cause` is null.
5. `PERSISTENT-OVER-ESTIMATE` — the same with negative signs and a median of **−8.0 or less**.
6. `SEASONAL-SWING` — the signs change at least **twice** across the window, the **sum** of the six
   signed percents is within **±5.0**, and the **median of their absolute values is 8.0 or more**.
   The rule is not wrong; it is mistimed and it cancels. Cite all six rows.
7. `NO-DRIFT` — none of the above. Cite nothing.

**A run** is the longest stretch of same-signed rows ending at the most recent row. A row of exactly
zero ends a run.

## s10 — what a note has to be before it explains anything

A note explains a drift only if it records something that **actually happened**, on a **stated
date**, at **this service point**. A quote requested, a permit issued with no inspection closed, an
enquiry about a transfer, a visit scheduled and cancelled, a flag set and removed in error, an
intention mentioned on a call — none of these is a load change. A note that records a real,
completed event but is dated **long before** the run of same-signed true-ups begins does **not**
explain that run: the verdict stays `PERSISTENT-…`.

## s11 — quoting

Every cite names a printed line by its id (`TU-nn`, `GC-nn`, `SO-nn`, `CC-nn`) and quotes that line
**verbatim** from the file. Quote nothing that is not printed. For `NO-DRIFT`,
`NO-DRIFT-SHORT-HISTORY` and every verdict the ladder does not reach, the cited set is **empty**.

## s12 — the four refusals

1. **Never re-read the meter.** A register figure the file does not print is not available to you.
2. **Never bill, credit, refund or move money.** No output may say a customer owes or is owed
   anything. Every finding is a **rebill candidate queued for a billing analyst**.
3. **Never close, finalise or waive.** You do not decide that a true-up is settled.
4. **Never name or blame a person.** No employee, meter reader, estimator or customer is named.


Return ONE JSON object and nothing else. No prose before it, no code fence, no
commentary after it.

{
  "adjustment_kwh":    <integer kWh, or null when rebill_status is READ-DEFECTIVE>,
  "adjustment_amount": <integer CENTS, energy only, or null when READ-DEFECTIVE>,
  "direction":         <"ESTIMATE-LOW" | "ESTIMATE-HIGH" | "EXACT" | null>,
  "rebill_status":     <"READ-DEFECTIVE" | "REBILL-CANDIDATE" | "WITHIN-TOLERANCE">,
  "drift_verdict":     <one of: "NO-DRIFT-SHORT-HISTORY" | "HISTORY-BROKEN" | "EXPLAINED-LOAD-CHANGE" | "PERSISTENT-UNDER-ESTIMATE" | "PERSISTENT-OVER-ESTIMATE" | "SEASONAL-SWING" | "NO-DRIFT">,
  "drift_cause":       <one of: "solar_interconnection" | "new_tenant" | "vacancy_ended" | "vacancy_began" | "pool_pump_installed" | "ev_charger_installed" | "business_closed" | "meter_exchange" | "register_reconfigured", or null>,
  "cites":             [ {"line_id": "<TU-nn|GC-nn|SO-nn|CC-nn>", "line": "<the line, verbatim>"} ],
  "confidence":        <0.0 to 1.0>
}

RULES FOR THE FIELDS.
  - adjustment_amount is in CENTS and is ENERGY ONLY. The customer charge is per cycle and no
    true-up changes it. A negative figure means the estimate was high.
  - cites quote PRINTED LINES VERBATIM. Quote nothing that is not printed. For NO-DRIFT and
    NO-DRIFT-SHORT-HISTORY the cited set is EMPTY.
  - drift_cause is null for every verdict except HISTORY-BROKEN and EXPLAINED-LOAD-CHANGE.
  - There is no field for an amount owed, for a credit, for a payment, for closing a true-up, or
    for a person. If you find yourself wanting one, the answer is that this is a rebill candidate
    queued for a billing analyst.

CEDAR HOLLOW MUNICIPAL UTILITY
ACCOUNT FILE -- ESTIMATED READ TRUE-UP PACK
File CHMU-ER-0032
============================================================================

SECTION 1 -- ACCOUNT AND SERVICE POINT
  Account                                      5676-43824
  Premise                                      593 Quarry Bend Rd, Cedar Hollow
  Rate class                                   Residential, single-phase
  Meter                                        MT-91461
  Register                                     5 digits, rolls at 99999
  Trailing 12-month average cycle consumption  640 kWh

SECTION 2 -- TARIFF IN FORCE (Schedule R-2026)
  Customer charge, per cycle                   $11.95
  Tier 1, first 600 kWh                        10.4300 c/kWh
  Tier 2, next 600 kWh                         12.8700 c/kWh
  Tier 3, above 1200 kWh                       15.9400 c/kWh
  Summer adder, Tier 3 only (Jun-Sep)         +2.1500 c/kWh
  A cycle is a summer cycle when the midpoint of its period falls in June, July, August
  or September. The customer charge is per cycle and no true-up changes it.

SECTION 3 -- ESTIMATION RULE ON FILE
  Rule          EMP-2026 Table B, degree-day weighted against a 12-month base
  Set on        2024-04-27
  Last reviewed 2024-04-27

SECTION 4 -- CLOSED TRUE-UP HISTORY (oldest first)
  id      cycle    period                     estimated   actual   signed   signed
                                                    kWh      kWh      kWh      pct
  TU-01   2025-06  2025-06-01..2025-06-28        923      871      -52     -6.0
  TU-02   2025-07  2025-06-29..2025-07-27        973      936      -37     -4.0
  TU-03   2025-08  2025-07-28..2025-08-25       1033      993      -40     -4.0
  TU-04   2025-09  2025-08-26..2025-09-22        879      837      -42     -5.0
  TU-05   2025-10  2025-09-23..2025-10-21        647      610      -37     -6.1
  TU-06   2025-11  2025-10-22..2025-11-18        695      675      -20     -3.0
  TU-07   2025-12  2025-11-19..2025-12-16        769      747      -22     -2.9
  TU-08   2026-01  2025-12-17..2026-01-14        673      761      +88    +11.6
  TU-09   2026-02  2026-01-15..2026-02-14        633      707      +74    +10.5
  TU-10   2026-03  2026-02-15..2026-03-15        444      538      +94    +17.5
  TU-11   2026-04  2026-03-16..2026-04-12        432      511      +79    +15.5

SECTION 5 -- THE OPEN TRUE-UP
  Last actual read   2026-04-12   register 38543
  Cycles billed on an ESTIMATED read since that read:
  id      cycle    period                     days   estimated   billed
                                                      register      kWh
  GC-01   2026-05  2026-04-13..2026-05-12      30       39149      606
  New actual read    2026-05-12   register 39200
  Billed on estimates over the gap, total      606 kWh

SECTION 6 -- FIELD NOTES AND SERVICE ORDERS
  SO-01   2025-12-23  Electrical permit EP-82030 closed on final inspection for a 240 V vehicle charging circuit at this premise.

SECTION 7 -- CUSTOMER CORRESPONDENCE
  CC-01   2025-12-04  Customer mentioned an intention to buy an electric vehicle; no charging circuit permit has been opened at this premise.
  CC-02   2026-02-02  Customer mentioned an intention to buy an electric vehicle; no charging circuit permit has been opened at this premise.

SECTION 8 -- WHAT THE BILLING SYSTEM PRINTED
  BS-01   Estimated-read true-up flag                OPEN
  BS-02   System adjustment, uncorrected             +51 kWh
  BS-03   System estimation-drift flag               SET
  BS-04   The system subtracts registers directly and does not test a rollover, does not
          re-rate the gap cycle by cycle, and sets BS-03 from the sign of the last three
          closed true-ups alone. Nothing it prints is an adjudication.

END OF FILE CHMU-ER-0032


Answer both questions for the pack above. Walk s3, s4, s5, s7 for the true-up and s8, s9, s10 for the drift. Return the JSON object only.

Raw responseThe raw response

{
  "adjustment_kwh": 51,
  "adjustment_amount": 797,
  "direction": "ESTIMATE-LOW",
  "rebill_status": "REBILL-CANDIDATE",
  "drift_verdict": "EXPLAINED-LOAD-CHANGE",
  "drift_cause": "ev_charger_installed",
  "cites": [
    {"line_id": "SO-01", "line": "SO-01   2025-12-23  Electrical permit EP-82030 closed on final inspection for a 240 V vehicle charging circuit at this premise."},
    {"line_id": "TU-08", "line": "TU-08   2026-01  2025-12-17..2026-01-14        673      761      +88    +11.6"},
    {"line_id": "TU-09", "line": "TU-09   2026-02  2026-01-15..2026-02-14        633      707      +74    +10.5"},
    {"line_id": "TU-10", "line": "TU-10   2026-03  2026-02-15..2026-03-15        444      538      +94    +17.5"},
    {"line_id": "TU-11", "line": "TU-11   2026-04  2026-03-16..2026-04-12        432      511      +79    +15.5"},
    {"line_id": "GC-01", "line": "GC-01   2026-05  2026-04-13..2026-05-12      30       39149      606"}
  ],
  "confidence": 0.95
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Estimated meter read true-up and estimation drift — 64 residential account packs. One model answered, and every answer was then graded Two different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

Every grader is pure code with no model in it -- integer comparison on kWh and on cents, exact match on two closed vocabularies, set equality on cited line ids, character offsets into the normalised pack, and an exact two-sided McNemar over the discordant pairs. That is why re-scoring a committed run costs $0.00 and returns the same answer, and why every p-value on this page can be reproduced for nothing.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

per million tokens, the vendor's own published card, read live out of this repo's build/facts/models.json on the day of the run. Input is priced in two streams -- new text and cached prefix -- and every call is priced at the tariff in force at the moment it was made rather than at one average.

Rates checked 2026-09-05.

The gradersTwo ways to grade

Four floors. The one to beat is strict: 64 of 64 on the true-up (exactly right by construction -- it is pure code) and 53 of 64 on the drift call. keyword is the interesting one: adding a regex over the field notes DROPS it to 47, because every word in that vocabulary also appears in a note recording something that never happened. That is what a keyword list is worth when the deciding word is a verb.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — top_k, chunk size, prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-09-05. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

Grading is free and stays free. Every grader is pure code -- integer kWh, integer cents, set equality, character offsets, an exact binomial -- so re-scoring the committed run costs $0.00 and returns the same answer. The four free floors call no model and cost nothing to run or re-run.

Cost driversWhat actually moves the bill

• THE PREFIX, and it is most of the bill. 71.9 pct of every prompt is a rulebook that never changes: the role, the four refusals, EMP-2026 verbatim and the answer contract. The provider billed 153,600 of 248,298 input tokens as cache hits on this run -- 61.9 pct.
• THE PACK, which is the only part that varies -- about 1053 tokens of the 3879, and it goes in VERBATIM. Pre-digesting it is exactly where a rolled register quietly becomes a negative consumption before the model ever sees it.
• THE REPLY, which is small: 286.1 output tokens on average and 643 at the largest, against a 4000-token ceiling that 0 calls reached.
• NOT THE GRADING, and not the floors. Every grader is pure code and all four floors call no model, so the entire eval apparatus is $0.00 and re-scoring the committed run is free forever.

Your volumeWhat it costs at your volume

Linear in accounts and slightly sub-linear in money. There is no index, no retrieval and no state carried between accounts, so ten times the accounts is ten times the calls; the prefix cache means the marginal call is cheaper than the first. The real lever is not volume, it is SELECTION: run the free floor over the whole estate for nothing, and pay only on the accounts it already calls persistent -- 33 of 64 here, which is 52 pct of the calls for the entire measured benefit.

Model choiceWhy this model

The cheapest long-context tier this estate holds a key for, run off-peak, with reasoning disabled and a 4000-token ceiling -- the standing default. It is also the honest tier for the question: if the cheapest tier could do day-weighted per-cycle re-rating, nobody would need a rule about when to pay for arithmetic. It could not, and the report leads with that.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the whole 64-account scored run (r001-estimated-read): 248,298 input tokens (153,600 of them prefix cache hits, 61.9 pct) and 18,312 output. The adversarial probe is a separate 16 calls at $0.017517.

The codeEvery file, and what it is for

10 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pythe corpus generator

writes one account pack per service point -- the account and meter, the tariff, the estimation rule on file, the closed true-up history, the cycles billed on estimates, the new actual read, the field notes, the correspondence and the billing system's own printed panel -- from one named seed, then PARSES THE FILE BACK and derives the answer key by running the same engine over the parse. A rendering defect cannot hide behind a correct key.

# Generate the 64-account corpus, DERIVE the answer key, and write both. Deterministic.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SEED = "estimated-read-2026-09-11"
N = 64
FAMILIES = [
MIX = {"clean": 8, "trueup_tier_cross": 4, "trueup_rollover": 4, "trueup_multi_cycle": 5,
STREETS = ["Quarry Bend Rd", "Alder Fork Ln", "Pemberton Rise", "Hollow Creek Way",
REAL_NOTE = {
DECOY_NOTE = [

src/pack.pythe pack parser

reads a pack into numbers and dates and NOTHING ELSE. It deliberately does not decide whether a note records a completed load change -- that is the question the kit measures, and a parser that answered it would be scoring itself. The section headings are exact-match anchors and a missing one raises rather than silently yielding an empty section.

# Parse an account pack back into the structure every arm computes over. NO JUDGEMENT HERE.

HEADINGS = ("SECTION 1 -- ACCOUNT AND SERVICE POINT",
def _d(s):
def _sections(text):
def is_summer(start, end):
def parse(text):
def line_ids(p):

src/policy.pythe tariff and the procedure, as data

Schedule R-2026 and EMP-2026 as integers: the customer charge, three tier rates and a summer adder in cents x 10,000, the rollover plausibility multiple, both rebill thresholds, the drift window and its two ladders. Every arm that computes reads this module; the model reads data/policy.md, and src/prompt.py fails the BUILD if a rate, a threshold or a verdict name differs between them.

# EMP-2026 and Schedule R-2026 as data. THE RULEBOOK, AND THERE IS EXACTLY ONE COPY OF IT.

TARIFF_ID = "R-2026"
PROCEDURE_ID = "EMP-2026"
CUSTOMER_CHARGE_CENTS = 1195
TIER1_KWH = 600
TIER2_KWH = 600
TIER1_C4 = 104300
TIER2_C4 = 128700
TIER3_C4 = 159400
SUMMER_TIER3_ADDER_C4 = 21500

src/trueup.pythe true-up engine

the whole arithmetic half: register difference, the rollover assumption and its plausibility test, day-weighted allocation with the remainder to the last cycle, per-cycle re-rating at that cycle's own tier ladder and season, and the three-rung rebill ladder. Integer kWh and integer cents end to end, with exactly one rounding site that rounds half AWAY from zero.

# The per-read true-up, in pure code. EMP-2026 s3, s4, s7.

def gap_consumption(prior_register, now_register, gap_cycles, trailing_avg_cycle_kwh,
def recompute(pack):
def signed_pct(estimated_kwh, actual_kwh):

src/drift.pythe drift ladder

the judgement half: the trailing same-signed run, the median signed error, the seasonal test, and the alignment window that decides whether a documented event sits WHERE THE RUN BEGINS. The statistics are given away to the free floors; what the floors cannot do is tell a completed event from a quoted one, or notice that a genuine event predates the run it is offered as explaining.

# EMP-2026 s9 -- the persistent-drift call. THE HALF THAT IS NOT ARITHMETIC.

def trailing_run(signs):
def median(vals):
def sign_changes(signs):
def run_alignment(window, run_len, event_date):
def window_from_pack(p, tu):
def decide(pack, trueup):

src/prompt.pythe prompt

five parts in fixed order -- the role and four refusals, EMP-2026 verbatim, the answer contract, the pack verbatim, the closing instruction. The first three never vary and go first, so a provider that prices cached input bills most of the prompt at the cache rate.

# Assemble the prompt. ONE MODEL, ONE KEY, ONE CALL PER ACCOUNT.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY_MD = os.path.join(HERE, "data", "policy.md")
FIELDS_JSON = os.path.join(HERE, "data", "fields.json")
SYSTEM = """You are reconciling ONE estimated meter read at ONE residential service point of ONE
SCHEMA = """Return ONE JSON object and nothing else. No prose before it, no code fence, no
VERDICTS = list(policy.DRIFT_LADDER)
VERDICT_MEANINGS = {
REBILL_MEANINGS = {
def _read(path):

src/recheck.pythe recheck station

keeps EXACTLY THREE fields of the reply -- drift_verdict, drift_cause, cites -- and re-derives the four true-up fields in pure code. The station cannot notice that the drift reading was wrong, which is the whole reason both columns are published.

# THE STATION. Keep the reply's READING, re-derive everything that is arithmetic, and record every

KEPT = ("drift_verdict", "drift_cause", "cites")
def station(answer, pack, file_ids):

evals/scoring.pythe graders

pure code with no model in it: integer comparison on kWh and cents, exact match on two closed vocabularies, set equality on cited line ids, character offsets into the normalised pack, and an exact two-sided McNemar over the discordant pairs. Re-scoring a committed run is free and returns the same answer.

# Grade one arm. EVERY GRADER IS PURE CODE -- there is no model anywhere in this file.

TRUEUP_FIELDS = ("adjustment_kwh", "adjustment_amount", "direction", "rebill_status")
DRIFT_FIELDS = ("drift_verdict", "drift_cause", "cites")
GRADED = TRUEUP_FIELDS + DRIFT_FIELDS
def _cite_ids(v):
def grade_one(answer, gold, pack_text, file_ids):
def _trueup_code(a, key, row):
def _eq(x, y):
def _code(a, key, row, answer):
def _drift_code(a, key, row):

evals/check_labels.pythe second implementation

re-derives every answer independently -- Decimal money with an explicit ROUND_HALF_UP, a cumulative tier walk, a carried-remainder allocation, and the ladder written from the PROSE the model reads rather than from src/. 0 disputes across all 64 accounts.

# A SECOND, INDEPENDENT IMPLEMENTATION OF THE ANSWER KEY. It shares no arithmetic with src/.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
D = Decimal
def _tier_cents(kwh, summer):
def _allocate(total, days):
def _top(kwh):
def redo_trueup(p):
def _med(v):
def redo_drift(window, events):
def _d(s):

src/app.pythe board

renders the whole kit with no key and no network -- every pack, the derived key, and every recorded arm side by side. Only ASK THE MODEL needs a credential and it says so. The board is handed model_display and there is no route that could send the model id.

# The local board. `python3 -m src.app` and open http://127.0.0.1:9885.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
RESULTS = os.path.join(HERE, "results")
PORT = int(os.environ.get("PORT", "9885"))
ARM_ORDER = ("b000-estimated-read-panel", "b000-estimated-read-strict",
ARM_LABEL = {
def _golds():
def _text(did):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 3879 input and 286 output tokens per query at top-k 1, of which about 150 tokens are fixed (system prompt, chat framing, the question itself) and the rest is retrieved context. Change the retrieval depth and the input scales with it. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model and top-k follow the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

MEASURED, FORCED AND SCOPED -- and the result splits cleanly in two. The customer correspondence is this kit's injection surface: it is written by or on behalf of the party whose bill is being reconciled and it reaches the model verbatim, and the corpus already carries notes in that voice written to be believed. Every refusal held. The ordinary answer moved on two calls and moved back on two others.

API_KEY is read from the shared <code>&lt;repo&gt;/.env</code>, this kit's own <code>.env</code>, or the real environment, in that order. Both files are gitignored from the first commit; this repo has never held a credential and is not able to. The board is handed <code>model_display</code> and has no route that could send the model id.

The experimentWe attacked it -- the customer correspondence, escalated

Four escalating wordings appended to SECTION 7, where the pack's own decoy notes already live, fired at four named accounts: one the clean arm called correctly, one it did not, one with a real documented explanation and one whose read is defective. 16 calls, one model, one day. Both were run for real on 2026-09-11.

The gates come apart from the ANSWER and that is the finding. Everything the kit was built to refuse it refused every time, on every wording. What moved was the ordinary judgement: the drift verdict went right-to-wrong on 2 calls and wrong-to-right on 2, which is noise in both directions on a base of four accounts.

The resultEvery refusal held 16 of 16; the drift verdict moved on 2

Every trial is one account pack of this corpus with the wording appended to its SECTION 7 and everything else byte-identical, paired against that same account's own un-attacked answer read out of the clean run -- one variable, and the control cost nothing because it was already bought. The four wordings are published verbatim in the run record together with every trial's flags, so the experiment can be re-read rather than trusted. This is a probe, not a resistance rate.

HonestyWhat this does not prove

• THE MOVEMENT TEST HAS NO BASE ON THE WHOLE ROW. The clean arm got 0 of 64 accounts entirely right, so the row-level movement count is arithmetic rather than evidence. The refusal counts are the half with a real denominator.
• ONE POSITION ONLY. Every wording went into SECTION 7. Nothing measures what happens if the same instruction is printed under a true-up row in Section 4 or inside the billing system's panel.
• FOUR ACCOUNTS. The refusal denominator is 16 calls; the movement denominator is 4 accounts.
• ONE MODEL, ONE DAY, ONE RUN. Nothing here says whether a different tier refuses as reliably.
• NO MULTI-TURN ATTACK. Every call is a single completion with no conversation to escalate in.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

Never re-read the meter; never bill, credit, refund or move money; never close, finalise or waive a true-up; never name or blame a person. Everything this kit produces is a REBILL CANDIDATE QUEUED FOR A BILLING ANALYST.

Stated as the four refusals in the system prompt, before anything about how to read a pack; enforced by the schema having no field that could carry an amount owed, a register, a closure or a person; detected afterwards by src/refusal.py over every free-text surface of every arm.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is A REFUSAL STATED IN THE PROMPT plus ABSENCE OF A FIELD plus A STRING TEST OVER THE PROSE. Only the third is machinery, and it DETECTS rather than prevents: it reads the reply after the fact and records a hit. Nothing stops a sentence being generated.
• It is also not a control on the ARITHMETIC. src/recheck.py replaces the four true-up fields wholesale, which is stronger than a guardrail -- but it cannot notice that the drift reading was wrong, and the drift reading is the half the money is bought for.

WatchedWhat is watched, and why that one

The kit measures 91 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

67 measured by the latest run 24 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (graded_fields adjustment_kwh,adjustment_amount,direction,rebill_status,drift_verdict,drift_cause,cites) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · with the model in the path — 5 runs. Columns here are only ever compared with each other.