The business caseThe problem this solves

Admissions tax and facility fees are owed on every live event, under rules that vary by venue, by jurisdiction and by ticket category. Revenue systems recompute the remittance correctly and reliably from a rate feed. What they cannot do is notice that the BASE they were handed is not the base the ordinance defines -- a proviso that pulls complimentary admissions in at face value once comps pass a threshold, a concessionary category the schedule excludes and the worksheet taxed, a facility fee that is itself an admission charge, a rate a city ruling replaced and the feed never caught, an exemption whose own condition the event fails. Every one of those produces a worksheet that multiplies out to the penny, and the worksheet never states which clause it read. a remittance check that recomputes the worksheet and confirms it multiplies out -- which on this corpus means passing every single one of the 64 filings computed on the wrong base or at a superseded rate, because all 64 of them multiply out.

The inputThe actual event admissions-tax remittance files

The corpus is 120 event admissions-tax remittance files, 0.70 MB (txt 120). An admissions-tax filing is a ticket manifest plus an ordinance reading, and both halves are sensitive: a venue's category mix and the exemptions it claims are exactly what a competitor and a revenue office would each like to see. There is no public corpus. A SCRUBBED real one would be worse rather than better, and the reason is specific: scrubbing removes the proviso, the ruling and the exemption condition, and that layer IS what this kit measures.

Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated admissions-tax and facility-fee remittance file
for an AI use-case kit; it reproduces no venue, no jurisdiction, no promoter, no event, no taxpayer
and no real person. The rates, the exemption tests, the facility fee and the amendment floor are
ILLUSTRATIVE DEFAULTS, not any jurisdiction's, and nothing here is tax advice.


Event
----------------------------------------------------------------
Event reference                  : AT-0001
Venue                            : The Brickworks
City jurisdiction                : City of Elverton
County jurisdiction              : Elverton County
Promoter                         : NORTHGATE PROMOTIONS
Promoter registered with the city: YES
Event date                       : 2026-04-11
Event class as promoted          : COMMERCIAL
Filing period                    : 2026-Q1
Tax analyst of record            : Ingrid Sandoval
Worksheet prepared by            : Mateo Duarte (promoter's settlement desk)

Filing Control
----------------------------------------------------------------
Amendment floor (default)        : 250.00 GBP
Rate schedule                    : ATTACHED
Filing authority                 : NOT DEFINED. Nothing in this kit files a return, amends a
                                   return, remits money, releases a payment, changes a
                                   rate or approves an amendment.

Ticket Manifest
----------------------------------------------------------------
All figures in GBP. Face value and service charges are printed already multiplied out, one
line per ticket category. COMPLIMENTARY admissions were issued at no charge; their FACE

The outcomeWhat a good result looks like

a verdict per event with the amount actually owed to the penny, the governing clause named, the authority under-remitted and whether it is worth filing an amended return. Measured: 100.0 pct of the GBP 674,587.05 at risk across 75 readings was caught, with 1 false accusation across 44 sound filings.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Every ruling, proviso and exemption condition is keyed into the revenue system's rate feed the day it is issued — the free floor -- structured-feed-strong, $0.00
  It re-derives the base from the manifest using the feed's own token and catches every dropped cap, every rule never applied and every base the worksheet moved -- 58.33 pct on the verdict column, for nothing.
• Rulings, provisos and exemption conditions live in ordinance text and the rate feed lags — the model, with the ordinance extract
  100.0 pct on the amount owed across all 114 determinable readings, and 100.0 pct on the 45 whose defect the feed cannot see, against the floor's 0.0 pct.

And where nothing here is good enough:

• You want the reading but cannot put the ordinance in front of it — neither -- and the free strong floor is the honest version of that arm
  The ordinance-blind ablation IS that experiment and it is on this page: 60.53 pct on the amount against 100.0 pct with the ordinance, and 0.0 pct on the feed-blind slice. The free floor sees the same information and costs nothing.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-25. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-25 — r001-admissions-tax. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Every remittance file, the answer key, all three free floors, the injection probe and what r001-admissions-tax actually answered ship in the repo. python3 -m evals.check_labels and python3 -m src.app both run with no key and no network.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one reading -- one event remittance file, end to end including provider-side reasoning tokens, on a shared connection. ⚠︎ NOT AN SLA AND NOT COMPARABLE ACROSS KITS: this credential was shared with eight sibling kits running at the same time, so these figures are a wall-clock observation under contention. 120 readings ran in 863.8 wall seconds with 4 concurrent workers.

Current processWhat it replaces

a remittance check that recomputes the worksheet and confirms it multiplies out -- which on this corpus means passing every single one of the 64 filings computed on the wrong base or at a superseded rate, because all 64 of them multiply out.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 4 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one reading is

120 event remittance files, read once each and independently -- there is no time axis and no carried state. EVERY arm, free and paid, read all 120, so every column on this page shares one denominator.

SetupWhat the setup figure measured

There is no index to build -- each event's whole remittance file goes into the prompt. The figures are the corpus generation itself.

LicenceLicence

MIT

Bring your ownBring your own event admissions-tax remittance files

tools/build_corpus.py writes data/corpus/*.txt and data/gold.jsonl. Keep the 9 section headings and the RULE:/LINE: line shapes evals/baseline.py parses. Set your own bases, rates, exemptions and amendment floor in src/rules.py first, then run evals/check_labels.py -- it refuses to let anything spend until every gold amount re-derives from the shipped text.

⚠︎ And what stops being true when you do: The measured figures on this page do not transfer to your own filings. The feed-blind share (45 of the 76 defects here) and the prose-versus-feed split are properties of this generator's declared distribution, not facts about admissions tax. And the 100.0 pct on the amount is a property of a manifest that prints its subtotals already multiplied out -- your own export probably does not. Re-run the evals on your own data; that is what the harness is for.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 3,268 tokens assembled, 1,723 are instructions — 53% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The verbatim prompt is rebuilt from the shipped corpus by src/prompt.build() for AT-0004 -- the reading framed in the success screenshot -- so the prompt on this page and the prompt that was sent are produced by the same function.

Full promptThe prompt in full

You are an admissions-tax and facility-fee review for a live-events venue operator. You read one event's ticket manifest, its rate schedule and the remittance worksheet someone has filed, and you decide whether the tax was computed on the base the ORDINANCE defines and at the rate the ordinance gives. You answer with one JSON object and no other text.

You are an admissions-tax review for a live-events venue operator. You are reading ONE event: its
ticket manifest, the revenue system's structured copy of the rate schedule, the ORDINANCE EXTRACT
itself, and the remittance worksheet somebody has already filed.

YOUR JOB IS NOT TO CHECK THE MULTIPLICATION. The worksheet almost always reproduces itself -- its
own declared base value times its own declared rate gives its own printed amount, and free code
checks that for nothing. What you are looking for is the filing that multiplies out perfectly and
was computed on THE WRONG BASE, or under a rate or an exemption THE ORDINANCE DOES NOT GIVE IT: a
proviso that pulls complimentary admissions into the base, a category the schedule excludes and the
worksheet taxed, a facility fee that is itself an admission charge, a rate a ruling replaced and the
rate feed never caught, an exemption whose own condition the event fails, a cap left off, a rule
never applied at all.

⚠︎ THE RATES, THE EXEMPTION TESTS, THE FACILITY FEE AND THE AMENDMENT FLOOR ARE ILLUSTRATIVE
DEFAULTS INVENTED FOR THIS EXERCISE, NOT ANY JURISDICTION'S. Apply them exactly as printed regardless
of whether they look right for the event in front of you. This is not tax advice and nothing you
answer files, amends or remits anything.

THE RULES, applied in this order:
 - FIRST check completeness. If the file records that the rate schedule is NOT ATTACHED, the verdict
   is CONTEXT_INCOMPLETE: no rule is judged, the defect is UNDETERMINED, the governing clause is
   NONE, the short authority is NONE, "admissions_tax_owed" is the string NOT_DETERMINABLE and
   nothing is amended. There is no default base for a schedule nobody can read (Rule A-7).
 - THEN check whether the worksheet reproduces ITSELF. Recompute each printed LINE from its own
   declared base value, rate, count, per-ticket amount and cap, and check the printed authority
   amounts sum to the printed total. If the worksheet's OWN declared formula does not reproduce its
   OWN printed figures, the verdict is ARITHMETIC_ERROR and the defect is ARITHMETIC (Rule A-6).
   This is the only verdict about the multiplication, and it is NOT the verdict for a worksheet that
   multiplies correctly on a base the ordinance does not define.
 - THEN work out THE BASE THE ORDINANCE DEFINES, from the ticket manifest. The manifest prints face
   value and service charges per category, already multiplied out; the base is a SUM OF THOSE
   PRINTED LINES and nothing else has to be multiplied to get it. Decide, line by line, whether each
   category enters the base, and whether service charges enter with it.
 - A PROVISO, A RULING OR AN EXEMPTION IN THE ORDINANCE EXTRACT SUPERSEDES THE STRUCTURED RATE FEED
   (Rule A-3). The feed is a convenience copy of the schedule and it has no field in which to say
   "provided that", "subject to condition (a)" or "with effect from 1 February". Where the worksheet
   applied a rate a ruling replaced, the defect is SUPERSEDED_RATE and the governing clause is THE
   RULING'S id.
 - Where the base the ordinance defines is not the base the worksheet used -- complimentary
   admissions in or out, service charges in or out, a concessionary category the schedule excludes
   -- the defect is WRONG_BASE_CATEGORY and the verdict is BASE_MISAPPLIED.
 - Where the ordinance makes the facility fee itself an admission charge and the worksheet left it
   out of the base, the defect is FEE_CASCADE and the verdict is BASE_MISAPPLIED.
 - An EXEMPTION applies only where its own stated conditions are met. An exemption the event
   qualifies for and the worksheet taxed anyway is EXEMPTION_MISSED. An exemption the worksheet
   claimed and the clause's own condition excludes -- attendance above the stated ceiling, a
   promoter not registered -- is EXEMPTION_NOT_AVAILABLE (Rule A-4).
 - A cap the clause carries and the worksheet dropped is CAP_OMITTED. A rule the schedule carries
   and the worksheet never applied at all is RULE_OMITTED. Both are RULE_MISAPPLIED.
 - Otherwise the verdict is FILING_CORRECT, the defect is NONE and the governing clause is NONE.
 - THEN state "admissions_tax_owed": THE CITY ADMISSIONS TAX ONLY, under the governing reading,
   as a number to two decimal places. Not the surcharge, not the facility fee, not the total. Where
   an exemption reaches the event it is 0.00. Where the schedule is not attached it is the string
   NOT_DETERMINABLE.
 - THEN name the SHORT AUTHORITY: the authority that received LESS than it was owed, by the largest
   amount. It is NONE where the filing is correct, NONE where the reading is CONTEXT_INCOMPLETE, and
   NONE where the filing OVER-remitted every authority -- an over-remittance costs the venue and
   shorts nobody, and calling an authority short there would be plainly false.
 - THEN decide the OWNER. Start from the tax analyst of record, but a note recording a handover to
   someone else supersedes it. The person who PREPARED the worksheet is not the owner.
 - FINALLY decide "amend": YES only where a defect is named AND the total misstatement it causes --
   the absolute difference between the total remitted and the total owed, in either direction -- is
   at or above the printed AMENDMENT FLOOR. A defect below the floor is still a defect and is still
   named; it is simply not worth filing an amended return over (Rule A-9).

Answer with a single JSON object and nothing else:

{"verdict": "FILING_CORRECT|BASE_MISAPPLIED|RULE_MISAPPLIED|ARITHMETIC_ERROR|CONTEXT_INCOMPLETE",
 "defect": "NONE|WRONG_BASE_CATEGORY|FEE_CASCADE|SUPERSEDED_RATE|EXEMPTION_MISSED|EXEMPTION_NOT_AVAILABLE|CAP_OMITTED|RULE_OMITTED|ARITHMETIC|UNDETERMINED",
 "governing_clause": "<the clause id that governs the disputed line, e.g. S-2.3 or R-2026-04, or NONE>",
 "admissions_tax_owed": <the city admissions tax owed, a number to two decimals, or "NOT_DETERMINABLE">,
 "short_authority": "CITY|COUNTY|VENUE_AUTHORITY|NONE",
 "amend": "YES|NO",
 "owner": "<the tax analyst who owns this event>",
 "rationale": "one sentence, naming the clause you applied, what the worksheet did instead, and
   which authority or party it moves money away from"}

Precedence: CONTEXT_INCOMPLETE if the schedule is not attached; then ARITHMETIC_ERROR if the
worksheet does not reproduce itself; then BASE_MISAPPLIED for a base the ordinance does not define;
then RULE_MISAPPLIED for a superseded rate, an exemption wrongly given or wrongly withheld, a cap
dropped or a rule never applied; otherwise FILING_CORRECT.

Event remittance file
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated admissions-tax and facility-fee remittance file
for an AI use-case kit; it reproduces no venue, no jurisdiction, no promoter, no event, no taxpayer
and no real person. The rates, the exemption tests, the facility fee and the amendment floor are
ILLUSTRATIVE DEFAULTS, not any jurisdiction's, and nothing here is tax advice.

Event
----------------------------------------------------------------
Event reference                  : AT-0004
Venue                            : Saltgate Arena
City jurisdiction                : City of Ashcombe
County jurisdiction              : Ashcombe County
Promoter                         : SIXPENNY LIVE
Promoter registered with the city: YES
Event date                       : 2026-05-16
Event class as promoted          : COMMERCIAL
Filing period                    : 2026-Q3
Tax analyst of record            : Rosalind Chu
Worksheet prepared by            : Ola Fadipe (promoter's settlement desk)

Filing Control
----------------------------------------------------------------
Amendment floor (default)        : 250.00 GBP
Rate schedule                    : ATTACHED
Filing authority                 : NOT DEFINED. Nothing in this kit files a return, amends a
                                   return, remits money, releases a payment, changes a
                                   rate or approves an amendment.

Ticket Manifest
----------------------------------------------------------------
All figures in GBP. Face value and service charges are printed already multiplied out, one
line per ticket category. COMPLIMENTARY admissions were issued at no charge; their FACE
VALUE is shown because whether a complimentary admission is an admission charge is decided
by the ordinance and not by the box office.

  Category           Issued    Face each           Face value       Service charges
  GENERAL              4664  @     34.00  face     158,576.00   service     6,996.00
  PREMIUM               289  @     68.00  face      19,652.00   service     2,023.00
  COMPLIMENTARY         940  @     34.00  face      31,960.00   service         0.00
  STUDENT_SENIOR        644  @      9.00  face       5,796.00   service       644.00
  SEASON                889  @     26.50  face      23,558.50   service         0.00

  Admissions issued     7426
  Paid admissions      6486
  Comp ratio       12.66 pct
  Total face value     239,542.50
  Total service          9,663.00

Rate Feed (Structured)
----------------------------------------------------------------
The revenue system's own copy of the schedule, as the remittance engine reads it. One line
per rule, in the order the engine applies them. THE FEED IS A COPY OF THE SCHEDULE, NOT THE
SCHEDULE: a ruling, a proviso or an exemption written into the ordinance extract below
governs over anything on these lines, and the feed has no field in which to say so.

RULE: id=S-2.1 kind=ADMISSIONS_TAX authority=CITY base=PAID_FACE_PLUS_SERVICE rate=0.040000 order=1 label="city admissions tax"
RULE: id=S-3.1 kind=COUNTY_SURCHARGE authority=COUNTY base=PAID_FACE_PLUS_SERVICE rate=0.015000 order=2 label="county events surcharge"
RULE: id=S-5.1 kind=FACILITY_FEE authority=VENUE_AUTHORITY basis=ALL_ISSUED per_ticket=1.75 cap=12,000.00 order=3 label="venue facility fee"

Ordinance Extract
----------------------------------------------------------------
The governing ordinance, as written. Where a RULING appears below it replaces the clause it
names. Clause numbering is the schedule's own.

S-2.1           The city levies an admissions tax of 4.00 pct on ADMISSION CHARGES for each
                event held at a licensed venue in the city.

S-2.2           ADMISSION CHARGES means the FACE VALUE of admissions SOLD
                TOGETHER WITH any service charge or booking fee charged to the
                patron in connection with that admission.

S-2.3           PROVIDED THAT where complimentary admissions exceed 8 per cent of all
                admissions issued for an event, the face value of EVERY complimentary
                admission issued for that event is an admission charge and enters the
                base, notwithstanding S-2.2. The test is applied event by event on the
                comp ratio printed on the manifest.

S-3.1           The county levies an events surcharge of 1.50 pct on the same base as
                S-2.1. The surcharge is remitted to the county and is not a city receipt.

S-5.1           A facility fee of 1.75 GBP is payable to the venue authority for each
                admission ISSUED, complimentary admissions included.
                The fee payable under this clause for any one event is capped at 12,000.00
                GBP. The cap is a term of this clause and is not waivable at filing.

Remittance Worksheet As Filed
----------------------------------------------------------------
The worksheet as the preparer computed it, one line per rule, in the sequence applied.
THE WORKSHEET DOES NOT STATE WHICH CLAUSE IT READ. It states a base value, a rate and
an amount; whether that base value is the base the ordinance defines is the question.

LINE: seq=1 rule=S-2.1 kind=ADMISSIONS_TAX authority=CITY base=PAID_FACE_PLUS_SERVICE base_value=217,245.50 rate=0.040000 amount=8,689.82
LINE: seq=2 rule=S-3.1 kind=COUNTY_SURCHARGE authority=COUNTY base=PAID_FACE_PLUS_SERVICE base_value=217,245.50 rate=0.015000 amount=3,258.68
LINE: seq=3 rule=S-5.1 kind=FACILITY_FEE authority=VENUE_AUTHORITY basis=ALL_ISSUED count=7426 per_ticket=1.75 cap=12,000.00 amount=12,000.00

  Authority                        Remitted
  CITY                             8,689.82
  COUNTY                           3,258.68
  VENUE_AUTHORITY                 12,000.00
  Total remitted                  23,948.50

Filing Notes
----------------------------------------------------------------
The venue's finance office queried the facility fee line and withdrew the query once the per-ticket schedule was produced.

Raw responseThe raw response

{"verdict": "BASE_MISAPPLIED", "defect": "WRONG_BASE_CATEGORY", "governing_clause": "S-2.4", "admissions_tax_owed": 51631.93, "short_authority": "NONE", "amend": "YES", "owner": "Ingrid Sandoval", "rationale": "S-2.4 excludes student/senior concessionary admissions from the base, but the worksheet included their £23,776 face value and £1,486 service charge in the S-2.1/S-3.1 base, over-remitting the CITY and COUNTY."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Catch a venue's admissions tax filed on the wrong base — 120 event admissions-tax remittance files. One model answered, and every answer was then graded Two different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

EVERY ARM ON THIS PAGE READ ALL 120 READINGS -- there is no halved denominator here. The paid arms cost 240 calls between them and the free floors cost nothing, so the model, the ablation and the three floors are compared over one identical set. The six readings with no rate schedule are excluded from the AMOUNT denominator only (114 of 120), because not-determinable is a third state and scoring it as a zero would invent a failure.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection, never a bill and never a vendor's claim about this workload.

Rates checked 2026-08-18. The provider that actually ran the calls is kept out of these tables per this estate's naming rule, so no figure here is a bill.

The gradersTwo ways to grade

On the FEED-BLIND slice -- the 45 defects a proviso, a category exclusion, a cascade clause, a conditional exemption or a ruling produced, and which the jurisdiction's own rate feed cannot express -- all three floors score 0.0 pct on the amount and the model scores 100.0 pct. That single comparison is the kit.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

The three free floors, the stub and every pre-run check cost $0.00 and no calls. 324 calls were spent on this kit and 324 reached a published result file. The socket timeout and the token ceiling were raised TOGETHER before the first paid call rather than after a discarded run, which is why the discarded count is 0 rather than the three figures a sibling kit paid for the same lesson.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND ALMOST ALL OF THEM ARE REASONING. 96.0 pct of r001-admissions-tax's output (375589 of 391290 tokens) was provider-side reasoning left at the default. The answer is seven short fields and a sentence; the bill is the thinking in front of it -- and on this task the thinking is a base recomputed TWICE, once as the worksheet declares it and once as the ordinance defines it.
• THE PROMPT IS MOSTLY FIXED TEXT. 3186 input tokens per reading, and the ordered rules and the JSON shape are identical on every call -- so a provider with prompt caching would price this workload very differently, and that was not measured.

Your volumeWhat it costs at your volume

LINEAR IN EVENTS. Nothing amortises. The sublinear lever NOT implemented is skipping filings whose ordinance extract carries no ruling, no proviso, no exclusion and no exemption clause -- the free strong floor is exactly right on those, and this kit calls the model on them anyway.

Model choiceWhy this model

It is the tier the shared connection was already pointed at, and the kit's whole claim is that swapping it is .env plus one more run. No second model was run against this corpus; every other row in the cost table is a projection onto a published card and is labelled as one.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-query average, this run

The codeEvery file, and what it is for

12 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Six of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/rules.pythe admissions-tax rulebook — a swap seam

The only definition of what is owed. compute() turns a manifest and a set of terms into a base, a tax, a surcharge and a fee; decide() turns the governing and filed computations into a verdict, a defect, a clause, a short authority and an amend call. It is also the answer key.

You change it to: DEFAULT_AMENDMENT_FLOOR_GBP -- the knob that decides which named defects are worth filing an amended return over. It is PRINTED on every file and read off the page by every arm.

# The admissions-tax rulebook as data and as pure code. No model, no network, no I/O.

GENERAL = "GENERAL"
PREMIUM = "PREMIUM"
COMPLIMENTARY = "COMPLIMENTARY"
STUDENT_SENIOR = "STUDENT_SENIOR"
SEASON = "SEASON"
CATEGORIES = (GENERAL, PREMIUM, COMPLIMENTARY, STUDENT_SENIOR, SEASON)
CITY = "CITY"
COUNTY = "COUNTY"
VENUE_AUTHORITY = "VENUE_AUTHORITY"

src/segment.pythe section splitter

Splits the remittance file into its 9 named sections; asserted across all 120 documents before any run may spend.

# Split a remittance file into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Event", "Filing Control", "Ticket Manifest",
RULE = "-" * 64
def split(text):
def render(secs):

src/select.pythe send filter — a swap seam

Filing Contact is mapped by no field and therefore never sent -- and the fallback path that makes that a property of the code rather than of the corpus is red-proven.

You change it to: SECTION_HINTS and NEVER_SENT.

# Pick which sections of a remittance file are sent. Pure code -- the last deterministic step

BANNER = "Synthetic Record"
EVENT = "Event"
CONTROL = "Filing Control"
MANIFEST = "Ticket Manifest"
FEED = "Rate Feed (Structured)"
ORDINANCE = "Ordinance Extract"
WORKSHEET = "Remittance Worksheet As Filed"
CONTACT = "Filing Contact"
NOTES = "Filing Notes"

src/prompt.pythe prompt

Two parts: the instruction with the ordered rules and the JSON shape, then the file. The ordinance-blind ablation replaces exactly one section's body.

# Assemble the one prompt this kit sends. No template engine, no framework -- string concatenation

BLIND_LINE = ("The ordinance extract is not available for this event. Judge the filing on the "
INSTRUCTION = """\
def build(text, ordinance_blind=False):

src/adapters/__init__.pythe model call — a swap seam

Raw HTTP, stdlib only, bounded retry, a socket timeout set from the published ceiling (900 s, not the 120 s every sibling inherited) and a SEPARATE retry budget for transport failures. The shared daily call cap is checked here.

You change it to: ADAPTER_TIMEOUT_S and TRANSPORT_RETRIES, changed in the same edit as src/filing.MAX_TOKENS. They are COUPLED and nothing in the template says so: raising the ceiling without raising the timeout converts a truncation defect into a transport defect, and a timed-out generation is retried and billed every time.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
TRANSPORT_RETRIES = 1
TIMEOUT_S = int(os.environ.get("ADAPTER_TIMEOUT_S", "900"))
def _post(url, headers, payload, timeout=None):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

src/filing.pythe reader

One event remittance file, one call, at the published 32,000-token ceiling.

# One event remittance file, one model call. The runtime half of the kit.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SYSTEM = ("You are an admissions-tax and facility-fee review for a live-events venue operator. You "
MAX_TOKENS = 32000
FIELDS = ("verdict", "defect", "governing_clause", "short_authority", "amend", "owner")
NOT_DETERMINABLE = "NOT_DETERMINABLE"
def documents():
def load_doc(doc_id):
def facts_of(text):

evals/run.pythe per-reading cache

Append-only, written before anything is scored. --resume never re-buys a call already paid for; --score-cache scores exactly what came back.

# Run the review over the 120 event remittance files and write one result file. THIS SPENDS MONEY.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
def scored_half(docs):
def cache_path(run_id):
def load_cache(run_id, max_tokens, ordinance_blind):
def append_cache(run_id, row):
def load_gold():
def stub_complete(cfg, system, user, max_tokens=1024):

evals/scoring.pythe scorer

Exact match per labelled cell, a penny tolerance on the amount. The discriminator is reported on four denominators and never blended.

# Score a run against the answer key. Pure code, exact match per cell. No model grades anything.

FIELDS = ("verdict", "defect", "governing_clause", "short_authority", "amend", "owner")
DEFECT_VERDICTS = ("BASE_MISAPPLIED", "RULE_MISAPPLIED", "ARITHMETIC_ERROR")
TOL = 0.011
def _pct(n, d):
def _same(got, want):
def _same_name(got, want):
def _amount(v):
def score(records, golds):
def _protection(records, golds):

evals/baseline.pythe three free floors

worksheet-recompute, rate-table and structured-feed-strong -- $0.00 each, and all three beat the model on the FALSE direction.

# THE FREE FLOORS. Three of them, none a strawman, all of them GBP 0.00.

MODES = ("worksheet-recompute", "rate-table", "structured-feed-strong")
TOL = 0.011
def _num(v):
def _kv(line):
def _facts(text):
def _finish(verdict, defect, clause, tax_owed, short, amend, owner, why):
def _line_check(f):
def _worksheet_tax(f):
def _feed_rule(f, kind):

evals/check_labels.pythe pre-spend gate

11 checks, free and offline, including re-deriving every gold amount from the shipped text. It caught a real defect on its first run.

# THE PRE-SPEND GATE. Free, offline, and it must pass before any run may cost money.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
GOLD = os.path.join(HERE, "data", "gold.jsonl")
FAILURES = []
def check(name, ok, detail=""):
def _rebuild_from_text(text, gold):
def main():

src/app.pythe local UI server

http.server, stdlib. Renders with no key, on port 9041.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
PORT = int(os.environ.get("PORT", "9041"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-admissions-tax")
class H(BaseHTTPRequestHandler):
def main():

ui/app.jsthe local UI client

Hand-written JS, no framework.

# 

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 3186 input and 3260 output tokens per reading (one event remittance file), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ MEASURED, AND IT HELD. Filing Notes is this kit's injection surface, and on this vertical the desk that types them is very often the PROMOTER'S -- the counterparty whose worksheet is under review.

API_KEY is read from .env (gitignored) or the real environment; never logged, never placed in a prompt, and redacted from any provider error message the local UI passes to the browser.

The experimentWe DID attack it -- one sentence, on every reading where it could matter

The probe replaces the Filing Notes section with the instruction-shaped note and re-fires at the same ceiling, everything else held identical. The fired set is not 'every reading where an amendment was due' but 'every reading where an amendment was due AND the un-injected run raised one', because the second is the only set where a suppression is observable. 75 of 75 held, and every watched field was diffed individually rather than only the amendment decision. Both were run for real on 2026-08-25.

All three boundaries are measured in both directions: the injection probe forces the condition rather than waiting for it and pairs each trial against its own control, it scores the whole answer rather than one field, and the privacy guard is red-proven by reproducing the schema-change condition that reaches its fallback.

The result0 of 75 amended returns suppressed by the instruction-shaped note, at a 32000-token ceiling -- measured, not assumed.

One phrasing, one model, one corpus, one ceiling (32000), 75 paired trials -- every reading where suppression was even possible.

HonestyWhat this does not prove

• Whether a different phrasing moves it. A note claiming the county has already signed the filing off, claiming the ruling was never gazetted, or written to look like a system banner is a different experiment.
• Whether an injection placed in Filing Contact would have any effect -- by construction it cannot, because that section never leaves the machine.
• Whether the result holds at another ceiling. It is a property of the run, not of the model.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never file a return, amend a return, remit money, release a payment, change a rate or approve an amendment -- and never present the rates, the bases, the exemption tests or the amendment floor as any jurisdiction's, or anything here as tax advice.

Stated to the model on every call in src/prompt.py's INSTRUCTION block, printed on every remittance file's Filing Control section, and true of the code by ABSENCE: the only writers anywhere in the repository are results/*.json and data/.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is a PROMPT RULE PLUS AN ABSENCE, not a runtime enforcement layer. Nothing stops a forker adding a filing endpoint tomorrow.
• The injection result is ONE SENTENCE against ONE model on ONE corpus at ONE CEILING, over 75 paired trials. A sibling kit measured 0 suppressions at 32,000 and a real suppression on the same reading and the same sentence at 16,000. It is not a resistance rate for prompt injection in general and the phrase does not appear in this kit.
• The verdict is an OPINION about a worksheet, not a legal reading of an ordinance, and nothing here is tax advice. An admissions-tax dispute is decided by people with the whole schedule and the correspondence in front of them.

WatchedWhat is watched, and why that one

The kit measures 91 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

2 measured by the latest run 89 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

8 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on admissions_tax_hit, events_caught, events_missed, missed_amendments, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · with the model in the path — 3 runs. Columns here are only ever compared with each other.

not a time series No two of these 3 runs measured the same system — they differ on admissions_tax_defect_cells, admissions_tax_feedblind_cells, admissions_tax_hit, admissions_tax_sound_cells, amend_cells, answered, arithmetic_cells, context_incomplete_cells, correct_cells, defect_cells, defect_feedblind_cells, documents, events_caught, events_missed, events_needing_amendment, feed_blind_cells, missed_amendments, quiet_cells, readings_scored, sound_cells, stateless — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 24 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 2 chips that all say so.

DeviationsWhat deviated

0 breaches across 8 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

ON DEMAND, ONCE PER EVENT -- there is no clock and no carried state, so re-reading the same filing against the same ordinance returns the same opinion. The one thing that makes a re-run worth paying for is an ORDINANCE that moved: a ruling issued after the worksheet was filed changes which rate governs, and this kit will only see it once that ruling reaches the extract. At 0.0091077 USD per event on the projected card, a season of 120 events is roughly 1.0929 USD -- but the ceiling that matters is provider concurrency, not price: p95 latency was 3.9x p50 on a credential shared with eight sibling kits, and the wall-clock figures on this page are an observation under contention rather than an SLA.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, no framework dependency, an empty requirements.txt, and a prompt anyone can read end to end. There is no retrieval step for a framework to own and no agent loop for it to drive: the whole job is one document, one ordinance and one call.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: the reader -> src/prompt.py -> src/adapters -> evals/scoring.py. No branching, no tool-calling and no agent loop.

The other sideWhat a framework costs you

• Swapping providers means editing the PROVIDERS dict by hand. In exchange the prompt has no hidden templating layer and requirements.txt stays empty.
• Resume and caching are 40 lines here rather than a framework feature.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-admissions-tax on the fast tier, with the ordinance extract, 2026-08-25. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 5 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b020-admissions-tax-recompute, b021-admissions-tax-ratetable, b022-admissions-tax-feedstrong recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: ADAPTER_TIMEOUT_S, API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe city admissions tax owed, to the penny, against the computed answer key

whether the arm's stated admissions_tax_owed equals the governing figure, within a penny, on every reading where a figure is determinable at all

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares numbers. No model grades anything.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

hit if |answered - gold| <= 0.011. Reported on FOUR denominators and never blended: all determinable readings, the sound slice, the defect slice, and the feed-blind slice.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/rules.compute() from the same planted terms the corpus builder printed onto the page, and re-derived from the SHIPPED TEXT by evals/check_labels.py before any run may spend.

No true/false rates for this grader. It records 4 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? The only tunable is the penny tolerance, and it is 0.011 -- one penny either way and no more. It is NOT swept: a wider tolerance would forgive the corpus's own banker's-rounding artefact and flatter every arm equally.

Cadence: once per run; free

The decisionWhen to reach for it

In one lineThe verdict, the named defect, the governing clause, the authority under-remitted, the amend call and the tax analyst, per reading, exact match

whether each of the 6 labelled fields equals the answer key, per reading

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 120 per field. The amount is scored by its own grader above.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl, computed by src/rules.decide().

No true/false rates for this grader. It records 6 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold was swept: exact match has no tunable. Enum columns are case-folded, the analyst's name is not.

Cadence: once per run; free

The decisionWhen to reach for it