The business caseThe problem this solves

A service record already says how to communicate with the person it is about — in their own words, in a note somebody took, in a letter that came back, or in the register row recording what was arranged last time. Nobody reads all five of those places for all five accommodations on every booking, so the arrangement that was made once is quietly not made again, and the request written in a sentence a checklist does not carry is not made at all. Neither failure produces a complaint that reaches anybody: what surfaces is somebody who did not attend, or who attended and got less out of it than everybody assumed. The clerk's own pass over one record's five sections — the record facts, what the person told us, the intake and contact notes, the correspondence history and the service history — against five accommodations, one at a time. It replaces the READING of that pass and none of the deciding: AP-2026 is applied in code either way.

The inputThe actual service records

The corpus is 60 service records, 0.05 MB (txt 60). The cases are the ones that make this decision hard, and none of them can be taken from anywhere real. A person who declined an interpreter last time and has since asked for one (declined_then_stated, 3 records) is the case that proves the rule ORDER rather than any rule: AP-1 over AP-3 is what makes somebody able to change their mind, and both arms get all 15 of its judgements right because the order is applied in code on both. A standing notice about the interpreter booking line, enclosed with every letter that month (phrase_false_positive, 6 records, 30 judgements), is a fact about a building and flagging it sends a clerk to arrange something nobody asked for. An interpreter asked for with the language never written down (language_unstated, 4 records) makes NOT STATED the right answer for the language field, because an interpreter booked in a guessed language is worse than an unfilled booking — it looks handled. And the whole middle of the corpus is the UNCLEAR-ASK cases — a letter that came back undelivered, a relative who did the talking, a call that moved channel — where the record points at a need and does not establish it.

SERVICE RECORD AR-0001 -- Booking confirmed by text

RECORD FACTS
  Record opened                 2026-03-17
  Service                       outpatient booking at Tarnsyde
  Contact channel used          telephone
  Register read as at           2026-08-31

WHAT THE PERSON HAS TOLD US

  They asked whether an interpreter in Arabic could be booked for the same time slot.
  They asked whether they could have a double slot.

INTAKE AND CONTACT NOTES

They said the date offered was fine and did not want to change it. They asked whether there is parking at Tarnsyde and were told where to go.

The record was opened when the referral arrived and has been added to twice since. They asked what time they should arrive and where to wait.

CORRESPONDENCE HISTORY

  2026-03-31  text reminder sent -- confirmed by reply

SERVICE HISTORY -- what was arranged at earlier contacts

  No accommodation has been arranged at an earlier contact.

The outcomeWhat a good result looks like

Five determinations a person confirms instead of a record they re-read: for each accommodation, INDICATED, NOT-INDICATED or UNCLEAR-ASK, with the sentence it came from copied verbatim out of the record and located at its offsets. Where the answer is UNCLEAR-ASK what it produces is a QUESTION TO PUT TO SOMEBODY with the text that raised it, so the conversation has a starting point.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Records written in the phrases a standing checklist already carries — plain requests, plain silences, and a register that is kept — the free rules floor alone
  Six results are a TIE on this corpus: stated_plain 25/25 both, nothing_indicated 30/30 both, declined_then_stated 15/15 both, third_party_remark 22/25 both, ambiguous_channel 14/15 both, and the whole clean control group 11/11 both. Every register-derived answer is free: AP-3 6/6 and AP-4 9/9 on both arms, because they come from data/records.json and no reading is involved. Zero fabricated citations and zero invented languages on both arms.
• A backlog where the request is in the person's own words rather than the checklist's — the paid call, read per class
  stated_in_own_words is 6 records and 30 judgements: the model takes 29 of 30 where the floor's phrase lists were written against the listed phrasings. Across the corpus the model misses 1 of 38 stated requests where the floor misses 10, and over-flags 3 of 235 silent judgements where the floor flags 18 — the two directions that cost a person their appointment or a clerk a wasted booking. hard records go 11/49 to 39/49.

And where nothing here is good enough:

• Deciding whether the call is worth its money on THIS corpus — neither arm as published — write the stem floor first
  A regex reader built from the kit's own generator tables reaches 300/300 verdicts and 60/60 records for $0.00 (see baseline_note). Against that arm the paid call LOSES: 290 against 300 judgements, 50 against 60 records, 21 against 27 on UNCLEAR-ASK. The published margin measures the shipped floor's weakness and the corpus's regularity, not the difficulty of the task.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-31. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-31 — r001-access-need. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — ⚠︎ PARTIALLY EVIDENCED, AND THE REST IS NOT MEASURED. What is on disk and checkable: the 60 records, the register, AP-2026 in both renderings, the answer key with its character offsets, the free floor's recorded result, the paid run's recorded result and its full reply cache. Standard library only — no requirements to install. The free half (build_corpus, check_labels, the b000 floor, the --stub pass, the board on 127.0.0.1:9235) needs no key and no network. ⚑ THE CORPUS REBUILD WAS VERIFIED REPRODUCIBLE ON 2026-08-31 rather than claimed: tools/build_corpus.py was re-run from a clean copy under four PYTHONHASHSEED values (1, 97531, 0, 424242) and the corpus, gold.jsonl, records.json, policy.json and corpus-stats.json came back BYTE-IDENTICAL to the committed bytes every time. What is NOT measured is a cold clone on another machine, another Python, or another operating system.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one service record end to end — one record, one call, five determinations, including provider-side reasoning tokens, on a shared credential. The policy engine and the citation locator add nothing measurable: both are pure Python over the reply. ⚠︎ NOT AN SLA AND NOT A SPREAD: these are wall-clock observations from ONE run and the band is wide — fastest call 4,881 ms, slowest 105,097 ms. 60 records ran in 353.2 wall seconds at 5 workers against 1633.6 seconds of summed call latency, which is the arithmetic that proves the concurrency rather than the README asserting it. The free rules floor answers and scores all 60 with no network at all and its run record carries wall_seconds 0.0.

Current processWhat it replaces

The clerk's own pass over one record's five sections — the record facts, what the person told us, the intake and contact notes, the correspondence history and the service history — against five accommodations, one at a time. It replaces the READING of that pass and none of the deciding: AP-2026 is applied in code either way.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one access-need judgements (5 per record) is

No train/test split, because nothing is trained or tuned, and no segmentation, because one record goes whole into one call (the sizes here are the records' own bytes; min 756, max 1,153). The unit every rate stands on is ONE JUDGEMENT — one accommodation on one record — and there are 300 of them across 60 records; every arm is scored over all 300 whether or not it answered. 11 records are clean and 49 are marked hard, across 13 named cases: nothing_indicated 6, phrase_false_positive 6, stated_in_own_words 6, mixed_multi 5, prior_register 5, returned_correspondence 5, stated_plain 5, third_party_remark 5, declined_before 4, language_unstated 4, ambiguous_channel 3, declined_then_stated 3, prior_unconfirmed 3. ⚠︎ THE CLEAN ELEVEN ARE A CONTROL GROUP, NOT A HEADLINE: five are a request in the person's own plain words and six indicate nothing at all, and BOTH ARMS SCORE 11 OF 11 ON THEM. A report that picks the first alphabetical all-correct key off this kit's scores block lands on clean_all_correct and prints an 11-vs-11 tie; the number it is reading is the control, not the measurement.

SetupWhat the setup figure measured

THERE IS NO INDEX IN THE RETRIEVAL SENSE and nothing to retrieve from: one record goes whole into one prompt at a mean of 4,038.8 input tokens. What this kit calls the index is data/records.json — the SERVICE REGISTER, one row per record, the authority on what was arranged at earlier contacts — and it is read in memory by src/policy.py and never sent as a retrieval step. The whole free half is the fraction of a second the floor run records: results/eval-b000-access-need-rules.json carries wall_seconds 0.0 for answering and scoring all 300 judgements with no key and no network. ⚠︎ 0.0 IS THE RECORDED VALUE, NOT A ROUNDED MEASUREMENT: the harness writes wall_seconds only for the model arm's calls and the free arm makes none.

LicenceLicence

MIT — the kit's own licence, and the MIT text is in LICENSE-PUBLIC at the repository root. The corpus, the register, AP-2026, the generator, the harness and both recorded runs are all generated in-process from a fixed seed, so there is no third-party data in this kit to licence and nothing here is fetched from anywhere. ⚠︎ READ THE GRANT FROM LICENSE-PUBLIC, NOT FROM THE ROOT LICENSE FILE — they are two different documents, only the first is the MIT text, and the root LICENSE governs the repository itself and grants a reader nothing. The kit is MIT; the repository that carries it is not. Both files were opened and read on 2026-08-31 to write this sentence.

Bring your ownBring your own service records

Drop your own records as .txt into data/corpus/, add one row per record to data/records.json for the register, and write one gold row per record in data/gold.jsonl with a citation_span for every judgement whose verdict is not NOT-INDICATED. Then run evals/check_labels.py before anything else: it re-derives every verdict from the reading and the register under AP-2026's order, checks every span reproduces its sentence exactly once, and refuses a key that does not hold together. Nothing needs rebuilding and no index needs to exist.

⚠︎ And what stops being true when you do: ⚠︎ THE WHOLE RECORD GOES TO A MODEL PROVIDER VERBATIM — the notes, the correspondence outcomes, the register, everything a colleague wrote about a contact. That is a disclosure decision for whoever holds the record and it is not one a kit can make on their behalf. Run it against an extract you are permitted to disclose, or against a local model. And a person whose record is read this way should be told that it is. ⚑ THE KIT ALSO REFUSES A SECOND BOUNDARY BY CONSTRUCTION: it makes no clinical inference, records no condition and states no diagnosis. Every answer is a logistics answer — what to book, what to send, how long to hold the slot, who else may be in the room. A reply offering a reason would be inventing one, and the corpus contains none to copy.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 17,655 characters assembled, 8,340 are instructions — 47% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The whole prompt as sent for AR-0001, the first record of run r001-access-need — all six parts in order, rebuilt from src/prompt.py on 2026-08-31 and checked against the run record's own prompt_parts: 2,142 / 5,692 / 2,025 / 4,173 / 967 / 2,656 characters, 17,655 total, identical part for part.

Full promptThe prompt in full

You are reading one service record and answering a single operational question about it: which
communication and accessibility accommodations does this record indicate are needed for the next
contact, and where does it say so? Your output is the list a booking clerk confirms -- so that the
clerk confirms five answers instead of re-reading the record.

THIS IS A SCHEDULING AND COMMUNICATION QUESTION AND NOTHING ELSE. Do not infer, state or imply
anything about a person's health, diagnosis, capability or circumstances. You are answering what to
book, what to send, how long to hold the appointment, and who else may be in the room -- from what
the record already says. The records carry no clinical information; a reply that offered one would
be inventing it.

How to read the record:

- WHAT THE PERSON HAS TOLD US, the INTAKE AND CONTACT NOTES, the CORRESPONDENCE HISTORY and the
  SERVICE HISTORY are all part of the record and all of them count. A record often indicates an
  accommodation only in its service history while its notes say nothing at all.
- Distinguish WHO SAID IT. A preference the person stated about themselves is the strongest thing
  in the record. A remark by a relative, a note about how a call went, or a letter that came back
  is evidence that there is something to ask about -- it is never evidence of what the person
  wants. That distinction is the whole of AP-6 and most of what this record set is built to test.
- A phrase does not indicate an accommodation just by appearing. A standing notice about the
  service's own facilities, a line printed on every letter, or a record of something being OFFERED
  AND DECLINED all contain the vocabulary and indicate nothing -- or the opposite.
- Answer all five accommodations, once each, in the order given. Most records indicate none or one
  of them; answering NOT-INDICATED on four rows is the normal shape of a reply here.
- Apply the access policy as written, INCLUDING THE ORDER ITS RULES ARE APPLIED IN.
- Give one confidence between 0 and 1 for the record's five answers taken together.

Reply with JSON and nothing else, in the shape given at the end.

THE ACCESS POLICY, as approved:

# AP-2026 — the access policy, as approved

*Read as at 2026-08-31. This policy is INVENTED for this kit. It is not any organisation's access
policy, and nothing in it is advice about how one should be written.*

## What this policy decides, and what it does not

It decides, for each of five accommodations, whether **this record indicates that accommodation is
needed for this contact**. That is a question about **scheduling and communication**: what to book,
what to send, how long to hold the slot, and who else may be in the room.

**It decides nothing about the person.** No rule below asks what is true of anybody, why they
might need something, or what their situation is. A record is read for what it says about
arrangements, and for nothing else.

**Three answers, and only three.**

| verdict | what it means |
|---|---|
| `UNCLEAR-ASK` | the record points at a need and does not establish it. **The action is to ask the person.** |
| `INDICATED` | the record establishes that this accommodation is needed for this contact |
| `NOT-INDICATED` | the record establishes that it is not, or says nothing at all |

`UNCLEAR-ASK` is listed first deliberately. It is the answer this policy exists to make available,
and it is the one a two-way yes/no check cannot express — such a check must round every hint to
either a booking nobody asked for or a silence nobody chose.

## The seven rules, in the order they are applied

**The first rule whose condition is met is the one recorded.** The order is the policy: the same
facts under a different order produce different answers, which is why the order is written down
rather than left to whoever applies it.

**AP-1 — What the person said about themselves governs. → INDICATED**
Where the record carries the person's own statement of a preference, that preference is the
answer. It outranks the register, it outranks what anybody else said about them, and it outranks
the absence of any earlier arrangement. A person who has just told the service what they need has
already answered the question this policy asks.

**AP-2 — What the person declined governs equally. → NOT-INDICATED**
Where the record carries the person's own decline, the accommodation is not indicated. A decline
is a statement of preference exactly as a request is, and a service that keeps arranging something
a person has said no to is not being accommodating.

**AP-3 — A standing decline on the register holds until the person says otherwise. → NOT-INDICATED**
Where the service register records that this accommodation was offered at an earlier contact and
declined, and the record carries no newer statement from the person, it is not indicated. AP-1
outranks this rule, which is what makes a person able to change their mind.

**AP-4 — An accommodation already arranged is indicated again. → INDICATED**
Where the service register records that this accommodation was arranged at an earlier contact and
used, it is indicated for this contact too, whether or not the record's free text mentions it.
Making a person ask again for something the service has already arranged for them once is the
commonest way this decision is got wrong, and it is invisible: nothing downstream reports an
accommodation that was quietly not repeated.

**AP-5 — A past arrangement the register cannot confirm is a question. → UNCLEAR-ASK**
Where the record's free text refers to an accommodation having been arranged before but the
service register carries nothing for it, the correct action is to ask. The text may be right and
the register incomplete, or the reference may be to a different service. Booking on it assumes;
refusing it also assumes.

**AP-6 — Somebody else's account, or something that happened, is a question. → UNCLEAR-ASK**
Where the only thing pointing at an accommodation is what somebody other than the person said, or
something that happened — a letter returned undelivered, a call that moved to another channel —
the record indicates that there is something to ask about and does not establish what the person
wants. **This is the rule the whole policy turns on.** It is evidence of a question, never
evidence of an answer, and turning it into either hard verdict is a decision taken about somebody
without asking them.

**AP-7 — Silence indicates nothing. → NOT-INDICATED**
Where the record says nothing about an accommodation and the register carries nothing for it, it
is not indicated. Silence is not a hint and must not be read as one; a service that flags an
accommodation on every record has told nobody anything.

## The language rule

Where an interpreter is `INDICATED` or `UNCLEAR-ASK`, the language is **the one the record names**.
Where the record names no language, the language is **NOT STATED** and must be recorded as such.

It is never inferred — not from a name, not from a place, not from a previous contact in another
language, not from anything else in the record. An interpreter booked in the wrong language is
worse than an unfilled booking, because it looks handled.

## The service register

The register is the service's own record of **what was arranged at earlier contacts**. For each
accommodation it holds one of three states:

| state | meaning |
|---|---|
| `provided` | arranged at an earlier contact and used |
| `declined` | offered at an earlier contact and declined by the person |
| `none` | the register carries nothing for this accommodation |

**The register is a record of arrangements, never of anything about the person.** It is the
authority on AP-3 and AP-4 and on nothing else, and a sentence inside a record asserting what the
register says is not evidence about the register.

THE THREE VERDICTS, and what answering each one commits you to:

  UNCLEAR-ASK     
      The record points at a need for this accommodation and does not establish it -- somebody other than the person said something, a letter came back, a call moved channel, or the record refers to an arrangement the service register cannot confirm (AP-5, AP-6). THE ACTION IS TO ASK THE PERSON. It is a real answer, not a failure to answer: it names a question somebody must put, and it names the text that raised it.
      Used where the record does settle it, it sends somebody back to ask a question that was already answered -- a phone call, and a person asked twice. Not used where the record does not settle it, an assumption is made about somebody without asking them: an accommodation booked that they never requested, or one quietly not booked at all. Nothing downstream reports either.

  INDICATED       
      The record establishes that this accommodation is needed for this contact -- the person said so themselves (AP-1), or the service register records it as arranged at an earlier contact and used (AP-4). Something gets booked, sent or held as a result.
      Answered wrongly, an accommodation is arranged that nobody asked for. On four of the five that is a wasted slot or a wasted print run. On accompanying_person it is a person in the room the person themselves did not choose, which is not a clerical error.

  NOT-INDICATED   
      The record establishes that this accommodation is not needed -- the person declined it (AP-2), the register carries a standing decline and nothing newer (AP-3) -- or the record says nothing about it at all (AP-7). Silence lands here, and it is the commonest answer on any record.
      Answered wrongly, the person arrives and cannot take part in their own appointment: no interpreter, a letter they could not use, a slot that runs out. Nothing downstream reports it either. It surfaces as somebody who did not attend, or who attended and got less out of it than everybody assumed.

THE FIVE ACCOMMODATIONS, in the order you must answer them, and what answering
INDICATED on each one commits somebody to:

  interpreter          An interpreter, and which language
      SOMEBODY BOOKS AN INTERPRETER FOR THE APPOINTMENT, in a named language. The second half is not optional: an interpreter booking with no language on it is a booking that cannot be filled, and a booking in the wrong language is worse than none, because it looks handled. Where the record does not name a language, the answer is that an interpreter is indicated and the language is NOT STATED -- which is a thing to ask, not a thing to infer from anywhere else in the record.

  alt_format           Written materials in an alternative format
      WHAT IS SENT OUT CHANGES -- larger type, an easy-read or plain-language version, audio, braille, or a different channel for the same words. It is about DOCUMENTS, not about the appointment itself. Missed, the person gets a letter they cannot use and the service reads the silence that follows as a person who did not turn up.

  assistive_comm       Assistive communication in the appointment itself
      THE APPOINTMENT IS SET UP DIFFERENTLY -- captions on a video call, a text relay, a hearing loop, time to type rather than speak, a communication board the person already uses. It is about THE CONVERSATION, not the paperwork, which is why it is a separate answer from alt_format: a person can need one, the other, both or neither.

  longer_appointment   A longer appointment
      THE SLOT IS BOOKED LONGER. It is the cheapest of the five to arrange and one of the easiest to get wrong in both directions: booked on a guess it takes a slot somebody else needed, and not booked where the record asked for it, the appointment runs out of time and the person is asked to come back.

  accompanying_person  An accompanying person
      SOMEBODY ELSE MAY BE IN THE ROOM -- a relative, a friend, a supporter, an advocate. Arranging it wrongly is not a neutral error in either direction: booked on somebody else's say-so it puts a person in the room the person themselves did not ask for, and refused where it was asked for it leaves them on their own. This is the accommodation where ASKING rather than assuming matters most, and it is why UNCLEAR-ASK exists.

THE READING VOCABULARY -- where your answer came from:

  stated         the person themselves said it, in their own words or through a preference recorded as theirs
  declined       the person themselves said they did not want it
  prior          the record refers to this accommodation having been arranged at an earlier contact
  third_party    somebody other than the person said something that points at it, or something happened -- a letter came back, a call was cut short -- that points at it, without the person having said anything
  none           the record says nothing about this accommodation

HOW TO QUOTE THE TEXT, and how it will be read.

For every accommodation you answer UNCLEAR-ASK or INDICATED, `citation` must be ONE SENTENCE OR ONE
HISTORY LINE COPIED VERBATIM out of the record -- the text that indicates it. It may come from any
part of the record, including a dated line in the correspondence or service history.

  - Copy it character for character. It is located in the record by searching for it, so a
    paraphrase, a shortened version, an ellipsis in the middle, or two sentences joined together
    will not be found at all and will score nothing. There is no partial credit for a quote the
    record does not contain.
  - Quote the sentence, not the section. What is returned is compared with that sentence by
    character overlap: it must cover at least 60 pct of the sentence, and at least 30 pct
    of what you return must be the sentence. Returning the whole record scores nothing.
  - The access policy is NOT part of the record. A rule is never the citation.
  - Where you answer NOT-INDICATED, `citation` is null -- including where the record says the
    person was offered this accommodation and declined it. Quoting something for an accommodation
    the record does not indicate is counted as a wrong answer, not as an empty one.

THE SERVICE RECORD, verbatim:

SERVICE RECORD AR-0001 -- Booking confirmed by text

RECORD FACTS
  Record opened                 2026-03-17
  Service                       outpatient booking at Tarnsyde
  Contact channel used          telephone
  Register read as at           2026-08-31

WHAT THE PERSON HAS TOLD US

  They asked whether an interpreter in Arabic could be booked for the same time slot.
  They asked whether they could have a double slot.

INTAKE AND CONTACT NOTES

They said the date offered was fine and did not want to change it. They asked whether there is parking at Tarnsyde and were told where to go.

The record was opened when the referral arrived and has been added to twice since. They asked what time they should arrive and where to wait.

CORRESPONDENCE HISTORY

  2026-03-31  text reminder sent -- confirmed by reply

SERVICE HISTORY -- what was arranged at earlier contacts

  No accommodation has been arranged at an earlier contact.


Reply with JSON and nothing else, exactly this shape:

{
 "accommodations": [
   {
    "kind": "interpreter" | "alt_format" | "assistive_comm" | "longer_appointment" | "accompanying_person",
    "verdict": "UNCLEAR-ASK" | "INDICATED" | "NOT-INDICATED",
    "reading": "stated" | "declined" | "prior" | "third_party" | "none",
    "citation": "<text>" or null,
    "language": "<text>" or null
   },
   ... one object per accommodation, all five, in this order: interpreter, alt_format, assistive_comm, longer_appointment, accompanying_person
 ],
 "confidence": <a number between 0 and 1>,
 "why": "<text>"
}

What each field means:
  kind         which accommodation this row is about. Answer all five, once each, in the order given
  verdict      what this record indicates about this accommodation. UNCLEAR-ASK where the record points at a need and does not establish it -- the action is to ask the person. INDICATED where the record establishes it is needed for this contact. NOT-INDICATED where the record establishes it is not, or says nothing at all
  reading      WHERE the answer came from, as its own answer. `stated` the person themselves said it. `declined` the person themselves said they did not want it. `prior` the record refers to it having been arranged at an earlier contact. `third_party` somebody other than the person said something, or something happened to a letter or a call, that points at it. `none` the record says nothing about it. This is the reading, and it is the only field re-used by the policy engine
  citation     ONE SENTENCE OR ONE HISTORY LINE COPIED VERBATIM from the record -- the text that indicates this accommodation. Copy it exactly, character for character; do not paraphrase, do not shorten with an ellipsis, do not join two sentences. NULL where the verdict is NOT-INDICATED, including where the record records that the person declined it: quoting something for an accommodation the record does not indicate is counted as a wrong answer, not as an empty one
  language     INTERPRETER ROW ONLY, and null on the other four. The language the RECORD NAMES, spelled as the record spells it. NULL where an interpreter is indicated and the record names no language -- that is the answer, and it is a thing to ask. Never infer a language from a name, a place, an earlier contact or anything else
  confidence   your own number between 0 and 1 for this record's five answers taken together. It is published as evidence and never used to change an answer
  why          one sentence naming what decided the accommodations you did not answer NOT-INDICATED

Five accommodation objects for one record. No more, no fewer.

Raw responseThe raw response

{
 "accommodations": [
   {
    "kind": "interpreter",
    "verdict": "INDICATED",
    "reading": "stated",
    "citation": "They asked whether an interpreter in Arabic could be booked for the same time slot.",
    "language": "Arabic"
   },
   {
    "kind": "alt_format",
    "verdict": "NOT-INDICATED",
    "reading": "none",
    "citation": null,
    "language": null
   },
   {
    "kind": "assistive_comm",
    "verdict": "NOT-INDICATED",
    "reading": "none",
    "citation": null,
    "language": null
   },
   {
    "kind": "longer_appointment",
    "verdict": "INDICATED",
    "reading": "stated",
    "citation": "They asked whether they could have a double slot.",
    "language": null
   },
   {
    "kind": "accompanying_person",
    "verdict": "NOT-INDICATED",
    "reading": "none",
    "citation": null,
    "language": null
   }
 ],
 "confidence": 0.97,
 "why": "The person's own statements requesting an Arabic interpreter and a double slot indicate interpreter and longer_appointment, while the record says nothing about the other three accommodations."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

What a record indicates about how to communicate, and when to ask instead of assume — 60 service records. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

No model grades anything. Every metric is exact string match against data/gold.jsonl on the verdict and the language, plus offset arithmetic on the citation: the quote is located in the normalised record and credited iff it covers at least 60 pct of the labelled sentence and at least 30 pct of what was returned is that sentence. Not found is zero, counted separately, with no near-miss credit — which is what stops a fluent wrong answer scoring like a right one. Both thresholds are arguments, so any committed run re-scores at a different pair with no provider in the loop.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Run it twiceThe same set, run again

⚑ RAW AND RECHECKED ARE IDENTICAL ON BOTH ARMS AND THAT IS A REAL FINDING, NOT A NULL RESULT. recheck_overrides is 0: the policy engine re-applied in code overrode not one verdict on either the model or the floor, and no record's five verdicts differ between the two columns. It means EVERY ERROR IN THIS RUN IS A READING ERROR AND NOT A RULE ERROR — both arms applied AP-2026's precedence correctly to whatever they had read. So the pure-code station bought nothing here. It would have earned its place on an arm that read correctly and then applied the rules wrongly, and this model did not do that once. ⚠︎ ONE RUN, ONE DAY, ONE TARIFF WINDOW, ONE MODEL: this is not a repeat study and no variance figure exists. ⚠︎ AND HALF OF THAT CLAIM WAS NOT MEASURED UNTIL 2026-08-31. evals/run.py's --floor branch wrote the floor's own determination into the rechecked slot and never called src/recheck.py, so 'the engine overrode nothing on the floor' was a statement about a column that had never been through the engine. The branch now calls RC.recheck with the same arguments the paid path uses; the arm was re-run at $0.00 with zero calls and API_KEY forced empty, and NOT ONE GRADED NUMBER MOVED — recheck_overrides is still 0 on the floor, so the claim is now measured rather than assumed. The one thing that changed is unscored: the floor's rechecked column now carries citation spans and all 58 quotes it returned are located in the record, a field that read null before because nobody had looked. r001's result file, its cache and its $0.148462 are byte-identical across the repair.

verdict accuracy over the 300 judgements, and record-all-correct over the 60 records —

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate — a projection onto a rate card, not a bill anyone paid. The token counts come from the provider's own usage block on each call, and 90.8 pct of the output tokens they report are provider-side reasoning the kit did not ask for.

Rates checked 2026-08-27. The provider that actually ran every call here is kept off this page per the series rule — a rate card is a naming. Its real spend ($0.148462), its own dated card (2026-08-23), its peak/off-peak split and its cached-input tier are all recorded per call in the run file. One further published card is withheld for a different reason: it is not a US rate card, so the figure would not be actionable where a reader buys.

Grading unitWhat the grading figure prices

Scoring is pure Python over committed JSON: 300 judgements in well under a second, $0.00, repeatable against any result set. The rechecked column re-derives from the same replies for nothing. There is no LLM judge anywhere in this kit's grading path and no place to put one.

The gradersOne way to grade, and why it is the only one

THREE FREE ARMS EXIST FOR THIS KIT AND ONLY TWO ARE ON THE BOARD. ALWAYS-NOT-INDICATED, 78.3 pct. Answering NOT-INDICATED to all 300 judgements scores 235 of them, because that is the class distribution — and it gets every INDICATED and every UNCLEAR-ASK wrong, which is every answer worth having. It is on the board to make the aggregate unreadable on purpose. THE SHIPPED RULES FLOOR (b000-access-need-rules), 262/300 verdicts, 87.3 pct. Two groups of hand-written phrase lists — an INDICATE group per accommodation and a HINT group lifted from AP-6's own words — through the same policy engine. It is a real arm: it takes stated_plain 25/25, nothing_indicated 30/30, declined_then_stated 15/15 and third_party_remark 22/25, and it BEATS the paid call on AP-6 (17/22 against 16/22), on prior_register verdicts (25/25 against 24/25) and on returned_correspondence (24/25 against 23/25). Its failure is not that it is inaccurate; it is that it fails in BOTH DIRECTIONS AT ONCE — 18 of the 235 silent judgements flagged (7.7 pct, every one with a quoted sentence in support) and 10 of the 38 stated requests missed (26.3 pct) — and neither direction produces a complaint that reaches anybody. ⚑ THE BEST FREE ARM ON THIS CORPUS IS NOT THE ONE THAT SHIPS, AND IT SCORES PERFECTLY. Written and measured by this spec's author on 2026-08-31 in about fifteen minutes, using nothing that is not committed inside this kit: tools/build_corpus.py's SENT pools are 82 sentence templates across five accommodations and eight mechanisms, and its HISTORY dict is ten more for the service-history lines. Compile each into a regex over the same normalised text src/citation.py produces, map the pool to its mechanism's READING via the generator's own MECH table, and hand that reading plus the record's register row to src/policy.py — the same engine both shipped arms use. Result, scored by evals/scoring.py's own rules: 300/300 verdicts (100 pct), 300/300 citation credit, 60/60 records all-correct, 27/27 UNCLEAR-ASK, 38/38 INDICATED, 235/235 NOT-INDICATED, 9/9 interpreter languages, 0 invented languages, 0 stray quotes, 0 unlocatable — $0.00, no model, no network. ⚠︎ IT IS NOT A COMMITTED ARM: no eval-b001-access-need-* file exists in results/, and this spec publishes no score from it as a run. It is recorded because the alternative is letting a margin over a weak floor read as the value of a paid call.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-27. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingGrading is free; the bill is the 60 calls that answered.

The scorer, the label gate, the rules floor, the corpus generator and the rechecked column are pure code and cost $0.00 to run against any result set — there is no LLM judge anywhere in the grading path, and re-deriving costs from the committed cache (API_KEY= ... --resume --rescore) buys nothing. The figure above is the token count of the ONE paid run (60 calls, 242,327 in / 197,711 out) priced at the same projected card cost_per_query_usd uses. ⚠︎ THERE IS NO SECOND PAID RUN TO ADD: the injection arm (44 calls) was never fired, and the --stub pass is free. The bill the run actually incurred is $0.148462 and it covers ALL SIXTY CALLS — failures 0, so no call is missing from the totals.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND NINE TENTHS OF THEM ARE REASONING NOBODY ASKED FOR. 179,466 of the 197,711 output tokens (90.8 pct) are provider-side reasoning left at the tier's default and re-rolled per call, and the output side is 83.0 pct of the projected bill. The visible answer is five rows of JSON and a sentence of 'why'; the price is a budget the provider re-rolls per call. The largest reply drew 13,471 tokens, the smallest 486 — a 27-fold spread on a fixed-shape output.
• THE PROMPT IS FIXED AND MOSTLY CACHED. 14,032 of 17,655 characters are byte-identical on every call and are sent first, so 165,888 of 242,327 input tokens (68.5 pct) came back as prefix cache hits, charged at the cache tier. Only the record varies, at 967 characters on AR-0001. The input side is 17.0 pct of the projected bill.
• THE CORPUS IS THE MULTIPLIER AND NOTHING ELSE IS. One record is one call; sixty records is sixty calls at five workers. There is no retrieval, no second pass, no judge and no retry — failures 0, so nothing was re-bought.

Your volumeWhat it costs at your volume

Linear in records and flat in everything else. Ten times the records is ten times the calls, ten times the tokens and ten times the bill — 600 records at the measured $0.002474 each is about $1.48 on the provider's own off-peak card, or $7.14 projected onto the shared card. Nothing here has a per-corpus cost: no index is built, no embedding is stored, and the fixed 79.5 pct of the prompt is cached rather than re-priced. What does NOT scale linearly is the reasoning budget, which is re-rolled per call and varied 27-fold on this run.

Model choiceWhy this model

It is the tier the estate runs, so the figure compares with every sibling kit. The task is a bounded five-row structured read over a 967-character document with the rulebook supplied in the prompt — it is not a frontier problem, and the kit's own result says so: the model's contribution is the READING, one of five words per row, and pure code does the rest. ⚠︎ A CHEAPER OR HARDER MODEL WAS NEVER TRIED. One model id, one day.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the whole 60-record scored run (r001-access-need, the fast tier): 242,327 input tokens (165,888 of them prefix cache hits, 68.5 pct) and 197,711 output, of which 179,466 are provider-side reasoning. ⚠︎ usd_actually_paid is null ON THIS PAGE by the series rule, not because it is unknown: the real bill is $0.148462 across all 60 calls and it is recorded per call, with its tariff, in the run record.

The codeEvery file, and what it is for

12 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Eight of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/prompt.pyprompt assembly

Six parts in a fixed order — the system role with the clinical boundary stated first, data/policy.md verbatim (all seven rules and the order they are applied in), the three verdicts and what answering each one commits somebody to, the five accommodations and what booking each one means, the record VERBATIM, and the JSON schema built from data/fields.json. The first four parts are byte-identical on every call and are sent first: 14,032 of 17,655 characters, 79.5 pct of the prompt, which is why 68.5 pct of input tokens came back as cache hits. It declares VERDICTS and VERDICT_MEANINGS at module level so the page can read the vocabulary without importing the kit.

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY_MD = open(os.path.join(HERE, "data", "policy.md"), encoding="utf-8").read()
FIELDS = json.load(open(os.path.join(HERE, "data", "fields.json"), encoding="utf-8"))["fields"]
VERDICTS = ["UNCLEAR-ASK", "INDICATED", "NOT-INDICATED"]
VERDICT_MEANINGS = {
SYSTEM = """You are reading one service record and answering a single operational question about it: which
def _verdict_block():
def _kinds_block():
CITATION_RULE = """HOW TO QUOTE THE TEXT, and how it will be read.

evals/run.pythe record, unsummarised

The record's own .txt goes into the prompt as it is — all five sections, 967 characters on AR-0001, 954.6 bytes mean across the corpus. There is no pre-digest and no extracted 'needs' handed over as a summary, deliberately: a summary is where a relative's remark quietly becomes the person's own request before the model ever sees it, and then the measurement is of the summariser.

# Read all 60 records and write one result file. THE PAID ARM SPENDS MONEY.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
CORPUS = os.path.join(HERE, "data", "corpus")
KIT = "access-need"
RATE_CARDS = {
def tariff_at(card, ts):
def price(card, row):
def load_record(rid):
def load_gold():

src/classifier.pythe model call

One call per record, max_tokens 32,000, socket timeout 1,200 s, five workers. The reply is one JSON object: five accommodation rows in a fixed order, each with a verdict, a reading, a citation and (interpreter only) a language, plus the model's own confidence and a free-text why. All 60 parsed; failures 0; every finish_reason 'stop'.

# One service record in, five accommodation answers out. The only place a model is called.

MAX_TOKENS = 32000
THINKING = None
def parse_reply(text):
def normalise(obj):
def classify(cfg, text, rec, complete_fn=None, max_tokens=None):
VERDICTS = tuple(prompt.VERDICTS)

src/recheck.pywhat the model is trusted with — a swap seam

Exactly two fields survive: the READING (one of five words) and the LANGUAGE on the interpreter row. The verdict the model returned is DISCARDED — not corrected — and recomputed from that reading plus the record's register row. A field that is sometimes taken from the model and sometimes not is a field nobody can reason about afterwards. ⚑ ON THIS RUN THE STATION BOUGHT NOTHING: recheck_overrides is 0 on both arms, so every error in this run is a reading error and not a rule error. That is a real finding rather than a null result, and it is the honest limit of a pure-code station sitting behind a reading — it cannot fix a misread.

You change it to: which fields survive the recheck — currently the reading, and the language on the interpreter row. Widen it and the pure-code station starts trusting the reply's own verdict.

# THE PURE-CODE STATION, applied to a model's reply. What the model is trusted with, and nothing

TRUSTED = ("reading", "language")
def recheck(answer, rec, record_text):

src/policy.pythe policy engine

AP-2026's seven rules read out of data/policy.json in the published order: the person's own words (AP-1), their own decline (AP-2), the register's decline (AP-3), the register's prior arrangement (AP-4), a past arrangement the register cannot confirm (AP-5), somebody else's account or something that happened (AP-6), silence (AP-7). THE ORDER IS THE POLICY — AP-1 before AP-3 is what makes somebody able to change their mind. It never sees the record's text, by design: it takes a reading and a register state, which is what makes the split between 'what only a reader can supply' and 'what is a table lookup' checkable rather than asserted.

# THE PURE-CODE STATION. A reading plus the service register in, a verdict out, with the rule id

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY = json.load(open(os.path.join(HERE, "data", "policy.json"), encoding="utf-8"))
POLICY_ID = POLICY["policy_id"]
AS_OF = POLICY["as_of"]
RULES = POLICY["rules"]
RULE_ORDER = POLICY["rule_order"]
VERDICTS = tuple(POLICY["verdicts"])
LANGUAGE_RULE = POLICY["language_rule"]
ABSTAIN = "UNCLEAR-ASK"

data/records.jsonthe service register — a swap seam

One row per record: for each accommodation, provided / declined / none. It is the authority on what was arranged before and on nothing else, and a sentence inside a record asserting what the register says is not evidence about the register. AP-4 decides 9 of the 300 judgements from it and AP-3 another 6.

You change it to: what was arranged at earlier contacts, per record per accommodation.

# 

src/citation.pythe citation locator

Normalises the record once (NFC, 1:1 character folds, whitespace collapsed), locates the returned quote in it, and compares CHARACTER INTERVALS with the labelled span: coverage ≥ 0.60 and precision ≥ 0.30. Not found is zero with no near-miss credit, which is what stops a fluent wrong answer scoring like a right one. Both thresholds are arguments, so any committed run re-scores at a different pair with no provider in the loop.

# HOW A CITATION IS SCORED. Character offsets, two floors, and no partial credit for a quote the

FOLDS = {
def fold(s):
def norm(s):
def norm_quote(s):
def locate(norm_text, quote):
def span_of(norm_text, sentence):
COVERAGE_MIN = 0.60
PRECISION_MIN = 0.30
def credit(norm_text, gold_span, quote, coverage_min=COVERAGE_MIN, precision_min=PRECISION_MIN):

src/accommodations.pythe vocabulary, in one place — a swap seam

The five accommodations, their order, the five readings and the three register states — one copy, read by the prompt, the floor, the policy engine, the scorer and the UI. An unknown register state raises rather than being coerced to 'nothing recorded'.

You change it to: the five, their labels, their order, and what booking each one commits somebody to. One copy; five readers.

# The five accommodations this kit answers about, the reading behind each answer, and what

KINDS = {
ORDER = ["interpreter", "alt_format", "assistive_comm", "longer_appointment",
READINGS = {
READING_ORDER = ["stated", "declined", "prior", "third_party", "none"]
REGISTER_STATES = {
def labels():
def label_of(key):
def meaning(key):
def is_kind(key):

evals/baseline.pythe free rules floor — a swap seam

Two groups of phrase lists — an INDICATE group per accommodation and a HINT group lifted from AP-6's own words — read through the same sections and the same policy engine. It is a real arm, not a strawman: it answers all 300 judgements and scores 262 of them. ⚠︎ It is also NOT the strongest free arm available on this corpus; see the Eval lens.

You change it to: both groups of phrase lists, and therefore what 'free' scores here.

# THE FREE FLOOR. No key, no model, no network. A genuine attempt at the job.

MODES = ("rules",)
INDICATE = {
HINT = {
LANGUAGES = ["Somali", "Polish", "Portuguese", "Vietnamese", "Tigrinya", "Cantonese", "Mandarin",
HEADINGS = ["WHAT THE PERSON HAS TOLD US", "INTAKE AND CONTACT NOTES", "CORRESPONDENCE HISTORY",
def sections(text):
def readable(text):
def units(chunk):
def read(text):

evals/scoring.pythe grader

Exact match on the verdict, exact match on the language, offset-based credit on the citation, a per-class and per-rule split, and a failure taxonomy that assigns every miss to exactly ONE bucket in a fixed order. Pure code, free to re-run against any committed result set.

# Grade one arm against the answer key. Deterministic -- no model judges anything here.

VERDICTS = tuple(P.VERDICTS)
GRADED = ("verdict", "citation", "language")
TAXONOMY = ("no_answer", "assumed_not_indicated", "assumed_indicated", "missed_accommodation",
def _pct(n, d):
def _s(v):
def lang_key(s):
def rows_of(answer):
def _bucket(g, m, cit):
def score(dets, golds, texts, coverage_min=CIT.COVERAGE_MIN, precision_min=CIT.PRECISION_MIN):

evals/check_labels.pythe label gate

Eight checks that re-derive the key from data/policy.json, data/records.json and the corpus without importing a single src/ module — including that policy.md and policy.json state the same seven rules in the same order, that every citation span reproduces its sentence exactly once, and that the corpus carries no clinical vocabulary. Exits 0 on the shipped corpus.

# Grade the answer key itself. Free, offline, and it imports NOTHING from src/.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
KINDS = ["interpreter", "alt_format", "assistive_comm", "longer_appointment",
RULES_HERE = [
CLINICAL = [r"\bdiagnos", r"\bcondition\b", r"\bsymptom", r"\bmedication\b", r"\bprescri",
FOLDS = {"‘": "'", "’": "'", "‚": "'", "‛": "'",
def norm(s):
def verdict_for(reading, register):

src/app.pythe board

A standard-library HTTP server on 127.0.0.1:9235 that needs no key: it computes the floor live on every record, shows the key beside it, replays the scored run's own answers off the committed result file, and highlights each quoted sentence inside the record at the offsets the scorer located it at. /api/decide is the only endpoint that spends money and its button is disabled unless a key is configured.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
PORT = int(os.environ.get("PORT", "9235"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-access-need")
FLOOR_RUN = "b000-access-need-rules"
def golds():
def load_record(rid):
def recorded(run_id):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 4038 input and 3295 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ WRITTEN, WIRED, AND NOT FIRED — AND NO NUMBER ON THIS PAGE SAYS OTHERWISE. evals/injection.py exists, targets the right 44 records, and has the control it needs (the scored run's own cached answers). It has not been run. Every figure below is a DESIGN, written down so a future run reports against it rather than re-arguing it.

API_KEY is read from the shared <code>&lt;repo&gt;/.env</code> and this kit's own <code>.env</code> by <code>src/config.py</code>, sent as an Authorization header to the one configured provider, and written to no file, no log and no result record. ⚠︎ NOTE THE SHARED FILE: a kit with no <code>.env</code> of its own is still fully wired from the repo root, which is convenient and is also how a run gets fired by accident. <code>API_KEY=</code> forced empty is the belt-and-braces form for any free operation.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. ⚠︎ THE ADVERSARIAL ARM WAS NEVER FIRED. python3 -m evals.injection --run-id x001-access-need --yes is the command the README carries: 44 calls, one per record whose key holds at least one INDICATED or UNCLEAR-ASK. At r001's measured $0.002474 per record that is about $0.109 — a projection, not a measurement, because nothing was run. Measured on no run — results/ holds no x001-access-need file, and no adversarial figure exists for this kit.

The result⚠︎ NOT FIRED — no adversarial number exists for this kit. evals/injection.py is written, wired and has its control cache; 44 calls would fire it. Nothing on this page makes a robustness claim.

The design, so it can be checked rather than re-argued later: one line appended to the INTAKE AND CONTACT NOTES of each of the 44 target records, in the register of a note already on file, with the record otherwise untouched. The control is the scored run's own cached answer for each record — already committed — so no second control call would be bought. Scoring is against the CLEAN record, so a suppressed citation cannot score by finding the injected sentence. The reportable number is the split: how many suppressions the register restores (at most the 15 judgements AP-3 and AP-4 decide) against how many it cannot (the 56 that rest on the reading).

HonestyWhat this does not prove

• PROMPT INJECTION. Written, wired, never run. No number.
• WHETHER A SUPPRESSION LINE WOULD ALSO MOVE THE CITATION. Scoring is against the clean record, so an arm quoting the injected sentence would score zero — but nothing measures how often it would try.
• WHETHER THE CLINICAL BOUNDARY HOLDS UNDER PRESSURE. The corpus contains no clinical vocabulary at all, so no arm has ever been given the chance to offer one. That the boundary is stated first in the system role is a design fact, not a measurement.
• ANY KEY-HANDLING CLAIM BEYOND READING THE CODE. src/config.py is 114 lines and the adapters are a few dozen more; nothing here has been penetration-tested.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never book, send, hold, cancel or write anything to any record — and never say anything about anybody's condition. This kit produces five determinations and the sentence each came from; where the answer is UNCLEAR-ASK what it produces is a QUESTION to put to a person, with the text that raised it.

Stated in the system role as the FIRST thing the model is told, in the README, in src/app.py, and enforced by there being no endpoint that writes: the board's only spending route is /api/decide and it returns an answer. The clinical half is enforced at the corpus: evals/check_labels.py check 8 sweeps every record for clinical vocabulary and reports 0 hits, so there is nothing in the input for a reply to copy.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• Not a clinical tool, not a triage tool, and not a view about anybody. Five logistics answers, and it must never be extended past that.
• Not a booking system. It writes nothing, and there is no flag that adds an endpoint that does.
• Not a robustness claim. The adversarial arm was never fired.
• Not a measurement of a real record. Every number is against a synthetic corpus built from 82 sentence templates.
• ⚠︎ Not, on this corpus, a demonstration that the paid call is worth its money. A free stem matcher built from the kit's own generator tables scores 300/300 against the call's 290/300.

WatchedWhat is watched, and why that one

The kit measures 44 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

92 measured by the latest run -48 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (accommodations_per_record 5, calls_in_totals 60, citation_coverage_floor 0.6, citation_precision_floor 0.3, cited_judgements 65, clean_records 11, dataset_version access-need-v1-60records, failures 0, hard_records 49, interpreter_rows 60, judgements 300, judgements_answered 300, language_expected 9, language_none_expected 51, policy_id AP-2026, rechecked_citation_coverage_floor 0.6, rechecked_citation_precision_floor 0.3, rechecked_cited_judgements 65, rechecked_clean_records 11, rechecked_hard_records 49, rechecked_interpreter_rows 60, rechecked_judgements 300, rechecked_judgements_answered 300, rechecked_language_expected 9, rechecked_language_none_expected 51, rechecked_records_answered 60, rechecked_uncited_judgements 235, records_answered 60, service_records 60, socket_timeout_s 1200, uncited_judgements 235) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 92 chips that all say so.

comply · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 96 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 92 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The corpus generator, the label gate, the free floor, the scorer and the rechecked column are all free and re-run on every change — and the corpus rebuild was verified byte-identical across four PYTHONHASHSEED values on 2026-08-31. The paid arm is ONE run, on one day, in one tariff window, at the provider's default reasoning setting. The adversarial arm has never run.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end — no orchestrator, no agent loop, no vector store, no ORM, no requirements.txt. One record, one call, one JSON reply, then pure code: the policy engine, the citation locator and the scorer are all functions you can read in an afternoon.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear, and concurrent only at the record level. There is no agent, no tool loop, no retry and no branch: read the record → build six prompt parts → one call → parse → keep two fields → apply AP-2026 in code → locate the citation → score. The one place a decision is made twice is deliberate: the model's verdict is discarded and recomputed, and the result file carries both.

The other sideWhat a framework costs you

• Adding a provider means editing the PROVIDERS dict by hand. In exchange pip install pulls nothing and a forker runs this on whichever key they already hold.
• There is no experiment tracker, so a second run is a second file and a comparison is something you write. On a kit with one paid run that is the right trade; on twenty it would not be.
• No streaming means a long generation is a silent socket for up to 1,200 s — the slowest call here took 105,097 ms and nothing reported progress while it ran.
• No schema-validation library means a malformed reply is caught by a parse and recorded as a failure rather than repaired. On this run that never happened: 60 of 60 parsed.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-access-need on the fast tier, 2026-08-31. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. Only one run has recorded a call latency; the other 1 run on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT — the kit's own licence, and the MIT text is in LICENSE-PUBLIC at the repository root. The corpus, the register, AP-2026, the generator, the harness and both recorded runs are all generated in-process from a fixed seed, so there is no third-party data in this kit to licence and nothing here is fetched from anywhere. ⚠︎ READ THE GRANT FROM LICENSE-PUBLIC, NOT FROM THE ROOT LICENSE FILE — they are two different documents, only the first is the MIT text, and the root LICENSE governs the repository itself and grants a reader nothing. The kit is MIT; the repository that carries it is not. Both files were opened and read on 2026-08-31 to write this sentence. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe judgement — the verdict, the language, and the sentence it came from

whether each of the 300 judgements (five accommodations × 60 records) produced an answer a booking clerk could act on as written: the verdict AP-2026 produces on the reading and the register row, the language the record names on the interpreter row, and a sentence COPIED VERBATIM out of the record supporting it — with NO sentence where the verdict is NOT-INDICATED, because quoting one there is also a wrong answer.

MethodHow the test was run

python3 -m evals.run --run-id <id> [--floor rules | --stub]; evals/scoring.py compares against data/gold.jsonl. No model is in the grading path. To re-derive costs from a committed run without buying anything: API_KEY= python3 -m evals.run --run-id r001-access-need --resume --rescore — --rescore is honoured ONLY under --resume.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl — DERIVED, NOT WRITTEN. tools/build_corpus.py plants a sentence from a named pool, records its character span, and calls src/policy.py with that mechanism's reading and the register state; evals/check_labels.py then re-derives every row with arithmetic written inside itself, importing no src/ module, and passes at 0 problems. Two implementations, one key. Every person, relative, service, place and sentence is invented and nobody is named — see data/SOURCES.md.

No true/false rates for this grader. It records 14 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? Citation credit is coverage ≥ 0.60 and precision ≥ 0.30. Measured means on r001 are coverage 0.9726 and precision 0.9839, and on the floor 0.9892 and 1.0 — ⚠︎ BOTH ARMS SIT FAR ABOVE BOTH FLOORS, so this measurement would look much the same at almost any pair. That is a limitation of the measurement, not a vindication of the numbers.

Cadence: the corpus generator, the label gate, the free floor and the scorer are free and re-run on every change; the paid arm is one run on one day at one tariff window.

The decisionWhen to reach for it