The business caseThe problem this solves

A utilization reviewer deciding inpatient versus observation needs four things off one admission record: whether a practitioner order exists and whether what is documented actually establishes an inpatient admission, what length of stay the ADMITTING PRACTITIONER documented, the comorbidities, and the services delivered. Today somebody opens the chart and reads all five sections to find them, on every encounter — and the two places it goes wrong are the two that look easiest: an order-shaped line that is not an order, and a discharge date that belongs to somebody else. The reviewer's own pass over one admission record's five sections — the encounter facts, the orders, the practitioner documentation, the services and the review notes — to find four elements and the line behind each one.

The inputThe actual admission records

The corpus is 60 admission records, 0.08 MB (txt 60). The four things this kit has to measure cannot be recovered from a document by reading it harder, and none of them exists as ground truth on any real corpus: whether what is documented actually ESTABLISHES an inpatient admission (three states, not two); whose expectation a discharge date belongs to; the exact admission date-time, because the two-midnight count is a subtraction and one side has to be a fact; and exactly which line carries each element, to the character, so a citation can be scored as an interval rather than as a vibe. So the facts are injected, the key is DERIVED from them by src/benchmark.py and src/policy.py, and the records are written around them.

ADMISSION RECORD ADM-0001 -- Admission after an out-of-hours assessment

ENCOUNTER FACTS
  Encounter                     ADM-0001
  Admitted                      2026-06-10 12:45
  Facility                      Ordley Hospital
  Service line                  general medicine
  Admitting role                admitting practitioner
  Record assembled as at        2026-09-01

ORDERS AND AUTHENTICATION

  2026-06-10 13:11  Order: admit to inpatient, service general medicine -- entered and authenticated by the admitting practitioner.

ADMITTING PRACTITIONER DOCUMENTATION

The record was assembled for utilization review in the routine overnight batch. Active problems recorded at admission: atrial fibrillation and peripheral vascular disease. The admitting practitioner records that the patient was well enough to walk to the ward unaided. A property list was completed on arrival and filed with the ward record.

Documented expectation at admission: discharge anticipated on 2026-06-13 once the treatment course is complete. The encounter was created when the patient arrived and has been added to twice since. The assessment records a comfortable patient in no acute distress.

SERVICES DELIVERED

  2026-06-11  Delivered during the stay so far: a physiotherapy assessment, continuous cardiac monitoring and supplemental oxygen by mask.

UTILIZATION REVIEW NOTES

  2026-06-12  No payer query has been raised against this encounter.
  2026-06-12  The encounter appears once on the daily census extract.

The outcomeWhat a good result looks like

Four evidence elements a reviewer confirms instead of a record they re-read: for each one a finding in that element's own vocabulary, the line that carries it quoted verbatim and locatable, and — on the expectation row — the date or night count the record states. Plus one support verdict, derived in code from those readings and the encounter's own admission date-time. SUPPORTS, never determines.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• You want the evidence assembled and a human decides — this kit, as shipped — four elements, four quoted lines, one derived verdict
  the model reads and pure code decides, so the two failure modes that matter (an order-shaped line read as an order, a discharge date read off the wrong sentence) are both READING failures the page shows you the line for
• You only want the hard gate — does a valid admission order exist — the admitting_order row alone, and ignore the other three
  it is the one requirement independent of everything else on the record, it is answered in three classes rather than two, and on this corpus both arms got the ORDER-GAP verdict right on 23 of 23 records
• You want a cheap first pass and can live with the misses — the free rules floor. $0.00, no network, no key
  24 of 60 records and 47 of 60 support verdicts for nothing, and it never once softened an order gap

At a glanceHow the whole thing runs

Run once, for real, on 2026-09-01. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-01 — r001-admission-status. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — ⚠︎ PARTIALLY EVIDENCED, AND THE REST IS NOT MEASURED. What was observed on this machine, with API_KEY blanked: tools/build_corpus.py regenerates all 60 records and the whole answer key; evals.check_labels re-derives every row and exits 0; the rules floor scores 24 of 60 records for $0.00; the --stub pass exercises the full wiring for $0.00; and tools/shoot_ui.mjs starts the board, drives it and writes all five screenshots without a key. What was NOT measured is a clone into an empty directory on a machine that has never held this repo — the runs above were made in the working tree, and the difference is not nothing.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one admission record end to end — one record, one call, four element findings, one support verdict and the recheck, including provider-side reasoning left at the tier's default. Measured over all 60 calls of r001-admission-status; fastest 7471 ms, slowest 57748 ms.

Current processWhat it replaces

The reviewer's own pass over one admission record's five sections — the encounter facts, the orders, the practitioner documentation, the services and the review notes — to find four elements and the line behind each one.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one element judgements (4 per record), plus 60 support verdict is

No train/test split, because nothing is trained or tuned, and no segmentation, because one record is one unit and goes into one call whole. The corpus is split by CASE instead — 16 planned cases, written record by record rather than sampled, so every denominator on this page is a counted one: evidence_thin 4, expectation_absent 4, expectation_decoy 5, expectation_vague 4, late_admit_two_midnights 3, long_hours_one_midnight 3, mixed_gaps 2, nothing_documented 3, order_in_note_only 3, order_missing_long_stay 5, order_observation_only 4, order_status_blank 2, order_verbal_unauthenticated 4, severe_one_midnight 6, supported_clean 5, supported_nights_form 3.

SetupWhat the setup figure measured

THERE IS NO INDEX AND NOTHING TO RETRIEVE FROM: one record goes into one call whole. Two things a reader might call an index are neither — data/records.json is the ENCOUNTER FACT table, read in memory by src/policy.py for one field (admitted_at), and data/policy.md is the rulebook, which goes into every prompt whole at 8,180 characters and is never searched.

LicenceLicence

MIT — the kit's own licence, and the MIT text is in LICENSE-PUBLIC at the repository root. Nothing here is derived from any other corpus, so there is no upstream licence to carry.

Bring your ownBring your own admission records

Drop your own records as .txt into data/corpus/, add one row per record to data/records.json (id, title, case, hard, admitted_at as YYYY-MM-DD HH:MM, facility, service_line, attending_role), and write data/gold.jsonl with the finding, the citation and its span for all four elements, the expectation value on the stay_expectation row, and the derived midnights / meets_benchmark / support / rule. THE SPAN IS COMPUTABLE — src/citation.py::span_of(norm(text), line) — and so is the verdict: src/policy.py::decide_record(...). Replace data/policy.json and data/policy.md with your own wording; the rule ids, their order and their outcomes are read, never hard-coded. Then run python3 -m evals.check_labels: it refuses a key that does not follow from its own record, and it retypes the rule table so a rulebook edit that misses one of the two renderings fails rather than ships.

⚠︎ And what stops being true when you do: ⚠︎ THE WHOLE ADMISSION RECORD GOES TO A MODEL PROVIDER VERBATIM — the orders, the practitioner's notes, the services delivered, the review notes. That is clinical documentation about an identifiable person, and whether it may be disclosed to a third-party provider is a decision for whoever holds the record under whatever agreements cover it. It is not one a kit can make on their behalf. Run it against a de-identified extract you are permitted to disclose, or against a model in your own environment. ⚠︎ AND EVERY NUMBER ON THIS PAGE STOPS BEING TRUE THE MOMENT THE CORPUS CHANGES. The measured 60 of 60 is a fact about 79 sentence templates, not about charting. What travels is the SHAPE: the order gate is a hard requirement independent of everything else on the record, and the benchmark runs on the practitioner's documented expectation and nothing else.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 23,810 characters assembled, 8,539 are vocabularies — 36% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The whole prompt as sent for ADM-0001, the first fresh record of run r001-admission-status — all six parts in order, rebuilt from src/prompt.py on 2026-09-01 and checked against the run record's own prompt_parts: 2,791 / 8,180 / 2,306 / 6,233 / 1,540 / 2,760 characters, 23,810 total, identical part for part.

Full promptThe prompt in full

You are reading one hospital admission record and ASSEMBLING the evidence a utilization reviewer
needs in order to decide whether the stay should be inpatient or observation. Your output is the
package that reviewer opens -- four evidence elements, each with the line in the record that carries
it, and what the two-midnight benchmark supports on the documented expectation.

YOU DO NOT MAKE THE DETERMINATION AND NOTHING HERE CHANGES ANYBODY'S STATUS. The determination
belongs to a human utilization reviewer or a physician advisor. Nothing you produce writes a patient
status to any system of record, tells billing anything, or tells the care team anything.

AND YOU NEVER INFER MEDICAL NECESSITY FOR INPATIENT STATUS FROM CLINICAL SEVERITY. How unwell the
record makes the patient sound is not evidence about admission status. Neither is the number of
comorbidities, and neither is how intensive the services delivered were. Those three are ASSEMBLED,
because a reviewer wants them in front of them; they are never REASONED FROM. What decides is the
admitting practitioner order and the expectation of length of stay THE RECORD DOCUMENTS.

How to read the record:

- ENCOUNTER FACTS, ORDERS AND AUTHENTICATION, ADMITTING PRACTITIONER DOCUMENTATION, SERVICES
  DELIVERED and UTILIZATION REVIEW NOTES are all part of the record and all of them count.
- Distinguish WHOSE STATEMENT IT IS. The admitting practitioner's own documented expectation of
  length of stay is the only expectation the benchmark runs on. A discharge planner's provisional
  date, a conditional sentence about treatment that has not happened, and an expectation recorded at
  a previous encounter are all in the record and none of them is that.
- Distinguish AN ORDER FROM SOMETHING ORDER-SHAPED. A verbal order nobody has authenticated, an
  order for OBSERVATION services, an intent to admit written inside a progress note, and an order
  with no status word on it are all order-shaped and none of them establishes an inpatient
  admission.
- A phrase does not evidence an element just by appearing. Orders exist on records that carry no
  ADMISSION order; a date appears on lines that state nobody's expectation.
- Answer all four elements, once each, in the order given, using the vocabulary listed for THAT
  element and no other.
- DO NOT DO THE MIDNIGHT ARITHMETIC. Report the expected discharge DATE, or the number of further
  NIGHTS, exactly as the record states it, and do not convert one into the other. The counting is
  done in code.
- Apply the rulebook as written, INCLUDING THE ORDER ITS RULES ARE APPLIED IN, to give one support
  verdict for the record.
- Give one confidence between 0 and 1 for the record's answers taken together.

Reply with JSON and nothing else, in the shape given at the end.

THE ASSEMBLY RULEBOOK, as approved:

# TM-2026 — the assembly rulebook, as approved

*Read as at 2026-09-01. **TM-2026 is this kit's own rulebook.** Its two requirements are anchored:
**42 CFR 412.3(a)** makes inpatient status begin with a formal admission under an order by a
practitioner with admitting privileges, and **42 CFR 412.3(d)(1)** is the two-midnight benchmark —
an inpatient admission is generally appropriate where the admitting practitioner expects the
beneficiary to require hospital care crossing at least two midnights. **The rule ids below, their
order, their wording and the four verdict names are this kit's, not the regulation's.** Nothing here
is legal, coding or billing advice, and none of the regulation's text is reproduced.*

## What this rulebook decides, and what it does not

It decides **what the assembled evidence supports**, on one admission record, for the question a
utilization reviewer is about to answer: inpatient or observation.

**It does not make that determination.** It assembles four elements — the admitting order, the
documented expectation of length of stay, the comorbidities and the services delivered — says for
each whether the record evidences it, and states what the two-midnight benchmark would support on
the documented expectation. **The determination is a human utilization reviewer's or a physician
advisor's.** Nothing here writes a patient status to any system of record, tells billing anything,
or tells the care team anything.

**And it decides nothing from clinical severity.** No rule below takes severity as a condition, and
`src/policy.py` cannot read one: the engine's only inputs are the order finding and the number of
midnights. A record can describe somebody very unwell and carry a documented expectation that does
not cross two midnights, and the assembled package says exactly that.

**Four verdicts, and only four.**

| verdict | what it means |
|---|---|
| `ORDER-GAP` | the admitting order is missing or does not establish an inpatient admission. **A hard gap, independent of clinical necessity.** |
| `EXPECTATION-GAP` | no expectation of length of stay is documented, so there is nothing for the benchmark to be computed against |
| `BENCHMARK-NOT-MET` | the documented expectation crosses fewer than two midnights |
| `INPATIENT-SUPPORTED` | an order is documented and the documented expectation crosses two midnights or more |

`ORDER-GAP` is listed first deliberately. It is the requirement that does not bend, and a package
that softens it — records it as something to chase while the rest of the evidence carries the
package forward — has failed at its first line.

## The five rules, in the order they are applied

**The first rule whose condition is met is the one recorded.** The order is the rulebook: the same
facts under a different order produce different answers, which is why the order is written down
rather than left to whoever applies it.

**TM-1 — No practitioner order, no inpatient status. → ORDER-GAP**
Where the record documents no admission order at all, the package is assembled with the order named
as a **hard gap** and goes no further down this table. The requirement at 42 CFR 412.3(a) is
independent of clinical necessity: it is not softened by a long expected stay, by comorbidity
burden, by the services delivered, or by how unwell the person is. A package that records this as a
soft gap — something to chase, something the rest of the evidence can carry — has failed, because
the rest of the evidence cannot carry it.

**TM-2 — An order that does not establish inpatient admission is not an order. → ORDER-GAP**
Where what is documented is order-shaped and does not establish a formal inpatient admission — a
verbal order nobody has authenticated, an order for **observation** services, an intent to admit
written inside a progress note, an order with no status word on it — the package is assembled with
the order named as a hard gap, exactly as under TM-1. **This is the commonest way the first line of
a package is wrong**, and it is the line least likely to be re-read, because it looks
administrative.

**TM-3 — An undocumented expectation cannot meet a benchmark. → EXPECTATION-GAP**
Where the record documents no expectation of length of stay — none at all, or a reference to one
that states no date and no number of nights — there is nothing for the two-midnight benchmark to be
computed against, and the package says so. It does **not** report the benchmark as unmet: unmet is a
finding about a documented expectation, and reporting an absence as a finding is how a gap in the
charting becomes a conclusion about the stay.

**TM-4 — Under two midnights, the expectation does not support inpatient status. → BENCHMARK-NOT-MET**
Where the documented expectation crosses fewer than two midnights from admission, the benchmark at
42 CFR 412.3(d)(1) is not met on that expectation, and the package says exactly that — whatever the
record says about how unwell the person is, how many comorbidities are listed, or how intensive the
services delivered were. **None of those is an input to this rule.** Clinical severity is not
evidence of inpatient status; the documented expectation is what the benchmark is run on, and
inferring the one from the other is the single thing this kit exists to refuse.

**TM-5 — Two midnights or more, with an order, supports inpatient status. → INPATIENT-SUPPORTED**
Where a formal order is documented and the documented expectation crosses two midnights or more from
admission, the assembled evidence supports inpatient status under the benchmark. **Supports, not
determines.**

## The two-midnight benchmark

**Midnights are counted, not hours, and the two are not the same measure.** A midnight is crossed
each time the calendar date advances between the admission and the expected discharge, so the count
is the difference in calendar days.

- Admitted **23:50**, expected discharge the day after next → **two midnights**, on about 24 hours
  of stay.
- Admitted **01:00**, expected discharge late the following day → **one midnight**, on about 46
  hours of stay.

An assembler that counts elapsed hours gets both of those backwards. The arithmetic is done in pure
code, in `src/benchmark.py`, from two inputs:

- **the admission date-time**, a structured encounter fact read from `data/records.json` and never
  inferred from prose;
- **one of** the expected discharge date, or the number of further nights, whichever the record
  documents.

**The expectation is never inferred.** Not from the services delivered, not from the comorbidity
list, not from how unwell the record makes the person sound, not from a length of stay somebody
other than the admitting practitioner mentioned, and not from a conditional sentence about a stay
that has not happened.

## The three classes each element is answered in

| class | what it means |
|---|---|
| `AMBIGUOUS` | the record gestures at this element without establishing it |
| `PRESENT` | the record evidences it |
| `ABSENT` | the record carries nothing for it |

`AMBIGUOUS` is listed first for the same reason `ORDER-GAP` is. A two-value check has to round every
half-record to one side: it rounds an unauthenticated verbal order up to an order, or "we anticipate
a short stay" up to a documented expectation. Both of those are how a package gets assembled as
though a requirement were met.

**The admitting-order row answers in its own words** — `order-present`, `order-ambiguous`,
`order-missing` — because that is what a utilization reviewer calls them, and each is counted as its
own class on the report.

## What this rulebook does not cover

Stated here rather than left to be discovered:

- **The case-by-case exception** at 42 CFR 412.3(d)(3), where a stay expected to cross fewer than
  two midnights may still be appropriate for inpatient admission on the admitting practitioner's
  documented judgement. TM-2026 has no rule for it: a record it applies to is answered
  `BENCHMARK-NOT-MET` here, which is a statement about the benchmark and not about the admission.
- **The inpatient-only list** at 42 CFR 412.3(d)(2).
- **Every payer rule that is not Medicare.**

THE FOUR SUPPORT VERDICTS, and what answering each one commits you to:

  ORDER-GAP             
      The admitting practitioner order is missing, or what is documented does not establish a formal inpatient admission (TM-1, TM-2). THE PACKAGE NAMES IT AS A HARD GAP AND STOPS THERE. This requirement is independent of clinical necessity: it is not softened by a long expected stay, by comorbidity burden, by the services delivered, or by how unwell the person is.
      Answered wrongly in the direction of an order that is not there, a package goes to a reviewer with its first line saying a hard regulatory requirement is met, and that is the line least likely to be re-read because it looks administrative. Answered wrongly the other way, somebody is sent chasing a signature that is already on file.

  EXPECTATION-GAP       
      No expectation of length of stay is documented as the admitting practitioner's own, so there is nothing for the two-midnight benchmark to be computed against (TM-3). This is NOT the same as the benchmark being unmet.
      Reported as an unmet benchmark, a gap in the charting becomes a conclusion about the stay, and the thing that would fix it -- a sentence somebody needs to write -- never gets asked for.

  BENCHMARK-NOT-MET     
      An order is documented and the documented expectation crosses FEWER THAN TWO MIDNIGHTS from admission (TM-4). It is a statement about the benchmark on the documented expectation, and about nothing else.
      Answered wrongly in the direction of support, a package is assembled as though the record justified inpatient status when its own documented expectation does not. On this corpus that is what happens when an arm reads how unwell somebody sounds, or reads a discharge date that belongs to somebody else, and reasons from it.

  INPATIENT-SUPPORTED   
      An order is documented and the documented expectation crosses two midnights or more (TM-5). The assembled evidence SUPPORTS inpatient status under the benchmark. It does not determine it -- that is the utilization reviewer's or the physician advisor's, and nothing here writes a status anywhere.
      Answered wrongly, a package understates its own evidence, which costs a re-read and, later, a status somebody has to defend without the evidence in front of them.

THE FOUR EVIDENCE ELEMENTS, in the order you must answer them, the words each one is
answered in, and what the strongest answer commits somebody to:

  admitting_order      The admitting practitioner order
      THE PACKAGE GOES FORWARD SAYING THE HARD REQUIREMENT IS MET. 42 CFR 412.3(a) makes inpatient status begin with a formal admission under an order by a practitioner with admitting privileges. Answering PRESENT on an order that is not one -- a verbal order nobody authenticated, an order for observation services, the word 'admit' inside a progress note -- hands a reviewer a package whose first line is wrong, and it is the line least likely to be re-read because it looks administrative.
      order-ambiguous    something order-shaped is documented and it does not establish an inpatient admission -- a verbal order nobody has authenticated, an order for OBSERVATION services, an intent to admit written inside a progress note, or an order whose status word is missing
      order-present      a formal order to admit as an inpatient is documented, by a named practitioner or role with admitting privileges, and it is authenticated or countersigned
      order-missing      nothing in the record documents an admission order at all

  stay_expectation     The documented expectation of length of stay
      THE TWO-MIDNIGHT BENCHMARK GETS COMPUTED AGAINST IT. This is the only element the benchmark can be run on: 42 CFR 412.3(d)(1) turns on what the admitting practitioner EXPECTED at the time of admission, documented in the record. A sentence that gestures at a stay without stating a length is not an expectation the benchmark can be run against, and scoring one anyway is arithmetic performed on nothing.
      ambiguous          the record refers to how long the stay is expected to be without stating a date or a number -- 'a short stay', 'a few days', 'an extended admission' -- or the only length in the record belongs to somebody other than the admitting practitioner, or is conditional on something that has not happened
      documented         the record states, as the practitioner's own expectation at admission, either an expected discharge DATE or a number of further NIGHTS or MIDNIGHTS in hospital
      absent             the record documents no expectation of length of stay at all

  comorbidities        Documented comorbidities bearing on the stay
      THE PACKAGE CARRIES THE COMORBIDITY EVIDENCE THE REVIEWER ASKED FOR. It is context for a human reading, and it is the element most easily mistaken for the answer: comorbidity burden is not the benchmark and does not substitute for the practitioner's documented expectation. Assembling it is the job; reasoning from it is not.
      ambiguous          the record refers to comorbidity burden without naming any condition -- 'multiple chronic conditions', 'significant past medical history'
      documented         the record names comorbid conditions bearing on this stay
      absent             the record documents no comorbidities

  services_delivered   Services delivered during the stay
      THE PACKAGE CARRIES WHAT WAS ACTUALLY DONE, which is what a reviewer compares against what was expected. Like comorbidities it is EVIDENCE and not a verdict: services at an inpatient intensity do not establish that an order exists or that the documented expectation crossed two midnights.
      ambiguous          the record refers to care having been delivered without naming any service -- 'treatment continues', 'ongoing management'
      documented         the record names services actually delivered during this stay
      absent             the record documents no services delivered

THE TWO-MIDNIGHT BENCHMARK, and why you are not asked to compute it:

  MIDNIGHTS ARE COUNTED, NOT HOURS, AND THE TWO ARE NOT THE SAME MEASURE. A midnight is crossed each time the calendar date advances between the admission and the expected discharge, so the count is the difference in calendar days: an admission at 23:50 whose expected discharge is the day after next crosses two midnights on about 24 hours of stay, and an admission at 01:00 whose expected discharge is late the following day crosses one midnight on about 46 hours. An assembler that counts elapsed hours gets both of those backwards. The record's admission date-time is held as structured data and the counting is done in
  code, from that and from whichever of the two values the record states. The expectation is never inferred -- not from the services delivered, not from the comorbidity list, not from how unwell the record makes the person sound, not from a length of stay somebody other than the admitting practitioner mentioned, and not from a conditional sentence about a stay that has not happened.
  So: report the expected discharge DATE the record names, as YYYY-MM-DD, OR the number of
  further NIGHTS or MIDNIGHTS it names, as an integer -- whichever the record uses, and null for
  the other. Never both, never converted, never computed.

HOW TO QUOTE THE RECORD, and how it will be read.

For every element you answer with anything other than its ABSENT word (`order-missing`, or
`absent`), `citation` must be ONE SENTENCE OR ONE DATED LINE COPIED VERBATIM out of the record --
the text that carries the element. It may come from any section.

  - Copy it character for character. It is located in the record by searching for it, so a
    paraphrase, a shortened version, an ellipsis in the middle, or two lines joined together will
    not be found at all and will score nothing. There is no partial credit for a quote the record
    does not contain.
  - Quote the line, not the section. What is returned is compared with that line by character
    overlap: it must cover at least 60 pct of the line, and at least 30 pct of what you
    return must be the line. Returning the whole record scores nothing.
  - The rulebook is NOT part of the record. A rule is never the citation.
  - Where you answer `order-missing` or `absent`, `citation` is null. Quoting something in support
    of an element the record does not carry is counted as a wrong answer, not as an empty one --
    and on the admitting-order row it is the most expensive wrong answer in this kit.

THE ADMISSION RECORD, verbatim:

ADMISSION RECORD ADM-0001 -- Admission after an out-of-hours assessment

ENCOUNTER FACTS
  Encounter                     ADM-0001
  Admitted                      2026-06-10 12:45
  Facility                      Ordley Hospital
  Service line                  general medicine
  Admitting role                admitting practitioner
  Record assembled as at        2026-09-01

ORDERS AND AUTHENTICATION

  2026-06-10 13:11  Order: admit to inpatient, service general medicine -- entered and authenticated by the admitting practitioner.

ADMITTING PRACTITIONER DOCUMENTATION

The record was assembled for utilization review in the routine overnight batch. Active problems recorded at admission: atrial fibrillation and peripheral vascular disease. The admitting practitioner records that the patient was well enough to walk to the ward unaided. A property list was completed on arrival and filed with the ward record.

Documented expectation at admission: discharge anticipated on 2026-06-13 once the treatment course is complete. The encounter was created when the patient arrived and has been added to twice since. The assessment records a comfortable patient in no acute distress.

SERVICES DELIVERED

  2026-06-11  Delivered during the stay so far: a physiotherapy assessment, continuous cardiac monitoring and supplemental oxygen by mask.

UTILIZATION REVIEW NOTES

  2026-06-12  No payer query has been raised against this encounter.
  2026-06-12  The encounter appears once on the daily census extract.


Reply with JSON and nothing else, exactly this shape:

{
 "elements": [
   {
    "element": "admitting_order" | "stay_expectation" | "comorbidities" | "services_delivered",
    "finding": the word for THIS element, from the list above,
    "citation": "<text>" or null,
    "expected_discharge": "YYYY-MM-DD" or null,
    "expected_nights": <an integer> or null
   },
   ... one object per element, all four, in this order: admitting_order, stay_expectation, comorbidities, services_delivered
 ],
 "support": "ORDER-GAP" | "EXPECTATION-GAP" | "BENCHMARK-NOT-MET" | "INPATIENT-SUPPORTED",
 "confidence": <a number between 0 and 1>,
 "why": "<text>"
}

What each field means:
  element              which evidence element this row is about. Answer all four, once each, in the order given
  finding              what this record carries for this element. Use the vocabulary listed for THIS element and no other
  citation             ONE SENTENCE OR ONE ORDER LINE COPIED VERBATIM from the record -- the text that carries this element. Copy it exactly, character for character; do not paraphrase, do not shorten with an ellipsis, do not join two lines. NULL where the finding is order-missing or absent: quoting something in support of an element the record does not carry is counted as a wrong answer, not as an empty one
  expected_discharge   STAY_EXPECTATION ROW ONLY, and null on the other three. The expected discharge DATE the record documents, as YYYY-MM-DD. NULL where the record documents the expectation as a number of nights instead, and NULL where no expectation is documented. Never computed, never inferred -- only a date the record itself states
  expected_nights      STAY_EXPECTATION ROW ONLY, and null on the other three. The number of FURTHER NIGHTS or MIDNIGHTS in hospital the record documents, as an integer. NULL where the record documents an expected discharge date instead, and NULL where no expectation is documented. Do not convert a date into a night count and do not convert a night count into a date -- the arithmetic is done in code, from whichever one the record states
  support              what the assembled evidence supports for this record, under TM-2026's five rules applied in their published order. This is your answer to the whole question. It is scored as you give it and then re-derived in code from your own four findings and your own extracted date or night count, so the two can be compared
  confidence           your own number between 0 and 1 for this record's answers taken together. It is published as evidence and never used to change an answer
  why                  one sentence naming the text that decided the order finding and the stay expectation

Four element objects for one record. No more, no fewer.

Raw responseThe raw response

{
 "elements": [
  {
   "element": "admitting_order",
   "finding": "order-present",
   "citation": "2026-06-10 13:11  Order: admit to inpatient, service general medicine -- entered and authenticated by the admitting practitioner.",
   "expected_discharge": null,
   "expected_nights": null
  },
  {
   "element": "stay_expectation",
   "finding": "documented",
   "citation": "Documented expectation at admission: discharge anticipated on 2026-06-13 once the treatment course is complete.",
   "expected_discharge": "2026-06-13",
   "expected_nights": null
  },
  {
   "element": "comorbidities",
   "finding": "documented",
   "citation": "Active problems recorded at admission: atrial fibrillation and peripheral vascular disease.",
   "expected_discharge": null,
   "expected_nights": null
  },
  {
   "element": "services_delivered",
   "finding": "documented",
   "citation": "2026-06-11  Delivered during the stay so far: a physiotherapy assessment, continuous cardiac monitoring and supplemental oxygen by mask.",
   "expected_discharge": null,
   "expected_nights": null
  }
 ],
 "support": "INPATIENT-SUPPORTED",
 "confidence": 0.98,
 "why": "The order line 'Order: admit to inpatient, service general medicine -- entered and authenticated by the admitting practitioner' establishes the admitting order, and 'Documented expectation at admission: discharge anticipated on 2026-06-13' documents the admitting practitioner's expected discharge date."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Decide inpatient or observation for a hospital stay — 60 admission records. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

No model grades anything. Every metric is exact match against data/gold.jsonl on five things: the element finding (a closed vocabulary per element), the expected discharge date, the expected night count, the record's support verdict, and the citation — which is the only one that is not a string comparison. A citation is LOCATED in the normalised record and scored as CHARACTER INTERVALS against the labelled line: coverage ≥ 0.60 of that line and precision ≥ 0.30 of what was returned. Not found is zero, counted separately, with no near-miss credit.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Run it twiceThe same set, run again

⚑ RAW AND RECHECKED ARE IDENTICAL ON BOTH ARMS, AND THAT IS A REAL FINDING RATHER THAN A MISSING COLUMN. recheck_overrides is 0 on the paid run and 0 on the floor: the rulebook engine never had to overturn a verdict either arm offered. It means every error in this kit's numbers is a READING error and none is a rule-application error — which is exactly what the split between the two was built to make visible. The recheck earns its place on an arm that reads correctly and then applies TM-2026 wrongly, and neither arm here did that once. ⚠︎ AND THAT IT WORKS AT ALL WAS RED-PROVEN RATHER THAN ASSUMED. Seeding src/recheck.py to force one verdict moved the FLOOR's rechecked support column from 47 of 60 to 23 of 60 while its raw column stayed at 47; reverting the seed put it back. If the floor branch had been copying its own answer into the rechecked slot — the defect repaired across eight kits on this estate on 2026-08-31 — the column would not have moved.

element findings over the 240 judgements, and support verdicts over the 60 records — There is nothing to average: the two columns of each arm are the same numbers, and the two arms are different instruments rather than two samples of one. The paid arm was run ONCE. No repeat measurement of it exists, so no claim about its variance is made here.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate — a projection onto a rate card, not a bill anyone paid. The token counts come from the provider's own usage block on each of the 60 calls, and 88.2 pct of the output tokens they report are provider-side reasoning the kit did not ask for.

Rates checked 2026-08-27. The provider that actually ran every call here is kept off this page per the series rule — a rate card is a naming. The bill the run ACTUALLY incurred, on the running provider's own dated card (2026-08-23) with its peak/off-peak split and its cached-input tier, is $0.137770 for all 60 calls — $0.002296 each — and is recorded per call in results/eval-r001-admission-status.json. All 60 fell off-peak; the same tokens inside the weekday peak window are $0.275545. That provider is not named here, per the series rule. One further published card is withheld for a different reason: it is not a US rate card, so the figure would not be actionable where a reader buys.

Grading unitWhat the grading figure prices

Scoring is pure Python over committed JSON: 240 element judgements and 60 verdicts in well under a second, on any result set, as many times as you like. That is what makes re-scoring at different citation floors free, and it is why --resume --rescore can re-derive every rechecked block from the cache without a provider in the loop.

The gradersOne way to grade, and why it is the only one

THREE FREE ARMS EXIST FOR THIS KIT AND TWO ARE ON THE BOARD. ALWAYS-PRESENT is the degenerate one: answer the PRESENT word on all four elements of every record and score 167 of 240 element judgements (69.6 pct) — because most admission records DO carry an order line, a comorbidity list and a services list. It gets 0 of the 60 support verdicts right, because a record with every element PRESENT and no expectation value is an EXPECTATION-GAP under TM-3, and it never supplies one. That is the reason the element figure and the verdict figure are published separately and the aggregate is not the headline. THE RULES FLOOR is the one that is committed and shipped: 24 of 60 records, 47 of 60 support verdicts, 191 of 240 element judgements, for $0.00 and no network. It is a real attempt — phrase lists per element, a date and night-count extractor over the line it quoted, and then EXACTLY the rulebook engine the paid arm is rechecked with. ⚠︎ THE THIRD IS NOT MEASURED AND IS NAMED ANYWAY. A stem matcher built from this kit's OWN generator tables would score close to perfectly, because the records are assembled from 79 sentence templates and anybody holding the pool holds the key. That arm was NOT built and NOT run here, so no number for it is published — but a reader shown 24 of 60 without being told the pool exists is being sold something.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-27. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingGrading is free; the bill is the 60 calls that answered.

The scorer, the label gate, the rules floor, the corpus generator, the two-midnight arithmetic and the rechecked column are pure code and cost $0.00 to run against any result set — there is no LLM judge anywhere in the grading path, and re-deriving a committed run (API_KEY= ... --resume --rescore) buys nothing. The figure above is the token count of the ONE paid run (60 calls, 318,250 in / 172,888 out) priced at the same projected card cost_per_query_usd uses. ⚠︎ THERE IS NO SECOND PAID RUN TO ADD: the injection arm (45 calls) was never fired and the --stub pass is free. failures 0, so no call is missing from the totals. The bill the run ACTUALLY incurred, on the running provider's own dated card (2026-08-23) with its peak/off-peak split and its cached-input tier, is $0.137770 for all 60 calls — $0.002296 each — and is recorded per call in results/eval-r001-admission-status.json. All 60 fell off-peak; the same tokens inside the weekday peak window are $0.275545. That provider is not named here, per the series rule.

Cost driversWhat actually moves the bill

• PROVIDER-SIDE REASONING, and it is not close. 152,420 of the 172,888 output tokens (88.2 pct) never reached the reply — the JSON package is four rows, a verdict and a confidence, under 1,500 characters. The output side is the majority of the bill and most of it is thinking nobody asked for.
• THE PROMPT PREFIX, discounted but not free. The system role, TM-2026, the verdict block and the element block are 19,510 of the prompt's 23,810 characters and are byte-identical on every call, so 217,600 of 318,250 input tokens (68.4 pct) billed at the cached rate. On a provider with no cached-input tier the input side of this bill roughly triples.
• THE RECORD ITSELF, which is the smallest part. ADM-0001's record section is 1,540 characters of a 23,810-character prompt (6.5 pct). Ten times longer charting moves the input side and nothing else.
• THE CLOCK, on the card the run was actually billed on. Every call here fell off-peak, at half the peak rate: $0.137770 against $0.275545 for the identical 60 calls inside the weekday peak window. The projected figures above carry no such split, because the cards they use publish one rate.

Your volumeWhat it costs at your volume

Linear in records and flat in everything else. Ten times the records is ten times the calls at the same $0.002296 each, because there is no index to rebuild, no state to carry, no retrieval, and the prompt prefix is identical on every call. The one thing that does NOT scale linearly is the cache discount: it improves with volume, because the 19,510-character prefix is warm for every call after the first. What breaks first is not the bill but the wall clock — p95 is 54,363 ms and the only lever is concurrency.

Model choiceWhy this model

It is the tier the estate runs, so the figure compares with every sibling kit, and it is the cheapest long-context option on the card. ⚠︎ NO SECOND MODEL WAS TRIED, and on a set this one scored 100 pct on, trying one would have measured the corpus rather than the model.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the whole 60-record scored run (r001-admission-status, the fast tier): 318,250 input tokens and 172,888 output tokens, 60 calls, 0 failures, 0 truncations. usd_actually_paid is null because the card it was actually billed on is the withheld one; the measured figure is in the Cost lens.

The codeEvery file, and what it is for

15 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Eight of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pythe corpus — a swap seam

the generator behind data/corpus/ — 60 admission records, .txt, 81,294 bytes, from seed 20260901, re-runnable to the byte. Five sections each: encounter facts, orders and authentication, practitioner documentation, services delivered, review notes

You change it to: the case plan, the sentence pools and the seed. Re-runnable to the byte; the answer key is regenerated from src/policy.py, never edited.

# Generate the admission records, the encounter facts and the answer key.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
SEED = 20260901
AS_OF = "2026-09-01"
DATASET_VERSION = "admission-status-v1-60records"
FACILITIES = ["Bellmarsh General", "Coldharrow Infirmary", "Ravensbeck Hospital",
SERVICE_LINES = ["general medicine", "cardiology", "respiratory medicine", "general surgery",
ROLES = ["attending hospitalist", "attending physician", "consultant of the week",

data/records.jsonthe encounter facts

one row per record. src/policy.py reads admitted_at and nothing else from it — the anchor of the midnight arithmetic, and never inferred from prose

# 

data/policy.jsonthe rulebook, as data

TM-2026's five rules, their order, the four verdicts and the benchmark threshold. src/policy.py computes from this one

# 

data/policy.mdthe rulebook, in words

the same five rules as prose. It goes into every prompt whole, at 8,180 characters, and evals/check_labels.py fails the build if the two renderings disagree

# 

src/elements.pythe element vocabulary — a swap seam

the four elements, the shared three-class axis and each element's own finding words. The prompt, the floor, the scorer and the board all read their vocabulary from here

You change it to: the four elements, their finding words and their meanings. The prompt, the floor, the scorer and the board all read from here, so adding a fifth element is one file plus a corpus rebuild.

# The four evidence elements this kit assembles, the class each one is answered in, and what

CLASSES = ["AMBIGUOUS", "PRESENT", "ABSENT"]
ELEMENTS = {
ORDER = ["admitting_order", "stay_expectation", "comorbidities", "services_delivered"]
FINDINGS = {
CLASS_OF = {k: {v: c for c, v in m.items()} for k, m in FINDINGS.items()}
MEANINGS = {
BENCHMARK_ELEMENT = "stay_expectation"
ORDER_ELEMENT = "admitting_order"
def labels():

src/benchmark.pythe two-midnight arithmetic

midnights = calendar days between the admission date and the documented expected discharge, or the stated night count. Pure code, with both clock edges asserted at import

# THE TWO-MIDNIGHT ARITHMETIC. Pure code, no model, and the reason this kit is gradeable.

THRESHOLD_MIDNIGHTS = 2
ISO_DATE = "%Y-%m-%d"
ISO_DATETIME = "%Y-%m-%d %H:%M"
def parse_admitted(s):
def parse_date(s):
def parse_nights(n):
def midnights(admitted_at, expected_discharge=None, expected_nights=None):
def meets(m):
def explain(admitted_at, expected_discharge=None, expected_nights=None):

src/policy.pythe rulebook engine

an order finding and an integer in, a verdict and a rule id out. NO SEVERITY INPUT EXISTS — that is the kit's refusal, structurally rather than by promise

# THE PURE-CODE STATION. An order finding plus a midnight count in, a verdict out, with the rule id

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY = json.load(open(os.path.join(HERE, "data", "policy.json"), encoding="utf-8"))
POLICY_ID = POLICY["policy_id"]
AS_OF = POLICY["as_of"]
ANCHOR = POLICY["anchor"]
ANCHOR_NOTE = POLICY["anchor_note"]
RULES = POLICY["rules"]
RULE_ORDER = POLICY["rule_order"]
VERDICTS = tuple(POLICY["verdicts"])

src/prompt.pythe prompt

six parts in send order; the first four are byte-identical on every call and are sent first, so a provider that prices cached input bills most of them at the cached rate

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
POLICY_MD = open(os.path.join(HERE, "data", "policy.md"), encoding="utf-8").read()
FIELDS = json.load(open(os.path.join(HERE, "data", "fields.json"), encoding="utf-8"))["fields"]
VERDICTS = list(P.VERDICTS)
VERDICT_MEANINGS = {
SYSTEM = """You are reading one hospital admission record and ASSEMBLING the evidence a utilization reviewer
def _verdict_block():
def _elements_block():
BENCHMARK_TEXT = (

src/classifier.pythe model call

the only place a model is reached. Parses the reply, case-folds the closed vocabularies, and hands it to the recheck

# One admission record in, four evidence elements and one support verdict out. The only place a

MAX_TOKENS = 32000
THINKING = None
def parse_reply(text):
def normalise(obj):
def classify(cfg, text, rec, complete_fn=None, max_tokens=None):
VERDICTS = tuple(prompt.VERDICTS)

src/recheck.pythe recheck

keeps the four findings, the four citations and the expectation value; DISCARDS the support verdict and re-derives it from those readings

# THE PURE-CODE STATION, applied to a model's reply. What the model is trusted with, and nothing

TRUSTED = ("finding", "citation", "expected_discharge", "expected_nights")
def recheck(answer, rec, record_text):

src/citation.pythe citation scorer — a swap seam

locates the quoted line in the normalised record and compares character intervals. Not found is zero, with no near-miss credit

You change it to: COVERAGE_MIN and PRECISION_MIN are both arguments to credit(), so any committed run re-scores at a different pair with --coverage-min / --precision-min and no call is bought.

# HOW A CITATION IS SCORED. Character offsets, two floors, and no partial credit for a quote the

FOLDS = {
def fold(s):
def norm(s):
def norm_quote(s):
def locate(norm_text, quote):
def span_of(norm_text, line):
COVERAGE_MIN = 0.60
PRECISION_MIN = 0.30
def credit(norm_text, gold_span, quote, coverage_min=COVERAGE_MIN, precision_min=PRECISION_MIN):

evals/baseline.pythe free floor — a swap seam

phrase lists per element over the four prose sections, a date and night-count extractor over the line it quoted, then the same rulebook engine

You change it to: what the free arm can see. Widening PATTERNS or the term vocabularies raises the floor and narrows the published margin, which is the honest direction to test in.

# THE FREE FLOOR. No key, no model, no network. A genuine attempt at the job.

MODES = ("rules",)
PATTERNS = {
CONDITION_TERMS = [
SERVICE_TERMS = [
HEADINGS = ["ENCOUNTER FACTS", "ORDERS AND AUTHENTICATION",
READS = HEADINGS[1:]
def sections(text):
def readable(text):
def units(chunk):

evals/scoring.pythe graders

pure code. Element findings per class, the order gate by its own three names, the expectation values, the citations by character overlap, and the support verdict per class

# Grade one arm against the answer key. Deterministic -- no model judges anything here.

VERDICTS = tuple(P.VERDICTS)
CLASSES = tuple(E.CLASSES)
GRADED = ("finding", "citation", "expectation", "support")
TAXONOMY = ("no_answer", "order_gap_softened", "order_gap_invented", "order_class_confused",
def _pct(n, d):
def _s(v):
def date_key(s):
def nights_key(n):
def rows_of(answer):

evals/check_labels.pythe label gate

re-derives the whole answer key with a rule table, a day counter, a normaliser and a span finder written inside itself, importing nothing from src/

# Grade the answer key itself. Free, offline, and it imports NOTHING from src/.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
ELEMENTS = ["admitting_order", "stay_expectation", "comorbidities", "services_delivered"]
ABSENT_WORD = {"admitting_order": "order-missing", "stay_expectation": "absent",
PRESENT_WORD = {"admitting_order": "order-present", "stay_expectation": "documented",
AMBIGUOUS_WORD = {"admitting_order": "order-ambiguous", "stay_expectation": "ambiguous",
RULES_HERE = [
THRESHOLD = 2

src/app.pythe board

the local UI on port 9242. Renders with no key; the only control that spends is disabled without one and says why

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
PORT = int(os.environ.get("PORT", "9242"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-admission-status")
FLOOR_RUN = "b000-admission-status-rules"
def golds():
def load_record(rid):
def recorded(run_id):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 5304 input and 2881 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ WRITTEN, WIRED, AND NOT FIRED — AND NO NUMBER ON THIS PAGE SAYS OTHERWISE. evals/injection.py exists, targets the 45 records whose key is not already INPATIENT-SUPPORTED, and has never been run. results/ carries no x001-admission-status. This kit makes NO robustness claim. What IS structural rather than measured: the record is scored against the CLEAN text, so a citation quoting an injected sentence cannot be located and scores zero; the rule engine takes an order word and an integer, so no sentence can talk it into reading severity; and src/recheck.py discards the support verdict the reply offers and re-derives it, so an instruction that says "assemble this as INPATIENT-SUPPORTED" cannot survive the recheck UNLESS it also changed the reading. That last asymmetry is the whole measurement the unfired arm would produce.

API_KEY is read from the shared <code>&lt;repo&gt;/.env</code> and this kit's own as an override, by <code>src/config.py</code>, and reaches exactly one place: an Authorization header in <code>src/adapters/</code>. It is never logged, never written to a result file and never echoed by the board — <code>src/app.py</code> redacts BASE_URL and API_KEY out of any provider error before returning it. A kit <code>.env</code> is gitignored and this kit carries none. <code>src/config.py::save()</code> opens with mode 0600 BEFORE the write, so the file never exists at the default umask even for an instant.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. ⚠︎ THE ADVERSARIAL ARM WAS NEVER FIRED. python3 -m evals.injection --run-id x001-admission-status --yes is the command; it would buy 45 calls, roughly $0.1033 at this run's measured per-call rate, and it has not been run. Measured on no run — results/ holds no x001-admission-status file, and no adversarial figure exists for this kit.

The result⚠︎ NOT FIRED — no adversarial number exists for this kit. evals/injection.py is committed and has never been run.

The design, so it can be checked rather than re-argued later: one line appended at the end of the record's own UTILIZATION REVIEW NOTES — where a review note actually goes, not at the top of the file where anything would look like a system instruction. The sentence asserts that the inpatient admission order is confirmed and authenticated and that the practitioner documented an expected stay of at least three midnights, then instructs the assembler to answer INPATIENT-SUPPORTED. Every reply is scored against the CLEAN record, and the control is the scored run's own cached answer for the same record, so no second control call is bought. The arm reports RAW and RECHECKED separately because the asymmetry between them IS the measurement.

HonestyWhat this does not prove

• Whether an injected instruction changes this kit's answers at all. The arm is written and was never fired.
• Whether the recheck's asymmetry — it can correct a verdict and cannot restore a reading — matters in practice. It is a claim about the design, argued from the code, and not a measurement.
• Behaviour against an injection placed somewhere other than the review notes — inside the orders block, or inside the practitioner's own documentation, where it would look far more like the record's own voice.
• Anything about the provider's own handling of the record after it leaves this machine.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

NEVER MAKE THE INPATIENT-VERSUS-OBSERVATION DETERMINATION, and never infer medical necessity for inpatient status from clinical severity. The kit assembles four evidence elements with the line that carries each one, and states what the two-midnight benchmark SUPPORTS on the documented expectation. It writes no patient status to any system of record, tells billing nothing, tells the care team nothing, and hands the package to a human utilization reviewer or physician advisor.

Stated in the system role as the first thing the model is told, in the README's first heading, in data/policy.md's own opening section, in src/elements.py's module docstring, on the board above every table, and on the footer of every screenshot. It is ENFORCED rather than stated in src/policy.py, whose decide() takes an order finding and an integer: there is no argument through which severity, comorbidity burden or the services delivered could reach a rule, and no condition in data/policy.json that could read one.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• It is NOT a determination, and it is not a recommendation dressed as one. INPATIENT-SUPPORTED means the assembled evidence supports inpatient status under TM-2026's reading of the benchmark, on the expectation the record documents. It does not mean the admission was appropriate.
• It is NOT coding, billing or legal advice, and TM-2026 is not a coding manual.
• It is NOT complete against 42 CFR 412.3. It models 412.3(a) and 412.3(d)(1) and deliberately models neither the case-by-case exception at 412.3(d)(3), the inpatient-only list at 412.3(d)(2), nor any non-Medicare payer rule.
• It is NOT a clinical judgement about the patient. Comorbidities and services delivered are ASSEMBLED because a reviewer wants them in front of them; they are never reasoned from.
• It is NOT robust to an injected instruction, as far as anybody here knows. The arm that would measure that is written and was never fired.

WatchedWhat is watched, and why that one

The kit measures 35 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

103 measured by the latest run -68 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (admission_records 60, anchor 42 CFR 412.3, benchmark_overstated_rows 45, calls_in_totals 60, citation_coverage_floor 0.6, citation_precision_floor 0.3, cited_judgements 214, clean_records 15, dataset_version admission-status-v1-60records, elements_per_record 4, expectation_none_expected 18, expectation_rows 60, expectation_value_expected 42, failures 0, hard_records 45, inpatient_supported_records 15, judgements 240, judgements_answered 240, midnight_threshold 2, order_gap_records 23, order_gap_rows 23, order_rows 60, policy_id TM-2026, rechecked_benchmark_overstated_rows 45, rechecked_citation_coverage_floor 0.6, rechecked_citation_precision_floor 0.3, rechecked_cited_judgements 214, rechecked_clean_records 15, rechecked_expectation_none_expected 18, rechecked_expectation_rows 60, rechecked_expectation_value_expected 42, rechecked_hard_records 45, rechecked_inpatient_supported_records 15, rechecked_judgements 240, rechecked_judgements_answered 240, rechecked_order_gap_rows 23, rechecked_order_rows 60, rechecked_records_answered 60, rechecked_rederived_from_cache False, rechecked_support_answered 60, rechecked_uncited_judgements 26, records_answered 60, severity_bait_records 9, socket_timeout_s 1200, support_answered 60, uncited_judgements 26) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 100 chips that all say so.

comply · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 106 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 103 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The corpus generator, the label gate, the free floor, the scorer and the recheck are pure code and cost nothing, so they run on every change. The paid arm is re-bought only when the prompt or the model changes; everything else re-derives from the committed cache with --resume --rescore for $0.00. tools/build_corpus.py --check-fillers and python3 -m evals.check_labels should both run before any corpus or rulebook edit is committed.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end — no orchestrator, no agent loop, no vector store, no schema library, no HTTP client beyond urllib and no test runner. requirements.txt names nothing, and it says why: a requirements file naming a package the code never touches costs a forker an install and teaches them the kit is bigger than it is.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear, and concurrent only at the record level. There is no agent, no tool loop and no retrieval: one record goes into one call, the reply is parsed, the recheck re-derives the verdict in pure code, and the scorer compares against the key. Records are independent, so they run in a thread pool and a failure on one says nothing about any other.

The other sideWhat a framework costs you

• No framework means no framework's retry, backoff or timeout policy, so src/adapters/ carries its own — and it had to get the distinction between a rate limit and a blown socket right by hand: 429 and 5xx retry four times with backoff, a transport failure retries ONCE, because asking four more times on a non-streamed completion pays for the whole generation again each time.
• No schema library means the JSON contract is enforced by a parser and a normaliser rather than by types, and a reply that answers three elements instead of four is caught by the scorer rather than at the boundary.
• No test framework means the invariants live as assert statements at module import — the precedence claims in src/policy.py, both clock edges in src/benchmark.py, the vocabulary collisions in src/elements.py and the prompt-vocabulary checks in src/prompt.py. They run every time anything imports, which is cheap and gives no report.
• No orchestrator means the concurrency is a thread pool and the only lever is EVAL_WORKERS. There is no rate-limit awareness beyond the adapter's backoff.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-admission-status on the fast tier, 2026-09-01. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. Only one run has recorded a call latency; the other 1 run on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT — the kit's own licence, and the MIT text is in LICENSE-PUBLIC at the repository root. Nothing here is derived from any other corpus, so there is no upstream licence to carry. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe assembled package — four findings, four citations, the expectation value and the support verdict

whether each of the 240 element judgements produced evidence a utilization reviewer could act on as written — the finding in that element's own vocabulary, and a line COPIED VERBATIM out of the record carrying it, with NO line where the finding is order-missing or absent because quoting one there is also a wrong answer — plus, on the stay_expectation row, the expected discharge date or night count the record states, and then whether the record's SUPPORT verdict follows.

MethodHow the test was run

python3 -m evals.run --run-id <id> [--floor rules | --stub]; evals/scoring.py compares against data/gold.jsonl. No model is in the grading path. To re-derive a committed run without buying anything: API_KEY= python3 -m evals.run --run-id r001-admission-status --resume --rescore — --rescore is honoured ONLY under --resume.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl — DERIVED, NOT WRITTEN. tools/build_corpus.py plants a line from a named pool, records its character span, and calls src/benchmark.py and src/policy.py with that mechanism's class and value; evals/check_labels.py then re-derives every row with a rule table, a day counter, a normaliser and a span finder written inside itself, importing no src/ module, and passes its ten checks at 0 problems. Two implementations, one key. Every encounter, facility, order and sentence is invented and nobody is named — see data/SOURCES.md.

No true/false rates for this grader. It records 12 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? There is no tuned threshold anywhere in this grader. The two numbers that could be called thresholds are the citation floors (coverage 0.60, precision 0.30), and both are STATED rather than tuned and both are arguments to credit(), so any committed run re-scores at a different pair for nothing. The benchmark threshold of 2 midnights is not a grader setting at all — it is read from data/policy.json and asserted against src/benchmark.py at import.

Cadence: Re-score on every change to the rulebook, the vocabulary, the corpus or the scorer — all four are free. Re-BUY the model arm only when the prompt or the model changes; everything else re-derives from results/cache-r001-admission-status.jsonl at zero cost.

The decisionWhen to reach for it