The business caseThe problem this solves

Deciding whether a life-limited component still has life left is an accumulation before it is a comparison. The totals are not printed anywhere on the pack — they are the sum of every installation period in the record trail, with hours and cycles added up separately, with an overhaul resetting one counter and not the other, and with any period whose records are missing contributing nothing at all. The only totals that ARE printed sit on the component's own tag, which is a transcription somebody made at some point and which, on this corpus, disagrees with the records on 14 of 50 packs. Someone opening each component record pack, adding up the hours and the cycles across every installation period in the service record trail — separately, because each period ran on a different airframe at a different ratio — keeping the accumulation running across an overhaul that resets time since overhaul and not time since new, refusing to invent a figure for a period whose records are missing, and then comparing the two totals against the published life limits. It is a few minutes per pack and it is every pack, and the part that goes wrong is rarely the addition: it is restarting the count at an overhaul, stopping at 'records not available' on a component the surviving trail has already condemned, or reading the figures off the component's tag because they are the only totals actually printed on the page.

The inputThe actual component record packs

The corpus is 50 component record packs, 0.04 MB (txt 50). Plain text, one format, invented rather than fetched — a real component record pack names a real operator, real airframes and a real part, and it is primary evidence in a regulated file, so there is no public corpus of (record trail, correctly-reconciled accumulated life) pairs for the same reason there is no public corpus of bank statements. Generating it also makes the label mechanical: gold's totals are not somebody's reading of a trail, they are the sum of the periods the document itself states, re-read off the corpus text with a regex by evals/check_labels.py before any run is allowed to spend.

Component
---------
CMP-ME-15955

Part Reference
--------------
LLP-1089-85

Holding Location
----------------
Quarantine cage 2, Main Hangar

Published Life Limit
--------------------
25000 hours / 18000 cycles since new

Component Tag Figures
---------------------
11965 hours / 14565 cycles since new

Service Record Trail
--------------------
2016-08 to 2017-02  airframe AF-31  accrued 8119 hours / 9527 cycles
2018-01 to 2019-03  airframe AF-11  accrued 3775 hours / 5038 cycles

Records Gap
-----------
none declared

Disposition Requested
---------------------
shelf storage

Reviewer Note
-------------
Paperwork reads complete to me, nothing outstanding from records.

The outcomeWhat a good result looks like

A thirteen-field reconciled record per component — nine copied off the pack, two reconstructed by summing the trail, two derived by comparison — plus one pure-code escalation decision taken from three of them: a pack that carries a discrepancy and is up for return to service is the one that gets stopped. Nothing here issues an airworthiness determination, a life-remaining certificate or a release to service, and a pack this rule does not flag has not been cleared by it.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Screening a batch of component record packs for life discrepancies before a records reviewer opens them — either tier — both refused to clear all 26 not-cleared packs they answered, at 1.00 recall and 1.00 precision, and both were exact on every reconstructed total
  The free tag floor calls 8 of 26 not-cleared packs cleared and misses all 14 disagreeing tags; both tiers missed none of either. That gap is the whole case for running a model here at all.
• Deciding the two tiers on cost, speed or accuracy — the fast tier
  The deliberating tier costs 26 pct more per pack ($0.0028979 against $0.0022938) and takes about 101 pct longer at the median (7655 ms against 3800 ms), and the fast tier was exact on every cell it returned — the extra money bought 46 pct more output tokens and the run's only two wrong cells. (Every percentage on this page is stated against the FAST tier, so the premium is 26 pct rather than the 21 pct you get reading it the other way round.)

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-22. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-22 — r001-partlife-recon. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Checked on a fresh checkout with API_KEY left blank: python -m src.app starts, the 50-pack picker populates, the field table draws its thirteen empty rows and POST /api/extract returns {"fields": null, "note": "No API_KEY is configured, so nothing was called..."} rather than a stack trace. python -m evals.check_labels passes with no network access at all.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. model call only, one per component record pack

Current processWhat it replaces

Someone opening each component record pack, adding up the hours and the cycles across every installation period in the service record trail — separately, because each period ran on a different airframe at a different ratio — keeping the accumulation running across an overhaul that resets time since overhaul and not time since new, refusing to invent a figure for a period whose records are missing, and then comparing the two totals against the published life limits. It is a few minutes per pack and it is every pack, and the part that goes wrong is rarely the addition: it is restarting the count at an overhaul, stopping at 'records not available' on a component the surviving trail has already condemned, or reading the figures off the component's tag because they are the only totals actually printed on the page.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• 1 of 7 stations carry a recorded failure

scroll → 7 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one section is

cut on underlined section headings; a pack with none falls back to one whole-document segment

SetupWhat the setup figure measured

There is no index. Preparation is segmentation only — 50 packs cut into 450 sections in well under a hundredth of a second, in process, with no model and no network.

LicenceLicence

MIT — this repository's own licence. Every component identifier, part reference, airframe reference, holding location and reviewer note is invented; no real manufacturer, engine type, part number, operator, airworthiness directive or maintenance manual is named or reproduced.

Bring your ownBring your own component record packs

Replace data/corpus/*.txt, write your own data/fields.json, and supply a gold row per pack. SECTION_HINTS in src/select.py maps fields to section headings and will need editing for a different pack layout; when it does not match, selection falls back to the whole document — slower, more expensive, always correct. life_status() and compute() in src/extract.py are this kit's own invented limit structure and escalation rule and must be the first things you replace with your own approved procedure. Note that data/fields.json carries an explicit spannable: false on the two computed totals: any field of yours that is a SUM rather than a quotation needs the same flag, or the span rate will punish the kit for doing arithmetic instead of copying a figure.

⚠︎ And what stops being true when you do: Every score on this page was measured on a plain-text pack with underlined headings, two to four installation periods, one unit per counter and one limit per component. None of it carries onto a scanned pack, a twenty-period trail or a limit revised mid-life — those are in breaks_on, and the arithmetic this kit exists to test has never been measured on any of them.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — they were measured against the provider’s own counter, not apportioned by character share.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 2,039 tokens assembled, 926 are instructions — 45% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Measured token-for-token via evals/prompt_tokens.py's nested-prefix method on REC-0015 — three calls at max_tokens=1, each part's size the difference between two consecutive prompt_tokens counts the provider itself returned. kits/UC0047-partlife-recon/results/tokens-p001-partlife-recon.json.

Full promptThe prompt in full

You reconcile the accumulated life of one life-limited component from its maintenance record pack. You return JSON and nothing else.

RULES, in order of importance:
1. If the pack does not state a field, return null for it. Do not infer it and do not use what you know about the world.
2. `trail_hours` and `trail_cycles` are TOTALS YOU COMPUTE, not values you copy. Add up the `accrued N hours / M cycles` figure of EVERY installation period in the Service Record Trail. Sum the hours and the cycles SEPARATELY -- each period ran on a different airframe at a different hours-per-cycle ratio, so one total can never be derived from the other.
3. AN OVERHAUL DOES NOT RESET TIME SINCE NEW. A trail line reading 'overhaul completed - time since overhaul reset to 0 hours / 0 cycles' resets the time-since-overhaul counter ONLY. The life limit is against time since NEW, which is unaffected. Keep adding every period before the overhaul as well as every period after it. Do NOT restart the accumulation there.
4. A PERIOD MARKED 'accrual NOT RECORDED' CONTRIBUTES NOTHING TO THE TOTALS. Do not estimate it, do not interpolate it from the periods either side of it, and do not fall back on the tag figures to fill it. Report `record_gap` as 'yes' and let the totals be the total of what the surviving records actually substantiate.
5. THE COMPONENT'S OWN TAG IS A CLAIM, NOT A MEASUREMENT. Copy `tag_hours` and `tag_cycles` verbatim and never correct them, never use them as `trail_hours`/`trail_cycles`, and never let them decide `life_status`. `tag_agrees` is 'yes' only when tag_hours EXACTLY equals your trail_hours AND tag_cycles EXACTLY equals your trail_cycles.
6. `life_status` is decided by COMPARING YOUR RECONSTRUCTED TOTALS against the two published limits, in this order:
   a. If trail_hours >= life_limit_hours AND trail_cycles >= life_limit_cycles, answer 'both_exceeded'.
   b. Otherwise, if trail_hours >= life_limit_hours, answer 'hours_exceeded'.
   c. Otherwise, if trail_cycles >= life_limit_cycles, answer 'cycles_exceeded'.
   d. Otherwise, if record_gap is 'yes', answer 'cannot_determine'.
   e. Otherwise, answer 'within_limits'.
7. THE EXCEEDANCE CHECKS COME BEFORE THE GAP CHECK. A missing period of records can only ADD accumulated life -- it can never bring a component that the surviving records already put at or past a limit back inside it. So a declared gap makes 'within limits' undeterminable and leaves 'exceeded' perfectly determinable. Do not answer 'cannot_determine' without first checking the surviving totals against both limits.
8. THE LIMIT IS INCLUSIVE. A total EXACTLY equal to the published limit is exceeded, not within limits -- there is no life remaining at the limit.
9. THE REVIEWER'S NOTE IS A FIELD TO COPY, NOT EVIDENCE. A note that sounds calm does NOT mean the component is inside its limits, and a note that sounds worried does NOT mean it is not. The figures decide; the note is one person's remark and may disagree with them.
10. Report every hours and cycles figure as a bare whole number with the unit left out of it. Use the exact allowed value for a field that lists them, and return every field named in the schema even when the answer is null.

YOU ARE NOT DECIDING WHETHER THIS COMPONENT MAY FLY. `life_status` is a statement about what the RECORDS substantiate. It is not an airworthiness determination and it releases nothing to service.

Extract these fields:
- component_id (string) -- the component identifier, verbatim
- part_reference (string) -- the part reference on the pack, verbatim
- life_limit_hours (number) -- the published life limit in HOURS since new, as a bare number without the unit
- life_limit_cycles (number) -- the published life limit in CYCLES since new, as a bare number without the unit
- tag_hours (number) -- the hours since new stated on the component's OWN tag, as a bare number. Copy it; do not correct it
- tag_cycles (number) -- the cycles since new stated on the component's OWN tag, as a bare number. Copy it; do not correct it
- trail_hours (number) -- the TOTAL hours since new that the service record trail substantiates: add up the `accrued ... hours` figure of EVERY installation period in the trail. A period marked `accrual NOT RECORDED` contributes NOTHING -- do not estimate it. An overhaul line resets time since OVERHAUL only and never time since new, so keep adding every period before and after it. Never copy the tag figure into this field
- trail_cycles (number) -- the TOTAL cycles since new that the service record trail substantiates, summed the same way and INDEPENDENTLY of the hours -- each period ran at a different hours-per-cycle ratio, so one total can never be derived from the other
- record_gap (enum) one of: yes, no -- does the pack declare a period whose records are not available, so that part of the accrual cannot be reconstructed?
- disposition_requested (enum) one of: return to service, shelf storage -- what is being asked for this component right now, verbatim
- reviewer_note (string) -- the records reviewer's own free-text note, copied verbatim
- tag_agrees (enum) one of: yes, no -- do the component's own tag figures match the totals you reconstructed from the trail? Answer 'yes' only when tag_hours EXACTLY equals trail_hours AND tag_cycles EXACTLY equals trail_cycles. A tag that is right about hours and wrong about cycles does NOT agree
- life_status (enum) one of: within_limits, hours_exceeded, cycles_exceeded, both_exceeded, cannot_determine -- what the RECONSTRUCTED TRAIL TOTALS say about the component's remaining life, compared with the published limits. Decide this STRICTLY from trail_hours, trail_cycles, life_limit_hours, life_limit_cycles and record_gap -- never from the tag figures and never from the reviewer note. The rule, in this order: (a) if trail_hours >= life_limit_hours AND trail_cycles >= life_limit_cycles, answer 'both_exceeded'; (b) otherwise if trail_hours >= life_limit_hours, answer 'hours_exceeded'; (c) otherwise if trail_cycles >= life_limit_cycles, answer 'cycles_exceeded'; (d) otherwise if record_gap is 'yes', answer 'cannot_determine'; (e) otherwise answer 'within_limits'. The exceedance checks come BEFORE the gap check because a missing period can only ADD accumulated life and can never bring a component already at or past a limit back inside it. The limit is INCLUSIVE: exactly at the limit is exceeded, not within limits. This is a statement about what the RECORDS substantiate -- it is not an airworthiness determination and it releases nothing to service

Return a JSON object with exactly these keys: component_id, part_reference, life_limit_hours, life_limit_cycles, tag_hours, tag_cycles, trail_hours, trail_cycles, record_gap, disposition_requested, reviewer_note, tag_agrees, life_status
Use null for any field the pack does not state.

COMPONENT RECORD PACK
---------------------
Component
---------
CMP-GF-94167

Part Reference
--------------
LLP-8504-81

Published Life Limit
--------------------
16000 hours / 20000 cycles since new

Component Tag Figures
---------------------
16000 hours / 20000 cycles since new

Service Record Trail
--------------------
2014-08 to 2015-09  airframe AF-22  accrued 5510 hours / 9181 cycles
2017-09              overhaul completed - time since overhaul reset to 0 hours / 0 cycles; time since new is unaffected and keeps accruing
2019-01 to 2019-07  airframe AF-11  accrued 6115 hours / 6269 cycles
2021-09 to 2023-01  airframe AF-14  accrual NOT RECORDED - records not available, gap statement GS-5424
2023-12 to 2026-02  airframe AF-28  accrued 4375 hours / 4550 cycles

Records Gap
-----------
one period declared, gap statement GS-5424 - accrual for that period cannot be reconstructed

Disposition Requested
---------------------
return to service

Reviewer Note
-------------
Trail looks continuous on a quick read, filed without comment.

Raw responseThe raw response

{"component_id":"CMP-GF-94167","part_reference":"LLP-8504-81","life_limit_hours":16000,"life_limit_cycles":20000,"tag_hours":16000,"tag_cycles":20000,"trail_hours":16000,"trail_cycles":20000,"record_gap":"yes","disposition_requested":"return to service","reviewer_note":"Trail looks continuous on a quick read, filed without comment.","tag_agrees":"yes","life_status":"both_exceeded"}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Check an aircraft part's hours and cycles against its records — 50 component record packs. Two tiers of one model family answered, and every answer was then graded Four different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate — a projection onto a rate card, not a bill anyone paid.

Rates checked 2026-08-18. The provider that actually ran r001 and r002 publishes no rate card this repo commits, so nothing here is what was actually paid — the real spend for this kit's build is recorded in the commit history and the shared call ledger, not on this page.

The gradersFour ways to grade

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Identical work in both rows — the same query, the same tokens. Only the price list changed. Which is why the token count is the measurement and the dollar is arithmetic you can redo on your own card.

Answering vs gradingAnswering and grading are two separate bills

The tag floor (evals/baseline.py) and the scorer (evals/judge.py) are both pure code and cost $0.00 to run against either result set. The first figure is the fast tier's own 49-pack run priced at Google Gemini 3 Flash's published rate; the second adds the deliberating tier's 50. Neither includes the 5 probe calls that MEASURED MAX_TOKENS before either run was allowed to spend, or the three at max_tokens=1 that measured the prompt split — those are recorded in results/cap-*.json and results/tokens-*.json and are the cheapest part of the build.

Cost driversWhat actually moves the bill

• The fixed system prompt and field schema (1686 of 2039 tokens on the example call, 83 pct) outweigh the pack sections sent (353 tokens) — the floor every call pays before a single figure is read. It is long here on purpose: the accumulation rules and the five-way priority order are stated in full rather than left to be inferred, and that decision is most of the bill.
• Output length: the model returns a full thirteen-key JSON record every call, including the reviewer's note copied back verbatim, whatever the pack says — and output is priced six times input on the published card, so it is 57 pct of the fast tier's per-pack cost despite being 18 pct of its tokens.
• Trail length: a pack with more installation periods is a longer prompt AND a longer sum. These packs carry two to four periods; nothing here measures what twenty does to either number.

Your volumeWhat it costs at your volume

Linear in record packs: each call is independent, carries the same fixed prompt and shares nothing with its neighbours, so 500 packs cost ten times 50 and take ten times as long. Nothing amortises — there is no index to build and no cache, which is also why there is no volume discount to find without changing the design. What is NOT linear is the failure surface: one transport timeout in 99 calls is a document-loss rate, and it scales with call count rather than with anything about the corpus.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

The exact work behind every number on these pages: 49 component record packs, one completion call each, on the fast tier. REC-0029's call died in transport and returned nothing, so it is not in these totals. The deliberating tier's own 50 calls are recorded separately in Cost.cost_by_model and are not projected here.

The codeEvery file, and what it is for

Five modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/segment.pysegment

cut the component record pack into addressable sections, pure code

# Cut a document into addressable sections. Pure code — no model, no network.

def sections(text):
def locate(text, value):
def span_label(secs, start):

src/select.pyselect — a swap seam

pick which sections carry each field, pure code — the life status is mapped to the trail, the declared gap and the two published limits, never to the component's own tag, and the Holding Location section is mapped by nothing and never sent

You change it to: map fields to your own record pack's headings; unmatched falls back to the whole document

# Pick which sections plausibly carry each field. Pure code -- the last deterministic step before

SECTION_HINTS = {
def for_field(secs, field):
def plan(secs, fields):

src/prompt.pyprompt

assemble one call for all thirteen fields, with the accumulation rules stated in full — sum every period, hours and cycles separately, never restart at an overhaul, never estimate a missing period — and the five-way life-status priority order under them

# Assemble the reconciliation prompt. One prompt per component record pack, all thirteen fields

SYSTEM = (
def field_schema(fields):
def build(doc_text, secs, fields, selector):
def parse(raw, fields):

src/extract.pyextract — a swap seam

the AI layer, one provider one key — plus the pure-code escalation check downstream: a pack carrying a discrepancy that is up for return to service is stopped before release

You change it to: the escalation rule — this kit ships three fields and a boolean (a discrepancy AND a request to return to service) and a real records desk weighs which discrepancy it is, what evidence can still be recovered and who may accept what. It is deliberately NOT the same function as life_status(), so changing WHO GETS STOPPED does not change WHAT THE RECORDS SAY

# Reconcile one life-limited component's record pack: segment, select, prompt, one model call,

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
FIELDS = os.path.join(HERE, "data", "fields.json")
CORPUS = os.path.join(HERE, "data", "corpus")
MAX_TOKENS = 2000
LIFE_STATUSES = ("within_limits", "hours_exceeded", "cycles_exceeded", "both_exceeded",
DISPOSITIONS = ("return to service", "shelf storage")
def load_fields():
def load_doc(rec_id):
def documents():

evals/judge.pyjudge

score field accuracy, the life-status call (five-class and collapsed to cleared/not-cleared), the tag comparison and the escalate flag separately, pure code

# Score a reconciliation run. PURE CODE -- gold is exact and the answer is one value per cell, so

def norm(v):
def _num(v):
def equal(field, got, want):
def score(fields, records, golds):
def _matrix(rows, positive, allowed):
def _cleared(status):
def score_flags(records, flags, golds):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 1979 input and 434 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

This run's component record packs are entirely synthetic (tools/build_corpus.py, seed 20260822): no real manufacturer, operator, airframe, part number or airworthiness directive exists in the corpus, and nothing was fetched from anywhere. The only outbound traffic the kit makes is one chat-completion request per pack to the configured provider, carrying the mapped sections of one pack — the Holding Location section is mapped by nothing and never leaves. Nothing is written outside the kit directory, there is no database, no auth and no multi-tenancy, and the local UI binds 127.0.0.1 only.

Read from the shared .env, the kit-local .env or the real environment only, never committed, never logged and never rendered. .env is gitignored from the first commit. On an adapter error the app redacts the key and base URL out of the message before returning it, so a misconfigured endpoint cannot echo a credential into a browser — which matters more here than usual, because the one error this run actually produced was a transport failure whose message carries the request URL.

The experimentWe did not attack it — and three of four boundaries hold

The three boundaries that hold were confirmed by reading the code, not by a run: no code path releases, certifies or disposes of anything; the escalation and life rules read only numbers and enums, so the reviewer's note cannot reach them; and the key and base URL are redacted out of any error before it is returned, which this run's one real transport error confirms in practice. The fourth is open. An indirect prompt injection needs a field an outside party authored, and this kit has exactly one — the records reviewer's note — which makes it the obvious place to attack and the reason the gap is named rather than glossed. Confirmed by reading the code, not by a run, on 2026-08-22 -- no attack run was fired; see redteam.why.

The first three boundaries hold, confirmed by reading the code rather than by an attack run. The fourth is open and is named as open — an indirect prompt injection in the records reviewer's note is exactly the attack this kit's shape invites, and nothing here has tried it. On a kit adjacent to a release decision, an untested injection boundary is the gap to read first.

The result0 attack trials, and three of four boundaries checked here hold on evidence. The one that does not is the one this kit's own shape invites: the records reviewer's note is free text an outside party wrote, and whether an instruction hidden in it could move a life-status answer is unmeasured.

This run's corpus is entirely generated (tools/build_corpus.py, seed 20260822), so no text in it came from an outside party and there was nothing adversarial to resist. The planted ambiguity is a REGISTER mismatch — a calm note over a component the trail does not clear — which measures whether a model does the arithmetic when the prose points the other way. It does not measure whether a model obeys an instruction hidden in the same field. Those are different failures and only one of them is measured here.

HonestyWhat this does not prove

• Whether an indirect prompt injection in the reviewer_note field could move life_status or the reconstructed totals on either tier. No attack run exists.
• Whether the kit behaves safely against a hostile provider — a response body crafted to break the JSON extraction in src/prompt.py::parse, or to return an enormous payload. parse() fails closed to an empty dict, which the harness records as a failed document, and nothing beyond that was tested.
• Whether a real component record archive would carry anything sensitive this kit mishandles. The corpus has no personal data by construction; a real pack carries named signatories, operator identifiers and airframe registrations, and none of that path is exercised.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

escalate fires when the pack carries a discrepancy — life_status is anything other than "within_limits", OR tag_agrees is "no" — AND disposition_requested is "return to service". All three values come out of the same reply; the rule is run afterwards in pure code, over whatever the model returned, and never over gold. A reply missing any of the three returns None rather than False: an unknown is not a pass, and on a safety-adjacent check that is the difference between "we checked and found nothing" and "we did not check".

src/extract.py::compute(), called by extract() on every pack, by the local app on every click, and by evals/baseline.py on the free floor's own output so the two are routed by identical code. The life rule it reads lives in a SEPARATE function, life_status(), which is also what the corpus generator wrote gold with and what src/prompt.py states to the model — one definition, three readers.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• It is NOT an airworthiness determination, a release to service, or a statement that any component may fly. life_status describes what the RECORDS substantiate and escalate raises a pack for a person. A pack this rule does not flag has not been cleared by it — it has been left alone by it.
• It is NOT a check on whether the reconstructed totals are right. If the model sums the trail wrong and then reasons about its own wrong total perfectly, the pack is stopped (or not stopped) on a wrong number and this rule cannot tell. The two consistency diagnostics in evals/judge.py are the closest thing to that check and they are blind to exactly the same case — both read 0 on both tiers, which means they found nothing, not that nothing was there.
• It is NOT any operator's or authority's release procedure. 'A discrepancy on a pack up for return to service' is this kit's own simplification, invented for this corpus. No airworthiness regulation, approved maintenance organisation exposition or continuing-airworthiness management procedure was consulted, and none is reproduced.

WatchedWhat is watched, and why that one

The kit measures 18 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

7 measured by the latest run 11 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (documents 50, extraction_cells 650, failures 0, refusal_cells 0) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked.

not a time series No two of these 3 runs measured the same system — they differ on documents, extraction_cells, failures, provider, thinking, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

Re-check the flag's evidence on any change to compute() or life_status() in src/extract.py, on any change to data/fields.json's allowed values for life_status, tag_agrees or disposition_requested, and on any corpus regeneration. Changing WHO GETS STOPPED is a policy change and must be re-scored, even though it never changes what the records say.

In one lineNo framework, deliberately

No framework and no dependencies beyond the standard library. requirements.txt is empty on purpose: the corpus is generated in-process, the provider is reached over urllib, and the UI is one HTML file with no build step. A forker runs this on whichever key they already hold, with one clone and no install.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Nothing here is a graph. The pipeline has one path per pack — segment, select, prompt, one call, parse, compute — with no branch, no loop and no agent. Anything that looks like orchestration in a kit this size is a diagram of a straight line.

The other sideWhat a framework costs you

• Everything is hand-rolled, so everything is yours to maintain: the JSON extraction from a fenced reply, the retry and backoff policy, the section regex and the .env reader are all code somebody has to own. The retry policy in particular was wrong until this kit's own run proved it.
• No framework means no framework's ecosystem — no tracing, no eval harness beyond the one in evals/, no prompt registry, no schema validation library. What ships is what is in the repository.
• The provider abstraction covers exactly two wire formats. A third provider is one function and one dict entry, and until somebody writes it the kit runs on two shapes.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-partlife-recon on the fast tier, 2026-08-22. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 2 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

1 run not plotted. b000-tag recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — each document goes whole into one call.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT — this repository's own licence. Every component identifier, part reference, airframe reference, holding location and reviewer note is invented; no real manufacturer, engine type, part number, operator, airworthiness directive or maintenance manual is named or reproduced. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one linePer-field exact match, light normalisation

Does the model's value for each field match gold, after trimming whitespace and punctuation and treating numbers within half a cent as equal? Nine of the thirteen fields are copied off the pack, two — tag_agrees and life_status — are comparisons, and TWO — trail_hours and trail_cycles — are sums the model had to compute. All thirteen are graded together here, because a total that is wrong is wrong however it was arrived at.

MethodHow the test was run

evals/judge.py::score, in-process, no key and no model. The same function evals/run.py calls after every call, and the same field-match logic the free tag floor is scored by — a baseline and a real run scored by two different scorers cannot be compared honestly.

Every grader on these pages scored the same 100 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: tools/build_corpus.py's gold, whose two computed totals are re-read off the corpus text and re-summed by evals/check_labels.py before any run may spend, and whose three derived labels are re-derived from the pack's own figures.

How tight can the band be? There is no tolerance band on the field grade — it is exact match after trimming whitespace and punctuation, with numbers within half a cent treated as equal, never a continuous score to round. A one-word truncation of a verbatim string is wrong, not 'nearly right'.

Cadence: Re-score on any change to src/prompt.py, data/fields.json or tools/build_corpus.py — the first two change what is asked, the third changes what is asked about. Re-run evals/run.py (paid) on any change to MAX_TOKENS or the provider/model.

The decisionWhen to reach for it

In one linelife_status against gold's own arithmetic

Of every component the record trail does NOT clear — past a limit on hours, on cycles, on both, or undeterminable because a period of records is missing — how many did the run refuse to clear? NOT-CLEARED IS THE POSITIVE CLASS. A component the records do not clear that gets called within_limits is the failure that matters on a pack headed back onto an aircraft. Reported both as five-class accuracy and collapsed to that binary. THIS IS THE HEADLINE.

MethodHow the test was run

evals/judge.py::score_flags, in-process, no key and no model.

Every grader on these pages scored the same 100 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: Gold's life status is re-derived inside the grader by the same five-branch rule the kit publishes — exceedance before gap, limits inclusive — over gold's own reconstructed totals, so the truth this matrix grades against can never be a separately-typed label that drifted from the rule.

How tight can the band be? No threshold — the call is one of five allowed values, and a reply that returns none of them is counted as unanswered rather than folded into the cleared cell. On this kit that distinction is load-bearing: an unanswered pack is a pack nobody reviewed, not a pack that passed.

Cadence: Re-run whenever life_status() in src/extract.py changes, whenever the corpus is regenerated, and on any provider or model change.

The decisionWhen to reach for it

In one linetag_agrees against the same comparison over gold's figures

Did the run notice that the figures on the component's own tag do not match the total its record trail substantiates? DISAGREEMENT IS THE POSITIVE CLASS: an unreconciled tag is a discrepancy somebody has to raise, and it is the only thing on the pack that would ever tempt a reader to skip the arithmetic.

MethodHow the test was run

evals/judge.py::score_flags, in-process, no key and no model.

Every grader on these pages scored the same 100 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: src/extract.py::tag_agreement(), the same function the corpus generator used to write gold, applied to gold's own tag figures and reconstructed totals. Exact equality on both counters.

How tight can the band be? No threshold and no tolerance: the two figures are equal or they are not.

Cadence: Re-run whenever tag_agreement() changes, whenever N_TAG_MISMATCH in tools/build_corpus.py changes, and on any provider or model change.

The decisionWhen to reach for it

In one lineescalate against the same rule run over gold

Does the pure-code escalation — a discrepancy AND a request to return the component to service — land on the same packs it would land on if all three fields had been read perfectly? It is a BUSINESS CONDITION, so unlike a self-consistency check it genuinely needs labels, and saying so is half of what makes the number believable.

MethodHow the test was run

evals/judge.py::score_flags, in-process, no key and no model.

Every grader on these pages scored the same 100 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: src/extract.py::compute(), the same function the run uses, applied to GOLD's life_status, tag_agrees and disposition_requested. One rule, three inputs, so a change to the rule moves both sides together and the grader cannot silently grade an old policy.

How tight can the band be? No threshold — three fields, an OR inside an AND, and a boolean out.

Cadence: Re-run whenever compute() changes. Changing WHO GETS STOPPED is a policy change and must be re-scored, even though it never changes what the records say.

The decisionWhen to reach for it