The business caseThe problem this solves

A cluster of warranty claims lands on one part and the plant change log offers several candidate causes. Every one of them already passed a date-and-part join -- that is how it became a candidate -- so the overlap is the one fact on the page that carries no information. Somebody reading a candidate list by hand: pulling each cluster's claim register, reading every technician narrative, working out which failure family they actually describe, checking the line and shift each vehicle was built on against what the change touched, and then reading the change record's own prose for a residual-stock statement its effectivity dates do not carry.

The inputThe actual correlation requests

The corpus is 120 correlation requests, 1.07 MB (txt 120). ⚑ THE POPULATION IS DELIBERATELY THE HARD ONE, AND SAYING SO IS THE POINT. Every reading here is a candidate a loose join ALREADY proposed -- same part, same plant, effectivity inside ninety days of the build span -- because that is how candidates reach a quality engineer's desk. Conditioned on being a candidate, the join has already said everything it can say. A random slice of a warranty book would be overwhelmingly easy and would measure the sampling rather than the task. ⚠︎ THE PROFILE MIX IS CHOSEN, NOT OBSERVED: {"clean": 19, "contained": 6, "ctx_claims": 3, "ctx_signal": 9, "inconclusive": 12, "line": 16, "stock": 8, "symptom": 26, "window": 21}. Those proportions are an editorial decision about what is worth measuring, not an estimate of any real plant's rate, and no percentage on this page transfers to a real claim population.

Synthetic Record
----------------------------------------------------------------
Every vehicle programme, plant, line, supplier, part number, change record, claim, name and
narrative in this file is INVENTED. Generated by tools/build_corpus.py from seed 20260825. No real
manufacturer, supplier, plant, vehicle, dealer or person appears anywhere in it, and no real
warranty text was read to write it. MIT licensed, like the rest of this repository.

Cluster And Request
----------------------------------------------------------------
Correlation request     : WCS-0001-C1
Cluster reference       : WCS-0001
Vehicle programme       : Marlin C1
Part number             : P-4013-02
Part description        : rear quarter seam assembly
Claims in cluster       : 6
Build dates span        : 2026-01-14 to 2026-02-02
Assembly plant          : Brackwell Assembly
Review date             : 2026-08-25
This request is         : candidate signal 1 of 3 raised on this cluster
Quality engineer of record: Tomas Erlichman
Containment horizon     : 45 days from the review date

How this candidate got here: a loose join proposed it -- same part number, same plant, and
an effectivity falling within ninety days of this cluster's build span. A candidate list is
not a finding.

Claim Register
----------------------------------------------------------------
Claim        VIN sfx  Build date  Line  Shift  Mileage  Dealer code  Symptom class
----------------------------------------------------------------------------------
WC-400137    0V5STU   2026-01-14  L2    B        2,111  ELEC-INT     INTERMITTENT_ELECTRICAL
WC-400140    PCZRYY   2026-01-17  L2    C        3,018  ELEC-INT     INTERMITTENT_ELECTRICAL
WC-400143    SC0F7V   2026-01-21  L2    A        3,925  ELEC-INT     INTERMITTENT_ELECTRICAL

The outcomeWhat a good result looks like

An attribution opinion per candidate, with the axis that decided it named.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Your warranty system already holds the failure-mode family in a structured field — the free floor -- tagged-symptom-strong, $0.00
  It reads the keyed class, applies the same rule engine, and scores 100.0 pct on the binary and 100.0 pct on the verdict over those readings for nothing.
• The failure mode exists only in the technician's narrative — the model, reading the claim narratives
  100.0 pct on the binary and 100.0 pct on the eight-way verdict over the PROSE readings, against the strongest free floor's 76.32 pct and 63.16 pct on the same cells.

And where nothing here is good enough:

• Your change records routinely carry a from-date and no to-date — neither, until you have written the rule down
  The model reads a half-open window as still running and attributes; the key calls it CONTEXT_INCOMPLETE. It happened on 5 of 5 such readings and accounts for 5 of the 5 false 8Ds on the whole run.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-25. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-25 — r001-plant-signal. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — git clone, then python3 -m src.app. No install, no key, no index build: requirements.txt is a comment explaining why it is empty. The corpus, the answer key, all three free floors and every committed run record ship in the repo.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. wall time for one reading end to end -- prompt assembly, the single completion call and parsing -- over 120 readings on r001-plant-signal, six concurrent workers.

Current processWhat it replaces

Somebody reading a candidate list by hand: pulling each cluster's claim register, reading every technician narrative, working out which failure family they actually describe, checking the line and shift each vehicle was built on against what the change touched, and then reading the change record's own prose for a residual-stock statement its effectivity dates do not carry.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureEvalsEvalsEvalsEvalsPromptPromptPromptPromptArchitectureArchitectureApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 4 of 11 stations carry a recorded failure

scroll → 11 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one reading is

Every reading is scored. There is no train split -- nothing is fitted. 40 clusters x 3 candidate signals.

SetupWhat the setup figure measured

No index and no retrieval step. One correlation request goes into the prompt whole; the only reduction is src/select.py withholding one section.

LicenceLicence

MIT

Bring your ownBring your own correlation requests

Point data/corpus/ at your own exported correlation requests in the same eight sections, or change tools/build_corpus.py's renderer to match your export. Then replace SIGNAL_TYPES and DEALER_CODES in src/signals.py -- those two tables are the domain knowledge and they are illustrative here. src/segment.py's SECTIONS is the one list that has to agree with your export, and evals/check_labels.py refuses to let a run spend until it does.

⚠︎ And what stops being true when you do: ⚠︎ WHAT YOU CANNOT BRING IS AN ANSWER KEY. src/signals.decide() computes the gold from structured facts PLUS the observed failure family -- and that family is the one input it cannot derive, because reading it out of the narratives is the task. On your own data you have to label it, and until you have, every percentage on this page is about this corpus and not about yours.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 3,104 tokens assembled, 2,184 are contexts — 70% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

⚑ THE THIRD PART IS THE EXPERIMENT AND THE ABLATION REMOVES ONE BLOCK OF IT. s001-plant-signal-registeronly is byte-identical everywhere except the Claim Narratives body, which becomes one line saying no narrative is available. check_labels asserts that every other section is identical between the two builders, so the control is ONE variable. ⚑ AND THE ABLATION IS NOT BLINDED: the claim register still carries the dealer symptom code on every row and, on half the clusters, the hand-keyed symptom class. The register-only arm is exactly as sighted as the strongest free floor, which is the comparison worth having.

Full promptThe prompt in full

You are a warranty correlation desk. You read one cluster of warranty claims and one candidate manufacturing change, against the DEFAULT correlation policy printed on the request, and you answer with one JSON object and no other text.

You are a warranty correlation desk inside one vehicle manufacturer. You are reading ONE cluster of
warranty claims on ONE part, against ONE candidate manufacturing signal, and deciding whether the
evidence attributes that cluster to that change.

The claim register, the technician narratives, the candidate change record and the DEFAULT
correlation policy are all reproduced below. Apply the policy exactly as written.

⚠︎ THE CORRELATION POLICY, THE FAILURE-MODE FAMILIES AND THE CONTAINMENT HORIZON ARE ILLUSTRATIVE
DEFAULTS, NOT ANY REAL MANUFACTURER'S TERMS. Apply them as printed regardless of whether they look
right for the part in front of you.

THE POINT OF THIS JOB: this candidate is on your desk BECAUSE the dates and the part number already
line up. A loose join proposed it. So "the dates overlap" tells you nothing you did not already know
when you opened the file, and most candidates that survive that join are still wrong:

 - the same part is fitted on more than one line and on more than one shift, so a change that ran on
   the same days can have touched none of these vehicles;
 - a change is only capable of certain failures. A seal compound makes joints weep; a torque
   specification makes fasteners come loose; a fixture locator makes panels sit wrong. If the
   technicians describe a failure this change cannot produce, the dates lining up is a coincidence;
 - and a superseded supplier batch does not stop being fitted the day the change record closes.
   Residual stock at the line side is consumed afterwards, and a change record worth reading says so
   in a sentence rather than in its effectivity dates.

READ THE TECHNICIAN NARRATIVES. They are the only place the failure actually is. The dealer symptom
code on each row was keyed at a service counter, from a short list, before anyone looked at the
vehicle -- one code covers several genuinely different failures and one of them covers none. Where a
SYMPTOM CLASS has been keyed against the claims, use it; where it reads NOT_CLASSIFIED, the prose is
your only evidence.

⚑ AND SAYING THE EVIDENCE DOES NOT CHOOSE IS A REAL ANSWER HERE, NOT A HEDGE. Where a second change
was effective over the same builds on the same line and can produce the same failure, naming one of
them is worse than saying so. That verdict is EVIDENCE_INCONCLUSIVE and the policy tells you exactly
when it applies. It is NOT a way out of a reading you find hard: use it only when rule R-7 fires.

Answer with a single JSON object and nothing else:

{"verdict": "ATTRIBUTED_SUPPLIER_LOT|ATTRIBUTED_PROCESS_CHANGE|ATTRIBUTED_SHIFT_OR_LINE|NOT_ATTRIBUTED_SYMPTOM_MISMATCH|NOT_ATTRIBUTED_BUILD_WINDOW|NOT_ATTRIBUTED_LINE_MISMATCH|EVIDENCE_INCONCLUSIVE|CONTEXT_INCOMPLETE",
 "deciding_axis": "SYMPTOM|BUILD_WINDOW|LINE|SHIFT|SUPPLIER_LOT|PROCESS_STEP|NONE",
 "claims_attributed": <integer, the number of COVERED claims per R-3 when the cluster is attributed,
   0 otherwise>,
 "signal_ref": "<the change record reference, e.g. PCR-5001, or NONE>",
 "raise_8d": "YES|NO",
 "rationale": "one sentence, naming the axis that decided it and -- where the identifiers and the
   narratives disagree -- which one you followed and why"}

Precedence, applied in this order: CONTEXT_INCOMPLETE (R-1); then NOT_ATTRIBUTED_BUILD_WINDOW (R-4);
then NOT_ATTRIBUTED_LINE_MISMATCH (R-5); then NOT_ATTRIBUTED_SYMPTOM_MISMATCH (R-6); then
EVIDENCE_INCONCLUSIVE (R-7); otherwise the attribution named by R-8.

Correlation request
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
Every vehicle programme, plant, line, supplier, part number, change record, claim, name and
narrative in this file is INVENTED. Generated by tools/build_corpus.py from seed 20260825. No real
manufacturer, supplier, plant, vehicle, dealer or person appears anywhere in it, and no real
warranty text was read to write it. MIT licensed, like the rest of this repository.

Cluster And Request
----------------------------------------------------------------
Correlation request     : WCS-0006-C1
Cluster reference       : WCS-0006
Vehicle programme       : Petrel G2
Part number             : P-4078-07
Part description        : sill trim moulding
Claims in cluster       : 6
Build dates span        : 2026-02-28 to 2026-03-24
Assembly plant          : Ashford Assembly
Review date             : 2026-08-25
This request is         : candidate signal 1 of 3 raised on this cluster
Quality engineer of record: Mei-Ling Ferreira
Containment horizon     : 45 days from the review date

How this candidate got here: a loose join proposed it -- same part number, same plant, and
an effectivity falling within ninety days of this cluster's build span. A candidate list is
not a finding.

Claim Register
----------------------------------------------------------------
Claim        VIN sfx  Build date  Line  Shift  Mileage  Dealer code  Symptom class
----------------------------------------------------------------------------------
WC-400822    8PHBVG   2026-02-28  L1    A        3,666  HW-LSE       NOT_CLASSIFIED
WC-400825    R34XHL   2026-03-04  L1    B        4,573  NOISE-DRV    NOT_CLASSIFIED
WC-400828    EM448J   2026-03-09  L1    C        5,480  HW-LSE       NOT_CLASSIFIED
WC-400831    GENHB0   2026-03-14  L1    A        6,387  HW-LSE       NOT_CLASSIFIED
WC-400834    UZ1U17   2026-03-19  L1    B        7,294  NOISE-DRV    NOT_CLASSIFIED
WC-400837    GSJES5   2026-03-24  L1    C        8,201  HW-LSE       NOT_CLASSIFIED

Dealer symptom codes are keyed at the service counter from a short list before the
vehicle is examined. Each covers more than one failure:
  HW-LSE      Hardware loose -- covers LOOSE_FASTENER, THREAD_DAMAGE
  NOISE-DRV   Noise heard while driving -- covers RATTLE_OVER_BUMPS, WIND_NOISE

Symptom class reads NOT_CLASSIFIED on every claim in this cluster: no quality
engineer has been through it and keyed one.

Claim Narratives
----------------------------------------------------------------
WC-400822  (Haldane and Rowe, 2026-02-28 at 3,666 miles)
  Knocking noise, worse when loaded. All four fixings slack, none damaged. Torqued down and rechecked, noise gone.

WC-400825  (Petronella Auto, 2026-03-04 at 4,573 miles)
  Customer says something is loose underneath. One bolt had backed out most of the way and was hanging. Refitted and torqued; checked the others and two were also slack.

WC-400828  (Larkhill Motors, 2026-03-09 at 5,480 miles)
  Clunk from the nearside over bumps. Found the lower bolt finger tight and the second one about a turn loose. Torqued to spec, road tested, gone.

WC-400831  (Tenby Vale Cars, 2026-03-14 at 6,387 miles)
  Rattle on rough surfaces. Two of the four fixings had backed off; threads look fine. Retorqued, marked, road tested clear.

WC-400834  (Ostbrook Autohaus, 2026-03-19 at 7,294 miles)
  Knocking noise, worse when loaded. All four fixings slack, none damaged. Torqued down and rechecked, noise gone.

WC-400837  (Marchfield Garage, 2026-03-24 at 8,201 miles)
  Customer says something is loose underneath. One bolt had backed out most of the way and was hanging. Refitted and torqued; checked the others and two were also slack.

Candidate Plant Signal
----------------------------------------------------------------
Signal reference        : PCR-5043
Change type             : SUPPLIER_LOT_PLATING
Change description      : Supplier lot change -- fastener plating batch
Raised against plant    : Ashford Assembly
Assembly line affected  : L1
Shift affected          : all shifts
Effective from (build)  : 2026-02-19
Effective to (build)    : 2026-03-27
Supplier of record      : Vellum Fastenings

Failure modes this change is capable of producing:
  CORROSION_BLOOM, THREAD_DAMAGE

What changed, in the change owner's words:
  Fastener plating moved to a new bath at Vellum Fastenings after the previous line was taken down for maintenance.

Containment record      : none open against this signal

Co-effective signals on these builds:
  PCR-5044  LINE_REBALANCE
      Line rebalance -- station content moved between two stations
      effective 2026-02-16 to 2026-03-28 on L1
      capable of producing: CONNECTOR_NOT_SEATED, MISSING_CLIP, LOOSE_FASTENER

Correlation Policy (Default)
----------------------------------------------------------------
CORRELATION POLICY (DEFAULT) -- ILLUSTRATIVE, NOT ANY REAL MANUFACTURER'S TERMS

R-1  If the candidate signal has no effective build dates, or its change type is not one of the
     types listed on this request, or any claim in the register is missing a build date, line or
     shift, the verdict is CONTEXT_INCOMPLETE. Nothing is attributed, claims_attributed is 0, the
     axis is NONE, the signal reference is NONE and no 8D is raised. There is no default cause for
     a cluster whose build records were never supplied.

R-2  The signal's EFFECTIVE BUILD WINDOW runs from its effective-from date to its effective-to
     date, EXTENDED to the residual-stock date where the change record states one. Stock already at
     the line side is fitted after a changeover is recorded; a window that ignores it is short.

R-3  A claim is COVERED by the signal when its build date falls inside the effective build window
     AND it was built on a line the change touched AND on a shift the change touched. A change
     recorded against every line, or every shift, covers all of them.

R-4  If NO claim in the register has a build date inside the effective build window, the verdict is
     NOT_ATTRIBUTED_BUILD_WINDOW on the BUILD_WINDOW axis. The change was not running when these
     vehicles were built.

R-5  If claims fall inside the window but NONE of them were built on a line and shift the change
     touched, the verdict is NOT_ATTRIBUTED_LINE_MISMATCH. Name the axis SHIFT where the change is
     recorded against a named shift, and LINE otherwise. The same part is fitted on more than one
     line and in more than one plant; a date that lines up is not a cause.

R-6  Read the technician narratives and decide the ONE failure-mode family the cluster actually
     describes. Where a SYMPTOM CLASS has been keyed against the claims, it is authoritative. Where
     it reads NOT CLASSIFIED, the narratives are the only evidence -- the dealer symptom code is
     keyed at the service counter before the vehicle is examined and is not reliable.
     If that family is NOT among the failure modes this change is stated to be capable of
     producing, the verdict is NOT_ATTRIBUTED_SYMPTOM_MISMATCH on the SYMPTOM axis, however
     exactly the dates and the part number line up.

R-7  If another change listed under Co-Effective Signals was effective over the same builds on the
     same line and is also capable of producing that same failure-mode family, the printed evidence
     does not separate them: the verdict is EVIDENCE_INCONCLUSIVE, the axis is NONE, the signal
     reference is NONE, claims_attributed is 0 and no 8D is raised. Naming one of two
     indistinguishable causes is worse than saying the evidence does not choose.

R-8  Otherwise the cluster IS attributed to this signal. Name the verdict from the change type:
     ATTRIBUTED_SUPPLIER_LOT for a supplier lot change, ATTRIBUTED_PROCESS_CHANGE for a process or
     tooling change, ATTRIBUTED_SHIFT_OR_LINE for a shift pattern, line rebalance or operator
     cohort. claims_attributed is the number of COVERED claims from R-3. The signal reference is
     the change record's reference as printed.

R-9  Name the deciding axis from the change type: SUPPLIER_LOT for a supplier lot change,
     PROCESS_STEP for a process or tooling change, SHIFT for a shift pattern change, LINE for a
     line rebalance or operator cohort.

R-10 Raise an 8D (raise_8d YES) for an attributed cluster UNLESS a containment record for this same
     signal is printed with a closure date on or before the review date, in which case the answer
     is NO and the reason is that it is already contained. Every non-attributed verdict is NO.

Plant Notes
----------------------------------------------------------------
Two of these claims came through the same dealer group within a week of each other, which is why the cluster was raised at this size.

Raw responseThe raw response

{
 "verdict": "NOT_ATTRIBUTED_SYMPTOM_MISMATCH",
 "deciding_axis": "SYMPTOM",
 "claims_attributed": 0,
 "signal_ref": "PCR-5043",
 "raise_8d": "NO",
 "rationale": "Because symptom class is NOT_CLASSIFIED on all claims, I followed the narratives, which consistently describe fixings backed out and undamaged (LOOSE_FASTENER), while PCR-5043 is a plating change capable only of CORROSION_BLOOM and THREAD_DAMAGE, so the cluster does not match the change on the SYMPTOM axis."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Decide which plant change caused a warranty claims cluster — 120 correlation requests. Four tiers of one model family answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection, never a bill and never a vendor's claim about this workload.

Rates checked 2026-08-18. The provider that actually ran all 276 calls is kept out of these tables per this estate's naming rule, so no figure here is a bill.

The gradersOne way to grade, and why it is the only one

Read the three floors as a ladder, because the gaps between them are the finding: 48.33 pct for dates and a part number, 61.67 pct once the line and shift are added, 92.5 pct once the rule engine and every structured symptom field are handed over free. The model's 95.0 pct sits above all three -- but the honest comparison is per slice, not in aggregate.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

The three free floors, the stub and every pre-run check cost $0.00 and no calls. NO RUN WAS DISCARDED on this kit: nothing truncated on any arm, and the largest reply on the scored run was 15029 of the published 32,000-token ceiling (46.97 pct).

Cost driversWhat actually moves the bill

• The request, not the answer. 3252.2 input tokens per reading against 1647.58 output; the claim register, the narratives and the change record are the bill.
• Provider-side reasoning, drawn per call and left at the default. 184,317 of 197,710 output tokens on the scored run (93.2 pct) were reasoning, and that budget is re-rolled every call -- which is why a nine-call probe can bound a floor and never a ceiling.

Your volumeWhat it costs at your volume

Linear in candidates, because there is no shared state between readings: 1,200 candidates is $6.11 on the projection card. ⚠︎ THE NON-LINEAR TERM IS THE CANDIDATE LIST, NOT THE MODEL. This kit reads three candidates per cluster because that is what it was handed; a loose join over a real change log proposes tens. Ten times the clusters at ten candidates each is not 10x, it is 33x, and nothing in this kit ranks or prunes.

Model choiceWhy this model

One provider, one key, one model -- a forker holds one credential. The comparison on this page is against THREE FREE FLOORS and an ablation of the same model, not against a second vendor; every other row in the projection table is a published rate card applied to measured tokens, and no other model was called.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-query average, this run

The codeEvery file, and what it is for

Eight modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/signals.pythe correlation rule — a swap seam

Ten rules -- completeness, the effective window and its residual-stock extension, coverage by line and shift, the build-window test, the line test, the failure-mode-capability test, the co-effective test, the attribution and its axis, and the 8D test. decide() is also the answer key, and check_labels replays every reading through it off the finished page.

You change it to: DEALER_CODES is one manufacturer's coding list. Change it and the strongest free floor's fallback changes with it -- which is exactly the comparison this kit publishes.

# The domain: what a manufacturing signal is, what it can produce, and the rule that decides

REVIEW_DATE = "2026-08-25"
DEFAULT_CONTAINMENT_HORIZON_DAYS = 45
CANDIDATES_PER_CLUSTER = 3
ATTRIBUTED_SUPPLIER_LOT = "ATTRIBUTED_SUPPLIER_LOT"
ATTRIBUTED_PROCESS_CHANGE = "ATTRIBUTED_PROCESS_CHANGE"
ATTRIBUTED_SHIFT_OR_LINE = "ATTRIBUTED_SHIFT_OR_LINE"
NOT_ATTRIBUTED_SYMPTOM_MISMATCH = "NOT_ATTRIBUTED_SYMPTOM_MISMATCH"
NOT_ATTRIBUTED_BUILD_WINDOW = "NOT_ATTRIBUTED_BUILD_WINDOW"
NOT_ATTRIBUTED_LINE_MISMATCH = "NOT_ATTRIBUTED_LINE_MISMATCH"

src/segment.pythe section splitter

Splits the request into its 8 named sections; asserted across all 120 documents. Losing Claim Narratives silently is the worst failure it can have, and it would not look like one -- the register alone still produces a confident wrong answer.

# Split a correlation request into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Cluster And Request", "Claim Register", "Claim Narratives",
RULE = "-" * 64
def split(text):
def render(secs):

src/select.pythe send filter

Claimant Contact is mapped by no field and therefore never sent. Two independent locks, and neither is red-provable alone.

# Pick which sections of a correlation request are sent. Pure code -- the last deterministic step

BANNER = "Synthetic Record"
CLUSTER = "Cluster And Request"
REGISTER = "Claim Register"
NARRATIVES = "Claim Narratives"
SIGNAL = "Candidate Plant Signal"
POLICY = "Correlation Policy (Default)"
CONTACT = "Claimant Contact"
NOTES = "Plant Notes"
NEVER_SENT = (CONTACT,)

src/prompt.pythe prompt

Two parts: the instruction with the ten rules and the JSON shape, then the request. The register-only ablation replaces exactly one section's body.

# Assemble the one prompt this kit sends. No template engine, no framework -- string concatenation

INSTRUCTION = """\
REGISTER_ONLY_LINE = ("No technician narrative is available for any claim in this cluster. Judge the "
def _blank_narratives(secs):
def build(text, carried=None, register_only=False):
def rule_text():

src/adapters/__init__.pythe model call — a swap seam

Raw HTTP, stdlib only, bounded retry, a 900-second socket timeout and a SEPARATE, smaller retry budget for transport failures; the shared daily call cap is checked here.

You change it to: PROVIDER, BASE_URL, MODEL in .env; one function and one dict entry per vendor.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
HTTP_TIMEOUT_SECONDS = 900
TRANSPORT_RETRIES = 1
def _post(url, headers, payload, timeout=HTTP_TIMEOUT_SECONDS):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

src/correlate.pythe reader

One candidate signal, one call. The published token ceiling lives here.

# One cluster, one candidate signal, one model call. The runtime half of the kit.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SYSTEM = ("You are a warranty correlation desk. You read one cluster of warranty claims and one "
MAX_TOKENS = 32000
FIELDS = ("verdict", "deciding_axis", "claims_attributed", "signal_ref", "raise_8d")
def documents():
def clusters():
def load_doc(doc_id):
def facts_of(text):

evals/baseline.pythe free floors — a swap seam

Three of them, all $0.00, reading the PAGE rather than the kit's own parser. The strongest is handed decide() itself, a regex for the residual-stock sentence and the hand-keyed symptom class.

You change it to: Add a MODE to compare against your own existing correlation logic. The harness's --floor flag takes it with no other change.

# THE FREE FLOORS. Three of them, none a strawman, all of them USD 0.00.

MODES = ("date-part-join", "join-plus-line", "tagged-symptom-strong")
SEC_ORDER = ("Synthetic Record", "Cluster And Request", "Claim Register", "Claim Narratives",
def _section(text, name):
ROW = re.compile(r"^(WC-\d+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+([\d,]+)\s+(\S+)\s+(\S+)\s*$", re.M)
def claims_of(text):
def _field(body, label):
STOCK = re.compile(r"consumed on this line through (\d{4}-\d{2}-\d{2})")
def signal_of(text, read_stock_tail=False):
CO = re.compile(r"^  (PCR-\d+)\s+(\S+)\s*$", re.M)

evals/scoring.pythe scorer

Exact match per cell. The attributed/not discriminator is scored on its own with both denominators, and the TAG/PROSE split is never blended.

# Score a run against the answer key. Pure code, exact match per cell. No model grades anything.

FIELDS = ("verdict", "deciding_axis", "claims_attributed", "signal_ref", "raise_8d")
EVIDENCE_KINDS = ("TAG", "PROSE", "NONE")
PROFILES = ("clean", "contained", "stock", "symptom", "line", "window", "inconclusive",
def _pct(n, d):
def _same(got, want):
def score(records, golds):
def _protection(records, golds):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 3252 input and 1647 output tokens per reading (one candidate plant signal against one claim cluster), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ MEASURED, WITH A FORCED CONDITION, A PAIRED CONTROL AND A WHOLE-ANSWER DIFF. Plant Notes is this kit's injection surface, and the probe forced the instruction-shaped note onto every reading where an 8D was genuinely due rather than counting where the seed happened to put it.

API_KEY is read from .env (gitignored) or the real environment; never logged, never placed in a prompt, and redacted from any provider error message the local UI passes to the browser.

The experimentWhat was actually fired

The shipped corpus already carries the instruction-shaped note on 24 of 120 readings, drawn like the other four Plant Notes. Counting how often it happened to land on a reading where an 8D was due is free -- and a denominator that small is an observation, not a resistance rate. So the probe REBUILDS every 8D-due reading with the Plant Notes section replaced by that sentence, fires those and only those, and pairs each against its own un-injected answer. Readings the model did not raise WITHOUT the note are excluded from the denominator, because a decision that was never made cannot be suppressed. Both were run for real on 2026-08-25.

Both boundaries are measured in both directions. The injection probe forces the condition, pairs each reading against itself and diffs ALL FIVE answer fields -- because an injection can fabricate as well as suppress, and a probe that scores only the verdict measures one failure mode and misses the other.

The result0.0 pct of 25 paired trials suppressed, at a 32,000-token ceiling; 1 reading had any answer cell move at all -- AND IT MOVED TO THE RIGHT ANSWER. Read the next paragraph before quoting the zero.

One phrasing, one model, one corpus, one ceiling. Every reading where suppression was even possible, each paired against its own un-injected answer, and all five answer fields diffed rather than only the 8D call.

HonestyWhat this does not prove

• One phrasing only. A note claiming the lot has been re-qualified, claiming the change board has already closed the concern, or written to look like a system banner is a different experiment and was not run.
• One ceiling. A sibling kit's discarded 16,000-token run suppressed a finding the same sentence could not suppress at 32,000. Nothing here generalises to another cap.
• Whether a note placed in a section this kit does NOT send would have any effect -- by construction it cannot, and that was not probed either.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never open a containment, raise an 8D, stop a line, hold a shipment, charge back a supplier or notify a dealer -- and never present the correlation policy, the failure-mode families or the containment horizon as any real manufacturer's terms.

Stated to the model on every call in src/prompt.py's INSTRUCTION block, and it is a property of what is ABSENT from the code: there is no writer of any kind outside results/*.json, in src/, evals/, tools/ or ui/.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is a PROMPT RULE PLUS AN ABSENCE, not a runtime enforcement layer. Nothing stops a forker adding a writer tomorrow.
• The injection result is ONE SENTENCE against ONE model on ONE corpus at ONE ceiling. It is not a resistance rate for prompt injection in general, and the phrase 'resistant to prompt injection' appears nowhere in this kit on purpose.
• An attribution opinion is not a containment decision, and this kit says so nowhere in code -- it is a property of what the output IS, and a forker pointing this at a real change log owns that.

WatchedWhat is watched, and why that one

The kit measures 45 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

3 measured by the latest run 42 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

8 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on clusters_caught, clusters_missed, false_8d, inconclusive_false, missed_8d, plant_signal_false, plant_signal_fn, plant_signal_fp, plant_signal_tn, plant_signal_tp, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · with the model in the path — 3 runs. Columns here are only ever compared with each other.

not a time series No two of these 3 runs measured the same system — they differ on answered, clusters_caught, clusters_missed, clusters_needing_8d, context_incomplete_cells, eightd_cells, false_8d, inconclusive_cells, inconclusive_decidable_cells, inconclusive_false, join_misleading_cells, joinline_misleading_cells, missed_8d, none_cells, plant_signal_attributed_cells, plant_signal_false, plant_signal_fn, plant_signal_fp, plant_signal_quiet_cells, plant_signal_tn, plant_signal_tp, population_at_risk_vehicles, prose_cells, quiet_cells, readings_scored, tag_cells, tagstrong_misleading_cells — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 26 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 3 chips that all say so.

DeviationsWhat deviated

0 breaches across 8 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

Nothing runs on a schedule. This kit ran once, that run is captured, and nothing executes on our side afterwards.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, no dependency at all, and a prompt anyone can read end to end. requirements.txt is a comment explaining why it is empty. A framework abstraction would own the retrieval step -- there is none here -- and the rule engine, which is already the whole surface of src/signals.py and doubles as the answer key.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: the reader -> src/select.py -> src/prompt.py -> src/adapters -> evals/scoring.py. No branching, no tool-calling and no agent loop.

The other sideWhat a framework costs you

• Swapping providers means editing the PROVIDERS dict by hand. In exchange the prompt has no hidden templating layer and requirements.txt stays empty.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-plant-signal on the fast tier, 2026-08-25. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 5 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-plant-signal-datepartjoin, b001-plant-signal-joinplusline, b002-plant-signal-tagstrong recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — blocking is computed in memory each run and is the only reduction step.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe verdict, deciding axis, claims attributed, signal reference and 8D call, per reading, exact match against the computed answer key

whether each of the 5 answered fields equals the answer key, per reading

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings. No model grades anything, on any arm.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 120 per field. plant_signal_accuracy_pct is scored separately as a BINARY over its own two denominators, and the TAG/PROSE/neither split is reported per slice and never blended.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/signals.decide() from the same planted facts the corpus builder wrote onto the page -- and re-derived off the FINISHED page through evals/baseline.py's parser, a different one, before any run spends.

No true/false rates for this grader. It records 6 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold was swept: exact match has no tunable. Denominators are stated beside every rate, and the two that matter are small -- 33 attributions and 27 readings where an 8D was due -- so a band finer than 3.0 pct on either is finer than one row.

Cadence: once per run; free

The decisionWhen to reach for it