The business caseThe problem this solves

A DC has less of a SKU than every requesting store asked for, and a written policy says who gets what -- but by the time a shortage reaches the review meeting, the numbers have already been computed by whoever ran the split, and nobody has connected the outcome back to WHY the policy couldn't cleanly resolve it: a promo commitment over-promised, a customer pre-order the reservation system never saw, a genuine demand spike, or a supply shortfall on the DC's own receiving side. A planner opening every constrained SKU one at a time, hand-checking whether promo and customer commitments fit inside what the DC actually has, running the fair-share math themselves, then chasing down whoever flagged the shortfall to find out why before the review meeting can even start.

The inputThe actual review sessions

The corpus is 40 review sessions, 0.35 MB (jsonl 3 · md 1). A real allocation event names real SKUs, real store-level commitments and a real supply position -- exactly the material a retailer will not let leave the building. Worse: the eval's own gold label is 'what approved allocators actually did', and no public dataset pairs a shortage event with a written policy AND the human decision it produced -- real allocator decisions are not published anywhere, for the same reason a markdown calendar or a vendor scorecard is not published. A public retail-flavoured dataset (Kaggle among them) was considered and set aside for the same reason data-match's own SOURCES.md gives for its corpus: generating it removes the licence question and the labelling problem entirely, rather than bolting a public sales file onto a gold label it was never designed to support. Six scenarios were planted deliberately -- four traceable causes, one genuinely untraceable shortfall, and clean policy compliance -- to give the eval something real to tell apart.

# Constrained supply allocation — written policy

**This policy was authored for this kit. It is not a real retailer's policy, and it is not claimed
to be.** The atlas facet sheet this kit answers records, as its own first open item: *"no written
allocation policy exists; atlas premise requires an operator-supplied written policy before
build."* No operator-supplied policy exists to encode, so this document is the one the kit ships
— five numbered clauses, run exactly as written by `src/allocate.py`. A real deployment forking
this kit replaces this file's clauses with its own; nothing downstream cares what the clauses say,
only that they are numbered, ordered, and that `src/allocate.py` implements the same five in the
same order.

## The five clauses, in the order they apply

1. **Protect promo commitments first.** Any unit a store has been promised against an active
   promotion (`promo_committed_units`) is protected before anything else, up to the amount
   available. If total promo commitments across every store exceed what is available, the pool is
   shared across promo-committed stores proportional to each one's own commitment — nobody's
   promo commitment is protected in full at another's total expense.

2. **Protect committed customer orders next.** Units already promised to a specific customer
   (`customer_committed_units`) are protected from whatever clause 1 left, on the same
   proportional-sharing rule if the remaining pool cannot cover every commitment in full.

3. **Fair-share the remainder by trailing velocity.** What is left after clauses 1 and 2 is split
   across every store's still-unmet ask, weighted by that store's trailing velocity — a store that
   sells this SKU faster gets a larger share of what remains — and capped so no store is ever

The outcomeWhat a good result looks like

A drafted review brief: every event the written policy could not cleanly resolve, each with a probable cause traced to two source lines -- or an honest 'unknown' -- alongside the code-computed split it already received. Decision-free by design: nothing here proposes a different allocation or ranks which event to fix first.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• An event with two clear, SKU-labelled notes lines explaining it — the fast tier
  97.8% cause-tag agreement on every traceable event this run measured.
• An event the notes log genuinely does not explain — the fast tier
  It said 'unknown' correctly on 80.0% of gold-unknown events this run measured, rather than reaching for the nearest plausible-sounding line.
• A workflow that cannot tolerate a whole session silently returning no answer — the reasoning tier
  0 of 40 pro-tier replies failed to parse, against 3 of 40 on the fast tier -- every flagged event was answered.
• Feeding the brief's output straight into a system that keys on event_id — add a schema-validation pass before trusting the reply parses at all
  3 of 40 fast-tier sessions returned unparseable JSON outright -- this run's scorer never got to check cause or citations for any event in those sessions.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-18. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-18 — r001-alloc-short. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — A fresh clone with NO key configured reproduces the corpus byte-identically (python3 tools/build_corpus.py) and serves the whole panel -- flagged events, per-store split, merchant notes -- with no call made (python3 -m src.app). Measured 2026-08-18 with API_KEY blank: two commands, both exit 0, plain python3, no install step -- requirements.txt is deliberately empty. What it cannot do without a key is re-run the eval or press Draft review brief; the recorded run renders from the committed result file either way.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. END-TO-END per review session: one HTTP request carrying every flagged event's code-computed split plus the session's merchant notes log, and the reply parsed to a per-event cause/citation list plus a narrative. Running the five-clause policy and deciding which events need review happen before this clock starts and cost no network at all -- pure code over the session's own JSON.

Current processWhat it replaces

A planner opening every constrained SKU one at a time, hand-checking whether promo and customer commitments fit inside what the DC actually has, running the fair-share math themselves, then chasing down whoever flagged the shortfall to find out why before the review meeting can even start.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• jsonl
• md
• 2 of 8 stations carry a recorded failure

scroll → 8 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one flagged event is

One session, cut to its flagged-only event list by src/allocate.py::flagged_events -- no train/test split, every session is drafted once.

SetupWhat the setup figure measured

There is no index. tools/build_corpus.py generates 40 sessions, 192 shortage events across 6-10 stores each, their merchant notes and the gold flag/cause list, from a fixed seed, with no clock read and no model called.

LicenceLicence

MIT (this repository's own licence)

Bring your ownBring your own review sessions

This kit's actual job -- run a written priority policy against a scarce pool in code, itemize every event the policy could not cleanly resolve with a quantified split and a probable cause traced to real source lines, and draft the review brief -- applies to any written-policy allocation: a capacity-allocation policy, a budget-allocation policy, anything with a stated priority order and a scarce pool is another instance. Point tools/build_corpus.py and data/POLICY.md at your own clauses and your own event shape; src/allocate.py, src/pack.py and src/prompt.py read sessions by id and do not change.

⚠︎ And what stops being true when you do: The measured flag-completeness, cause-tag-agreement and fabrication figures are properties of THIS corpus's four templated cause-phrasings, its 40% equity floor and its 9-13-line notes logs. A cause phrased differently than this corpus's own templates, a floor set elsewhere, a store network larger than 10, or a notes log an order of magnitude longer -- none of that has been measured, and the 3-of-40 malformed-JSON slip (Business.not_good_enough) is evidence the fast tier does not always hold a schema perfectly even on this corpus's own modest scale.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 1,530 tokens assembled, 818 are instructions — 53% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Logged verbatim by evals/run.py on r001-alloc-short -- the literal user message sent for session SESS-001, with the fixed system prompt from src/prompt.py prepended.

Full promptThe prompt in full

You are drafting the allocation review brief for a supply-constrained-events review meeting. You are given one week's FLAGGED shortage events for a region -- SKUs where the written allocation policy (protect promo commitments, protect customer commitments, then fair-share the rest by trailing velocity, then check a 40% equity floor per store) could not be fully honored, already identified and split by code -- and that week's own merchant notes log.

The per-store unit counts you are given (allocated_units, ask_units, available_units) are FINAL. You never propose a different number for any store. Your job is limited to two things, for EACH flagged event in the list:

1. Decide which of these five causes explains why this event could not be cleanly resolved, using ONLY the notes log -- never outside knowledge, never a plausible-sounding guess:

  promo_overcommit       merchandising committed promo units across stores that together exceed what the DC actually has for this SKU this week.
  customer_overcommit    store reps took customer pre-orders beyond what the reservation system had actually held for this SKU.
  demand_surge           nobody over-promised anything -- a genuine, unplanned spike in sell-through this week made the ordinary fair-share math infeasible.
  supply_shortfall       the distribution center itself received fewer units than the supply plan assumed for this week.
  unknown                the notes do not support any of the four causes above for this specific event. This is the correct answer when the evidence is not there -- guessing a specific cause is a worse answer than saying so.

If the notes log supports a specific cause for that event's SKU, name it and cite the exact two notes lines (verbatim) that together support it. If the notes log does NOT support a specific cause for that SKU -- no line mentions it, or the lines that do mention it don't add up to one of the four named causes -- the cause is 'unknown' and both citation fields must be empty strings. Do not cite a line that does not actually name or clearly concern that SKU just because it is the closest-sounding one available: an unsupported 'unknown' is the correct, honest answer and is graded as such; a cause with a citation that is not really about that SKU is graded as a fabrication, which is worse.

2. State which protections held: echo back promo_protected and customer_protected exactly as given in the input for that event (true or false), and whether any store is at or below the equity floor (floor_breach_stores is non-empty in the input).

After the per-event entries, write a short narrative (3-6 sentences) for the meeting audience that ties the flagged events together. Every number you state in the narrative -- a unit count, a store count, a percentage -- must be a number that is actually present in the event list you were given; do not compute, round, or restate a figure that changes its value.

NEVER propose a different allocation for any store, and never rank or recommend which event should be resolved first. That decision belongs to the humans in the meeting this brief is for -- your job is limited to itemizing, tagging a probable cause (or saying unknown), and describing what happened, never what should be done about it.

Session SESS-001 -- East Region, 2026-W28

FLAGGED EVENTS (3)
--------------------
- EV-0002 (Basecamp Cookset): available=148 total_ask=348 | promo_protected=True customer_protected=True | floor_breach_stores=['ST-030', 'ST-020', 'ST-003', 'ST-038'] | ST-030: ask=49 alloc=8, ST-020: ask=48 alloc=13, ST-034: ask=68 alloc=51, ST-047: ask=22 alloc=16, ST-058: ask=28 alloc=21, ST-003: ask=63 alloc=19, ST-038: ask=70 alloc=20
- EV-0003 (Boulder Approach Shoe): available=5 total_ask=529 | promo_protected=False customer_protected=False | floor_breach_stores=['ST-007', 'ST-020', 'ST-029', 'ST-018', 'ST-039', 'ST-036', 'ST-014', 'ST-046', 'ST-031'] | ST-007: ask=82 alloc=0, ST-020: ask=43 alloc=0, ST-029: ask=60 alloc=2, ST-018: ask=34 alloc=0, ST-039: ask=71 alloc=1, ST-036: ask=40 alloc=0, ST-014: ask=69 alloc=0, ST-046: ask=74 alloc=2, ST-031: ask=56 alloc=0
- EV-0005 (Switchback Trekking Pole): available=7 total_ask=436 | promo_protected=False customer_protected=False | floor_breach_stores=['ST-005', 'ST-034', 'ST-007', 'ST-025', 'ST-043', 'ST-028', 'ST-012'] | ST-005: ask=71 alloc=3, ST-034: ask=57 alloc=3, ST-007: ask=90 alloc=0, ST-025: ask=20 alloc=0, ST-043: ask=38 alloc=1, ST-028: ask=84 alloc=0, ST-012: ask=76 alloc=0

MERCHANT NOTES LOG
-------------------
- Boulder Approach Shoe: store ops confirmed reps took customer pre-orders ahead of what the reservation system actually had on hold -- running about 18 units over.
- Planogram reset for the accessories wall lands next week -- separate meeting, not this one.
- Reminder: allocation review sign-off is due Thursday EOD this cycle, no exceptions.
- Northface Rain Poncho: DC receiving confirmed this week's inbound came in short of the supply plan -- about 146 units under what was expected.
- Switchback Trekking Pole: store ops confirmed reps took customer pre-orders ahead of what the reservation system actually had on hold -- running about 12 units over.
- Basecamp Cookset: sell-through spiked this week well past the trailing-velocity trend -- nothing on the promo or commitment side changed, it's genuine demand.
- New regional merchandiser starts Monday; route urgent items to the deputy until then.
- Correction thread on Boulder Approach Shoe -- the pre-order count in the POS doesn't match what allocation planning reserved; reps were working off last week's number.
- Correction thread on Switchback Trekking Pole -- the pre-order count in the POS doesn't match what allocation planning reserved; reps were working off last week's number.
- Supply note on Northface Rain Poncho -- vendor ASN matched the shorted quantity; replenishment plan assumed the full number and allocation is working off the shortfall.
- Flagging Basecamp Cookset -- confirmed with the category lead that the surge is real store-floor demand, not a data or commitment issue on our side.

Return a JSON object with two keys: "events", a list with exactly 3 entries, one per event above in the same order, each {"event_id": <id>, "cause": <one of: promo_overcommit, customer_overcommit, demand_surge, supply_shortfall, unknown>, "citation_1": <verbatim notes line, or "">, "citation_2": <verbatim notes line, or "">, "promo_protected": <true or false>, "customer_protected": <true or false>, "note": <one sentence>}; and "narrative", a string of 3-6 sentences.
Answer with the JSON object only.

Raw responseThe raw response

{
  "events": [
    {
      "event_id": "EV-0002",
      "cause": "demand_surge",
      "citation_1": "Basecamp Cookset: sell-through spiked this week well past the trailing-velocity trend -- nothing on the promo or commitment side changed, it's genuine demand.",
      "citation_2": "Flagging Basecamp Cookset -- confirmed with the category lead that the surge is real store-floor demand, not a data or commitment issue on our side.",
      "promo_protected": true,
      "customer_protected": true,
      "note": "Promo and customer commitments were protected, but the demand surge left some stores below the equity floor."
    },
    {
      "event_id": "EV-0003",
      "cause": "customer_overcommit",
      "citation_1": "Boulder Approach Shoe: store ops confirmed reps took customer pre-orders ahead of what the reservation system actually had on hold -- running about 18 units over.",
      "citation_2": "Correction thread on Boulder Approach Shoe -- the pre-order count in the POS doesn't match what allocation planning reserved; reps were working off last week's number.",
      "promo_protected": false,
      "customer_protected": false,
      "note": "Customer overcommitment left the event with zero allocation for many stores, and neither promo nor customer protections held."
    },
    {
      "event_id": "EV-0005",
      "cause": "customer_overcommit",
      "citation_1": "Switchback Trekking Pole: store ops confirmed reps took customer pre-orders ahead of what the reservation system actually had on hold -- running about 12 units over.",
      "citation_2": "Correction thread on Switchback Trekking Pole -- the pre-order count in the POS doesn't match what allocation planning reserved; reps were working off last week's number.",
      "promo_protected": false,
      "customer_protected": false,
      "note": "Customer overcommitment led to a severe shortage, with most stores receiving zero units and no protection held."
    }
  ],
  "narrative": "The flagged events for East Region W28 show three distinct shortage patterns. EV-0002 was driven by genuine demand surge, with 148 units available against 348 asked, and 4 stores breaching the equity floor. EV-0003 and EV-0005 both stem from customer overcommitments, where store reps took pre-orders exceeding the reservation system by 18 and 12 units respectively. For EV-0003, only 5 units were available against 529 asked, leaving 9 stores in floor breach. For EV-0005, 7 units were available against 436 asked, with 7 stores in floor breach. Neither EV-0003 nor EV-0005 had promo or customer protections held, and all three events left multiple stores at or below the equity floor."
}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Explain a store chain's stock shortages before the allocation review — 40 review sessions. Two tiers of one model family answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

The grader is evals/scoring.py and it is pure code: the event set the brief covers is compared to the gold flagged-event set (flag completeness), each matched event's cause is compared to gold (cause-tag agreement, split by whether gold's own cause was traceable or unknown), each non-unknown cause's two citations are checked as real, SKU-relevant substrings of that session's own notes (fabrication), every quantified claim in the narrative is checked against the packed event numbers (faithfulness), and every event's code-computed split is checked to sum to exactly what was available (conservation, independent of the model). No judge model and no rubric scored by a person. The protected-class guardrail is a separate 500-shuffle permutation test, evals/parity_check.py.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anyone paid.

Rates checked 2026-08-18. The provider that actually ran both scored evals is kept out of these tables per this estate's naming rule, so nothing here is what was actually paid -- the real spend for this kit's build (well under $0.25 across both tiers, the debug calls and the free baseline) is recorded in the commit history, not on this page.

Grading unitWhat the grading figure prices

Grading calls nothing -- an event set is compared by code, a cause string is compared by code, a citation is a substring check, and conservation and parity are arithmetic on the corpus itself. A measured $0.00, not an unpriced one.

The gradersOne way to grade, and why it is the only one

The floor gets completeness and protection-state echo for free -- both are pure copying of what it was handed -- and its narrative never invents a number, so it is faithful by construction. What it structurally cannot do is tell a traceable cause from an untraceable one: it always says unknown, so its cause-tag agreement (21.5%) is exactly the gold set's own unknown share -- the fast tier's 94.1% is the real, measured gap a model closes.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Identical work in both rows — the same query, the same tokens. Only the price list changed. Which is why the token count is the measurement and the dollar is arithmetic you can redo on your own card.

Answering vs gradingAnswering and grading are two separate bills

The baseline (evals/baseline.py), the corpus self-test (tools/selftest_allocate.py) and the parity check (evals/parity_check.py) are all pure code and cost $0.00 to run. Figures above are projected onto the cheapest published card, not the real spend.

Cost driversWhat actually moves the bill

• The system prompt, the single largest part at 3275 of about 6125 characters (~53%) -- the fixed cause vocabulary and the five-clause policy summary cost more per call than any one session's own event list or notes.
• Notes-log length: every line in the session's own merchant notes is sent whether or not it explains any event, by design (src/pack.py never pre-filters).
• Output length scales with how many events are flagged in a session (1-5 here, ~90-180 output tokens per event) -- a busier review session costs more per call, linearly.

Your volumeWhat it costs at your volume

Linear in sessions and cheap on the fast tier: 40 sessions cost about $0.0606 projected onto this card, so ten times the set is about $0.606. Each call is independent and self-contained -- no shared context or retrieval step to amortise.

Model choiceWhy this model

It is the tier the shared key is pointed at, and it is the cheap one -- the right place to start a question whose answer might be 'the free floor already wins part of this', which on flag completeness precision it exactly does. The reasoning tier was run too, at roughly 3.0x the price, and won on completeness recall (100.0% vs 90.8%) and on the fabrication guardrail (0 vs 1) but lost on cause-tag agreement -- see cost_by_model for the full trade rather than a single winner.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

Every fast-tier model number on these pages: 40 sessions, one call each.

The codeEvery file, and what it is for

Nine modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pycorpus generator — a swap seam

Generates 40 review sessions, 192 shortage events across synthetic stores, a merchant-notes log per session and the gold flag/cause list, from a fixed seed. Six scenarios are planted, each with a KNOWN true cause (or none, on purpose) -- see data/SOURCES.md.

You change it to: Point it at your own shortage events and your own store network. src/allocate.py, src/pack.py and src/prompt.py read sessions by id and do not care where the numbers came from.

# Generate the allocation review sessions, their shortage events, per-store facts, merchant

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
SEED = 20260818
SKUS = [
REGIONS = ["North Region", "South Region", "East Region", "West Region", "Direct-to-Consumer"]
WEEKS = ["2026-W28", "2026-W29", "2026-W30", "2026-W31"]
STORE_IDS = ["ST-%03d" % i for i in range(1, 61)]
TIERS = ["A", "B", "C"]
SCENARIOS = ["clean", "promo_overcommit", "customer_overcommit", "demand_surge", "supply_shortfall",

data/POLICY.mdthe written policy

Five numbered clauses -- protect promo, protect customer commitments, fair-share by velocity, check the equity floor, exclude protected-class fields -- authored for this kit because none existed to encode. src/allocate.py implements these five, in this order, in pure arithmetic.

# 

src/allocate.pythe cut — a swap seam

SEAM -- runs the five-clause policy on one event and decides whether it needs review (promo or customer protection could not be fully honored, or the equity floor breached). Pure code, no model. Every allocated unit in every published figure comes from here, never from the model.

You change it to: EQUITY_FLOOR_PCT is a stated judgement, not fitted to this corpus -- change it and re-run the eval, which costs one call per session.

# SEAM 1 -- the cut. Running the five-clause written allocation policy (data/POLICY.md) on one

EQUITY_FLOOR_PCT = 40.0
def _largest_remainder(pool, shares):
def _protect(pool, wants):
def allocate(event):
def allocate_session(session):
def flagged_events(session):

src/rubric.pythe rubric — a swap seam

The four-member cause vocabulary plus 'unknown', and the three graded axes (flag completeness, cause-tag agreement, narrative faithfulness), declared once and imported by the prompt, the app and the scorer.

You change it to: A cause this kit's fixed four-member list does not carry needs a new CAUSE_VOCAB entry here plus new template notes lines in tools/build_corpus.py.

# The rubric this kit's brief is graded against -- the sections a review brief must carry, the

CAUSE_VOCAB = (
CAUSE_MEANINGS = {
RUBRIC_AXES = (
FABRICATION_GUARDRAIL = (
PARITY_GUARDRAIL = (

src/pack.pyPack

Deterministic assembly of one session's flagged-only event list and its notes log into the context the model sees, capped to a stated budget. A clean event never reaches the model.

# SEAM 2 -- Pack. Deterministic assembly of one session's flagged events and its merchant notes

MAX_NOTES = 40
MAX_EVENTS = 20
def pack(session, notes, flagged):

src/prompt.pythe prompt

The cause vocabulary, the per-event answer schema and the narrative instruction, declared once and read from here by the prompt, the parser and the app.

# Assemble the one prompt this kit sends per session, and parse the one reply it gets back.

MAX_TOKENS = 3600
SYSTEM = (
DEFAULT_PROMPT = "v1"
SYSTEMS = {"v1": SYSTEM}
def _event_block(packed):
def _notes_block(packed):
def build(packed, prompt=DEFAULT_PROMPT):
def parse(raw):

src/adapters/__init__.pythe model adapter — a swap seam

SEAM -- the model. Raw HTTP for every provider, stdlib only.

You change it to: .env plus the same run again. Any OpenAI-compatible provider or Anthropic.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
def _post(url, headers, payload, timeout=120):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}
THINKING_OFF = {"type": "disabled"}
def complete(cfg, system, user, max_tokens=1024, thinking=None):

src/draft.pythe AI layer

Loads one session, allocates and packs its flagged events, calls the model once, parses the reply. The whole AI layer, deliberately short.

# SEAM 3 -- the AI layer. Loads one session, allocates and packs its flagged events, calls the

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
def _read_jsonl(path):
def sessions():
def sessions_by_id():
def notes_by_id():
def gold_by_id():
def draft(cfg, session, notes, complete=None, thinking=None, prompt=P.DEFAULT_PROMPT):

src/app.pythe app

Static files and a handful of JSON endpoints on the standard library. Runs the same citation-fidelity check evals/scoring.py runs, on every live draft, before the UI renders it.

# The minimal local UI. Standard library only -- python -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
PORT = int(os.environ.get("PORT", "8787"))
def _corpus():
class H(BaseHTTPRequestHandler):
def main():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 1617 input and 608 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

No untrusted external input surface. Every fact in the prompt -- the flagged events, their code-computed splits, the merchant notes log -- is computed directly from this kit's own generated corpus (tools/build_corpus.py, src/allocate.py); nothing arrives from outside the codebase the way an inbox message or an ingested document does on sibling kits.

The provider key lives only in .env (gitignored) or the shared repo-root .env, read by src/config.py and never logged. src/app.py strips api_key and base_url out of any exception message before it reaches the browser, so a misconfigured key cannot leak into a UI error.

The experimentWe did not attack it -- there is no surface to attack

An indirect prompt injection needs a field an outside party controls that reaches the prompt. This kit has none, so the five rows above are boundaries confirmed by reading the code and by measurement, not payloads run through it. Confirmed by reading the code, not by a run, on 2026-08-18 -- no attack run was fired; see the experiment section for why.

Each boundary above was checked by reading the call sites or by direct measurement -- there is no untrusted field to construct a payload against (see posture).

The result0 attack trials, by design -- but 5 boundaries around the model's reply and the allocation formula hold, confirmed by reading the code and by a measured permutation test, and the fabrication guardrail runs live, not only in a committed eval result.

This kit has no field an outside party controls that reaches the prompt -- the flagged events and the merchant notes log are all computed from data tools/build_corpus.py generated, never from correspondence, a document, or any other externally supplied text. A future version that ingests real merchant notes from a live external system would reopen this question and should be attacked before shipping.

HonestyWhat this does not prove

• Whether a future version of this kit that ingests real merchant-notes text from a live external system (rather than a generated corpus) would reopen an injection surface. Not applicable to the shipped version; worth re-examining if this kit is ever pointed at a live notes feed.

In one lineThe guardrail is enforced in code

Not a prompt instruction the model can ignore — a comparison applied after the reply is parsed, before anything downstream sees it.

A non-'unknown' cause is shown only alongside its two citations, and every citation is checked as a real, SKU-relevant line from that session's own notes before the UI renders it. No code path in this kit writes to an allocated unit, and the equity-floor and protection formulas never read a store's protected-class tag.

src/app.py's /api/draft endpoint and evals/run.py both return a per-event cause/citation/note plus a narrative; neither writes to data/sessions.jsonl or any other store. There is no re-allocation or apply path in this kit at all.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• IT IS NOT A RATE LIMIT, A POLICY CHECK, OR A GUARD THAT WITHHOLDS A REPLY FROM BEING SHOWN. It flags a fabricated citation in the UI; it does not suppress the brief -- a reader still sees the flagged line, marked, per the UI's own '.cite.bad' treatment.
• It does not make the cause-tag judgement itself correct -- see the malformed_reply finding in Eval.taxonomy, which this guardrail does nothing to prevent.
• It does not validate the narrative's numbers live, independent of the citation check -- narrative faithfulness is graded by evals/scoring.py after the fact in the eval harness, not enforced on a live draft the way the citation check is. See add_first.
• A future version wired to a real allocation-write system would need a real, separate approval gate before writing anything; see add_first.

WatchedWhat is watched, and why that one

The kit measures 25 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

15 measured by the latest run 10 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (conservation_events 192, gold_flagged_total 130, sessions 40, thinking False) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

2 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked.

not a time series No two of these 2 runs measured the same system — they differ on the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

DeviationsWhat deviated

0 breaches across 2 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

Re-run evals/baseline.py and evals/parity_check.py (both free) on any change to src/allocate.py. Re-run evals/run.py (paid, one call per session) on any change to src/prompt.py, src/allocate.py, src/rubric.py, data/POLICY.md or the corpus generator.

In one lineNo framework, deliberately

This kit has no framework and no dependencies at all -- json, os, random, re and http.server from the standard library. requirements.txt is an empty file with a comment explaining that the emptiness is load-bearing.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Nothing here is a graph. The pipeline has one path per session -- allocate, pack, prompt, call, parse, score -- with no branching and no state carried between sessions. A framework would add an orchestrator to a for-loop.

The other sideWhat a framework costs you

• A cause outside the four-member vocabulary needs a hand-written CAUSE_VOCAB entry plus new template notes lines in tools/build_corpus.py, rather than being configured declaratively.
• There is no built-in retry/backoff beyond src/adapters.py's own bounded retry -- a framework's queue and worker model is not here, so a real deployment adds its own scheduling.
• No built-in observability beyond what evals/run.py prints and writes to results/ -- a framework's tracing/dashboard integration is not here.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-alloc-short on the fast tier, 2026-08-18. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — one document is one unit, whole into one call.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT (this repository's own licence) Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe brief's per-event cause tag and citations against that session's gold flag list, and its narrative against the packed event numbers

Did the brief itemize the right set of flagged events, tag each with the right cause (including 'unknown' when gold says unknown), cite two real, SKU-relevant notes lines whenever it claimed a specific cause, and state only numbers actually in the packed event list or the notes log it was handed?

MethodHow the test was run

evals/scoring.py::score, in-process, no key and no model. The same function evals/baseline.py calls, and the same predicates src/app.py runs on every live draft.

The inputOne real row, seen by every grader

The formulaWhat it computes

flag completeness = |brief events ∩ gold flagged events| over gold (recall) and over brief events (precision); cause-tag agreement = correct causes / matched events, split unknown vs traceable; fabricated_cause = matched events with a non-unknown cause and a citation that is not both real and SKU-relevant; narrative faithfulness = sessions with zero unmatched quantified claims / sessions with a narrative.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: The corpus's own gold cause and citations, decided by tools/build_corpus.py at generation time, before src/allocate.py's flag check runs on the same event. This grader IS the reference, so its own TPR/TNR are not separately measured.

How tight can the band be? There is no tuned threshold anywhere in this grader: the equity floor is a stated constant in src/allocate.py, and the citation check is a literal substring-and-relevance test.

Cadence: Re-run on any change to src/prompt.py, src/allocate.py, src/rubric.py, data/POLICY.md or the corpus generator.

The decisionWhen to reach for it