The business caseThe problem this solves

A forecast exception report ranks by how wrong the forecast looks, and a planner's day is finite. The ordering that matters is by money at risk, and the inputs for it — unit cost and weeks of cover — are printed on the page while the sort is not. On this corpus the top ten by variance and the top ten by exposure overlap 16.7 pct, and £8,929,027.85 of exposure sits in lines the legacy sort buries. an exception queue sorted by variance percent — which puts a 40 pct miss on a slow-moving low-value item above a 6 pct miss on the line carrying the week's money, re-queues variances somebody explained two cycles ago, and cannot see a line that has been quietly biased the same way for a month.

The inputThe actual forecast line extracts

The corpus is 120 forecast line extracts, 0.77 MB (txt 120). A retailer's demand forecast and its unit costs are the two numbers that reveal its margin structure line by line. There is no public one. A scrubbed export is worse rather than better: scrubbing removes the planner's typed explanations, which is exactly the layer this kit measures.

Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated demand-forecast line review extract
for an AI use-case kit; it reproduces no retailer, no supplier, no store, no real product
and no real person. The tolerance bands and exposure thresholds are ILLUSTRATIVE
DEFAULTS.

Forecast Line
----------------------------------------------------------------
Line reference                 : FL-0001
Item                           : 4-pack cotton bath towel, ecru
Item code                      : ITM-92866
Category                       : Home textiles
Location group                 : NORTH METRO
Planner of record              : Ingrid Solberg
Unit cost                      : GBP 8.14
Remaining weeks of cover       : 5.4 weeks

Variance Detail
----------------------------------------------------------------
Forecast, this week            : 98,359 units
Actual, this week              : 87,992 units
Variance                       : -10.54 pct
Units of forecast error        : 10,367 units UNDER forecast

The financial exposure this line carries is NOT printed on this report and never has
been. It is the units of forecast error, times the unit cost above, times the remaining
weeks of cover above. Every input is on this page.

Exception Policy (Default)
----------------------------------------------------------------
The table below is an OPERATOR-TUNABLE DEFAULT, not a real retailer's actual policy -- no
documented tolerance percentage, exposure threshold or queue depth exists for this row.
Applied exactly as printed. The queue is ordered by FINANCIAL EXPOSURE, never by variance
percent.

  EXP_NONE       GBP 0 to under GBP 5,000           nothing is owed on it

The outcomeWhat a good result looks like

every exception placed in an exposure band with the money named, the ones already explained taken off the queue, and the persistently biased lines surfaced — 0 false raises across 88 quiet readings.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Your unit costs and cover figures are reliable, and every explanation is keyed into a structured reason code — the free floor — exposure-arithmetic or structured-log-strong, $0.00
  Ranking is a multiplication and the floor cannot be wrong about it: 100.0 pct against the model's 97.96.
• Explanations arrive as sentences in a planning channel, and lines drift quietly in one direction for weeks — the model, with the carried state
  0 false raises against the variance-only floor's 57, 100.0 pct of explained variances left alone against 11.11 pct, and 100.0 pct persistent-bias recall against 0.0 pct.

And where nothing here is good enough:

• You want the model's reading but cannot carry state between cycles — neither, as configured here
  The stateless control is the measurement: 35 false raises against 0, 33.33 pct of explained variances left alone against 100.0 pct, and persistent-bias recall 0.0 pct against 100.0 pct.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-25. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-25 — r001-forecast-exception. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Every corpus extract, the answer key, all three free floors, the cadence analysis, the injection probe and what run r001 answered ship in the repo. check_labels and the local UI both run with no key and no network.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one reading — one line, at one scheduled run, end to end including provider-side reasoning tokens, on the shared connection. Not a cold-start figure and not a per-line SLA: 120 readings ran in 405.3 wall seconds because different lines run concurrently while a single line's runs are strictly serial.

Current processWhat it replaces

an exception queue sorted by variance percent — which puts a 40 pct miss on a slow-moving low-value item above a 6 pct miss on the line carrying the week's money, re-queues variances somebody explained two cycles ago, and cannot see a line that has been quietly biased the same way for a month.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureArchitectureArchitectureEvalsEvalsEvalsEvalsPromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 3 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one reading is

40 forecast lines x 3 weekly cycles, walked strictly in cycle order per line so the established cause and the same-direction run carry forward the way they would in a real review.

SetupWhat the setup figure measured

There is no index to build — each reading's extract goes whole into the prompt. The 0.5s and $0.00 are the corpus generation itself.

LicenceLicence

MIT

Bring your ownBring your own forecast line extracts

tools/build_corpus.py writes data/corpus/*.txt and data/gold.jsonl. Point it at your own exception feed by keeping the 8 section headings and the field labels src/review.position_of parses. Set your own bands and your own exposure formula in src/exception.py first.

⚠︎ And what stops being true when you do: The measured figures do not transfer. The rank overlap (16.7 pct here), the explained share and the memory-dependent share are properties of this generator's declared distribution, not facts about demand planning.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — they were measured against the provider’s own counter, not apportioned by character share.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 1,595 tokens assembled, 1,414 are instructions — 89% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Reproduced from src/prompt.build for FL-0038-C2 with that reading's real carried state, not retyped — byte-identical to what evals/run.py sent.

Full promptThe prompt in full

You are a weekly demand-forecast exception review. You read one forecast line's week against the DEFAULT exception policy at one review cycle, and you answer with one JSON object and no other text.

You are the weekly demand-forecast exception review for one retailer's forecast lines. It runs on a
WEEKLY cadence and re-reads EVERY line. You are reading ONE forecast line (one item at one location
group) at ONE weekly review.

The line, its variance for this week, the DEFAULT exception policy, and this week's events are
reproduced in the extract below. Apply them exactly as written. You cannot see the earlier review
cycles; what is known about them is stated under "Carried state" and is the only history available
to you. Do not assume anything about earlier cycles beyond it.

⚠︎ THE TOLERANCE BANDS AND THE EXPOSURE THRESHOLDS ARE ILLUSTRATIVE DEFAULTS, NOT ANY REAL
RETAILER'S POLICY. Apply them as printed regardless of whether they look right for the line in
front of you.

THE ONE THING THAT MATTERS MOST HERE: the queue is ranked by FINANCIAL EXPOSURE, never by variance
percent. A 40 pct variance on a slow-moving low-cost line is noise; a 6 pct variance on a
high-value, long-cover line is the week's money. The variance percent is printed on the page and it
is NOT the sort.

How to work it out:
 - FIRST decide whether the line can be ranked at all. If the unit cost is blank on file, or the
   remaining weeks of cover are blank, the verdict is CONTEXT_INCOMPLETE: no exposure is computed,
   the exposure band is EXP_UNDETERMINED, no cause is determined and nothing is queued. THERE IS NO
   DEFAULT UNIT COST AND NO DEFAULT COVER FIGURE (Rule F-6).
 - THEN decide whether this line is already RESOLVED. If the carried state or this cycle's events
   record a rework or a forecast re-base, the verdict is RESOLVED and stays RESOLVED regardless of
   what this week's variance is doing (Rule F-7). Nothing here performs a rework, a re-base, an
   order or a closure -- you are only ever reporting one that has already happened elsewhere.
 - THEN compute the FINANCIAL EXPOSURE: the absolute units the forecast was wrong by, times the
   unit cost on file, times the remaining weeks of cover on file. Read the exposure band off the
   DEFAULT band table (Rule F-1, Rule F-2). Never band it on the variance percent.
 - THEN decide the CAUSE. Cause evidence ACCUMULATES from earlier cycles (see "Carried state") and
   from anything recorded THIS cycle -- in the structured event log or in a planner note. A note
   that describes a cause in ordinary prose counts exactly as much as a logged reason code; the
   planning system catching up later does not change what was already established (Rule F-4). Use
   UNDETERMINED only when the line is exposed and nothing anywhere says why.
 - If the cause is one that EXPLAINS the variance away -- a promotion that ran, a competitor's
   stock-out we absorbed, or a gap in the sales feed -- the verdict is EXPLAINED and work_now is NO
   however large the exposure is. The forecast was not wrong (Rule F-3). An explanation given on an
   earlier cycle still stands.
 - A SUPPLY_CONSTRAINT or a DEMAND_SHIFT is NOT an explanation of that kind. It is a real forecast
   miss with a known driver, and naming it does not take the line off the desk. Report the cause
   and carry on down this list.
 - If the exposure band is EXP_NONE, the verdict is IN_TOLERANCE and the cause is NONE, however
   large the variance percent is. Nothing is queued.
 - THEN check for a PERSISTENT BIAS. If the variance has run the SAME direction, on an exposed
   line, for three consecutive reviews counting this one, the verdict is PERSISTENT_BIAS: the
   finding is that the forecast needs re-basing rather than that this week is an exception (Rule
   F-5). The carried state tells you how many consecutive cycles it has already run and in which
   direction; this cycle counts as one more only if this week's variance leans the same way.
 - OTHERWISE: from EXP_MED upward the verdict is WORK_NOW -- the line belongs at the top of this
   cycle's exposure-ranked queue. At EXP_LOW the verdict is WATCH.
 - THEN decide the OWNER. Start from the planner of record on file, but a planner note recording a
   handover to somebody else, in this cycle or an earlier one (see "Carried state"), supersedes it.
 - FINALLY decide "work_now": YES only if the line needs a planner (WORK_NOW or PERSISTENT_BIAS)
   AND the exposure band is HIGHER than any band this review has already raised it at (see
   "Carried state"). Otherwise NO. A line already on somebody's desk at EXP_HIGH is not re-raised
   at EXP_HIGH; a line that has climbed from EXP_HIGH to EXP_CRITICAL is (Rule F-8).

Answer with a single JSON object and nothing else:

{"verdict": "IN_TOLERANCE|WORK_NOW|WATCH|EXPLAINED|PERSISTENT_BIAS|RESOLVED|CONTEXT_INCOMPLETE",
 "exposure_band": "EXP_NONE|EXP_LOW|EXP_MED|EXP_HIGH|EXP_CRITICAL|EXP_UNDETERMINED",
 "cause": "PROMOTION|COMPETITOR_STOCKOUT|FEED_GAP|SUPPLY_CONSTRAINT|DEMAND_SHIFT|UNDETERMINED|NONE",
 "owner": "<the current planner's name>",
 "work_now": "YES|NO",
 "rationale": "one sentence, naming the exposure figure you computed, the band it falls in, and the
   explanation or history you relied on"}

Precedence, applied in this order: CONTEXT_INCOMPLETE if the line cannot be ranked (Rule F-6); then
RESOLVED (Rule F-7); then EXPLAINED if the cause explains the variance away (Rule F-3); then
IN_TOLERANCE if the exposure band is EXP_NONE; then PERSISTENT_BIAS (Rule F-5); then WORK_NOW from
EXP_MED upward; otherwise WATCH. "exposure_band" is EXP_UNDETERMINED only when the verdict is
CONTEXT_INCOMPLETE.

Carried state
----------------------------------------------------------------
As at the previous review, this line's unit cost was settled at GBP 121.19 and its remaining cover at 15.8 weeks. No cause has been established for this line on any earlier cycle. Counting up to and including the previous review, the variance has run UNDER (actual below forecast) for 1 consecutive exposed cycle(s). This review has already raised this line at EXP_CRITICAL (before anything else) -- do not raise it again for that band or lower (Rule F-8). The planner as last settled is Yuki Tanaka -- carried forward unless this cycle's record hands the line to somebody else. It was last reported WORK_NOW.

Forecast line review extract
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
Every field below is invented. This is a generated demand-forecast line review extract
for an AI use-case kit; it reproduces no retailer, no supplier, no store, no real product
and no real person. The tolerance bands and exposure thresholds are ILLUSTRATIVE
DEFAULTS.

Forecast Line
----------------------------------------------------------------
Line reference                 : FL-0038
Item                           : Percale duvet cover, king
Item code                      : ITM-77152
Category                       : Bedding
Location group                 : NORTH METRO
Planner of record              : Yuki Tanaka
Unit cost                      : GBP 121.19
Remaining weeks of cover       : 15.8 weeks

Variance Detail
----------------------------------------------------------------
Forecast, this week            : 11,176 units
Actual, this week              : 11,638 units
Variance                       : +4.13 pct
Units of forecast error        : 462 units OVER forecast

The financial exposure this line carries is NOT printed on this report and never has
been. It is the units of forecast error, times the unit cost above, times the remaining
weeks of cover above. Every input is on this page.

Exception Policy (Default)
----------------------------------------------------------------
The table below is an OPERATOR-TUNABLE DEFAULT, not a real retailer's actual policy -- no
documented tolerance percentage, exposure threshold or queue depth exists for this row.
Applied exactly as printed. The queue is ordered by FINANCIAL EXPOSURE, never by variance
percent.

  EXP_NONE       GBP 0 to under GBP 5,000           nothing is owed on it
  EXP_LOW        GBP 5,000 to under GBP 25,000      watched -- nobody works it
  EXP_MED        GBP 25,000 to under GBP 100,000    a planner works this line this cycle
  EXP_HIGH       GBP 100,000 to under GBP 500,000   a planner works this line this cycle
  EXP_CRITICAL   GBP 500,000 and above              a planner works this line this cycle

For reference only -- THE LEGACY SORT THIS REPORT HAS ALWAYS USED, and NOT the policy
above. It bands a line on its absolute variance percent, which says nothing about how
much money is standing behind it:

  EXP_NONE       0 pct to under 5 pct
  EXP_LOW        5 pct to under 10 pct
  EXP_MED        10 pct to under 20 pct
  EXP_HIGH       20 pct to under 40 pct
  EXP_CRITICAL   40 pct and above

Approval authority             : NOT DEFINED. Nothing in this kit edits a forecast,
                                 places or amends an order, closes an exception, changes
                                 a safety stock, or writes anything back to a planning
                                 system.

Rule F-1  A line's place in the queue is read off its FINANCIAL EXPOSURE, not off its variance
          percent. Exposure is the units the forecast was wrong by, times the unit cost on file,
          times the weeks of cover on file. Both figures are printed above. A 40 pct variance on a
          slow-moving low-cost line and a 6 pct variance on a high-value line are not comparable
          and the percent does not rank them.
Rule F-2  The exposure band is read off the DEFAULT band table exactly as printed. The table is an
          operator-tunable placeholder, not any retailer's actual policy. From EXP_MED upward the
          line belongs on a planner's desk this cycle; EXP_LOW is watched; EXP_NONE is left alone
          however large the percentage is.
Rule F-3  A variance the record EXPLAINS is not a forecast failure and must not be re-queued. A
          promotion that ran, a competitor's stock-out we absorbed, or a gap in the sales feed all
          mean the forecast was not wrong. A supply constraint or a demand shift is a real miss
          with a known driver -- naming it does NOT take the line off the desk.
Rule F-4  Explanations, resolutions, ownership and variance direction ACCUMULATE across review
          cycles. "Events In This Window" shows only what was recorded since the PREVIOUS review;
          anything recorded on an earlier cycle is not repeated and is known only through the
          carried state. An explanation given two cycles ago still stands this cycle.
Rule F-5  A variance running the SAME direction and outside the tolerance band for three
          consecutive reviews is PERSISTENT_BIAS, not a fresh exception. The finding is that the
          forecast needs re-basing, and it cannot be reached from one cycle's page.
Rule F-6  If the unit cost is not on file, or the weeks of cover are not on file, the reading is
          CONTEXT_INCOMPLETE: no exposure is computed, no band is assigned, the line is not ranked
          and nothing is queued. THERE IS NO DEFAULT UNIT COST AND NO DEFAULT COVER FIGURE.
Rule F-7  Once the record says the line has been reworked or the forecast re-based, the line is
          RESOLVED and stays RESOLVED on every later cycle, whether or not this week's variance has
          settled yet. Nothing here performs the rework, the re-base, the order or the closure; it
          only reports one that has already happened elsewhere.
Rule F-8  "work_now" is about THIS cycle's queue, not about the line's state. It is YES only when
          the line needs a planner and the exposure band is HIGHER than any band this review has
          already put it in front of somebody at. A line already on the desk at EXP_HIGH is not
          re-raised at EXP_HIGH next week; a line that climbs from EXP_HIGH to EXP_CRITICAL is.

Review Position
----------------------------------------------------------------
Review week ending             : 2026-08-10   (review cycle 2 of this line)
Review cadence                 : weekly -- every Monday's forecast review
Previous review                : 2026-08-03
Next review                    : 2026-08-17
Lines in this week's review    : 40

Events In This Window
----------------------------------------------------------------
Everything recorded against this line between the previous review and this one.
THIS REVIEW WEEK ONLY -- an event recorded in an earlier week is not repeated here.

No events were recorded against this line in this review week.

Planner Notes
----------------------------------------------------------------
Supplier lead time on this line is long enough that anything decided this week lands
after the cover on file runs out.

Raw responseThe raw response

{"verdict": "WORK_NOW", "exposure_band": "EXP_CRITICAL", "cause": "UNDETERMINED", "owner": "Yuki Tanaka", "work_now": "NO", "rationale": "Computed exposure is 462 × £121.19 × 15.8 = £884,638.52, which falls in EXP_CRITICAL; with no cause explaining the variance and no persistent bias, the verdict is WORK_NOW, but the line has already been raised at EXP_CRITICAL this review so it is not re-raised."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Fix the forecast misses that cost a retailer most — 120 forecast line extracts. One model answered, and every answer was then graded Three different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate — a projection, never a bill and never a vendor's claim about this workload.

Rates checked 2026-08-18. The provider that actually ran all 282 calls is kept out of these tables per this estate's naming rule, so no figure here is a bill.

The gradersThree ways to grade

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

The three free floors, the stub, the cadence analysis and every pre-run check cost $0.00 and no calls. No run was discarded on this kit.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS. 171467 of 184169 output tokens on the scored run were provider-side reasoning left at the default.
• THE PROMPT IS MOSTLY FIXED TEXT — the band table, the exposure formula and the instruction are identical on every call, so a provider with prompt caching would price this very differently.

Your volumeWhat it costs at your volume

LINEAR IN LINES x CYCLES. Ten times the assortment is ten times the calls at the same per-reading cost; nothing amortises. The sublinear lever this kit does NOT implement is skipping lines inside tolerance with no events.

Model choiceWhy this model

It is the tier the shared connection was already pointed at, and the kit's claim is that swapping it is .env plus one more run.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-query average, this run

The codeEvery file, and what it is for

11 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/exception.pythe exposure rule and state machine — a swap seam

The exposure formula, the DEFAULT bands, the persistent-bias test and step() — which decides the next carried state from the answer key's inputs, never from a model reply. It is also the answer key.

You change it to: The review dates. Widening the cycle divides the bill and widens the window in which an exposed line goes unworked.

# The forecast-exception rule as arithmetic. Pure code, no model, standard library only.

EXP_NONE = "EXP_NONE"
EXP_LOW = "EXP_LOW"
EXP_MED = "EXP_MED"
EXP_HIGH = "EXP_HIGH"
EXP_CRITICAL = "EXP_CRITICAL"
EXP_UNDETERMINED = "EXP_UNDETERMINED"
BANDS = (EXP_NONE, EXP_LOW, EXP_MED, EXP_HIGH, EXP_CRITICAL)
BAND_INDEX = {b: i for i, b in enumerate(BANDS)}
DEFAULT_EXPOSURE_BANDS = (

src/state.pythe carried state

What the previous cycles settled: the established cause, the run of same-direction cycles, and the resolved flag. The bias count is what no single page contains.

# The carried state -- the thing that makes this a monitor and not another weekly variance report.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEFAULT_PATH = os.path.join(HERE, "data", "state.json")
def load(path=DEFAULT_PATH):
def save(store, path=DEFAULT_PATH):
def for_line(store, line_id):
BAND_WORDS = {
CAUSE_WORDS = {
DIR_WORDS = {exception.OVER: "OVER (actual above forecast)",
def describe(state):

src/segment.pythe section splitter

Splits the page into its 8 named sections; check_labels asserts all 8 parse in all 120 documents before a run may spend.

# Split a forecast-line review extract into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Forecast Line", "Variance Detail", "Exception Policy (Default)",
RULE = "-" * 64
def split(text):
def render(secs):

src/select.pythe send filter — a swap seam

Decides which sections reach the provider. Planner Contact is mapped by no field and therefore never sent.

You change it to: SECTION_HINTS and NEVER_SENT.

# Pick which sections of a forecast-line review extract are sent. Pure code -- the last

BANNER = "Synthetic Record"
LINE = "Forecast Line"
VARIANCE = "Variance Detail"
POLICY = "Exception Policy (Default)"
POSITION = "Review Position"
EVENTS = "Events In This Window"
CONTACT = "Planner Contact"
NOTES = "Planner Notes"
NEVER_SENT = (CONTACT,)

src/prompt.pythe prompt

Three parts: the instruction and JSON shape, the carried state, the extract. The stateless control replaces exactly one line.

# Assemble the one prompt this kit sends. No template engine, no framework -- string

INSTRUCTION = """\
def build(text, carried, stateless=False):

src/adapters/__init__.pythe model call

Raw HTTP, stdlib only, bounded retry; the shared daily call cap is checked here.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
def _post(url, headers, payload, timeout=120):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}
THINKING_OFF = {"type": "disabled"}
def complete(cfg, system, user, max_tokens=1024, thinking=None):

src/review.pythe reader

One line, one cycle, one call. The published token ceiling lives here.

# One forecast line, one weekly review, one model call. The runtime half of the kit.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SYSTEM = ("You are a weekly demand-forecast exception review. You read one forecast line's week "
MAX_TOKENS = 8000
FIELDS = ("verdict", "exposure_band", "cause", "owner", "work_now")
def documents():
def lines():
def load_doc(doc_id):
def position_of(text):

evals/scoring.pythe scorer

Exact match per cell. The exposure ranking and the false-raise rate are separate binaries over separate denominators, never averaged.

# Score a run against the answer key. Pure code, exact match per cell. No model grades anything.

NEEDS_A_PLANNER = ("WORK_NOW", "PERSISTENT_BIAS")
FIELDS = ("verdict", "exposure_band", "cause", "owner", "work_now")
def _pct(n, d):
def score(records, golds):
def _protection(records, golds):

evals/baseline.pythe three free floors

variance-only, exposure-arithmetic and structured-log-strong — $0.00 each, and two of them beat the model on the ranking column.

# THE FREE FLOORS. Three of them, none a strawman, all of them GBP 0.00.

MODES = ("variance-only", "exposure-arithmetic", "structured-log-strong")
def _section(text, name, nxt, flat=False):
def _facts(text):
def _finish(verdict, band, cause, owner, work_now, why):
TAG_PROMO = re.compile(r"TAG: PROMO ")
TAG_STOCKOUT = re.compile(r"TAG: STOCKOUT ")
TAG_FEEDGAP = re.compile(r"TAG: FEED-GAP ")
TAG_RESOLVED = re.compile(r"TAG: RESOLVED ")
TAG_CAUSE = ((TAG_PROMO, E.PROMOTION), (TAG_STOCKOUT, E.COMPETITOR_STOCKOUT),

src/app.pythe local UI server

http.server, stdlib. Renders with no key.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
PORT = int(os.environ.get("PORT", "8215"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-forecast-exception")
GOLD_ROWS = {r["doc_id"]: r for r in
def carried_for(doc_id):
def arithmetic(pos):
class H(BaseHTTPRequestHandler):

ui/app.jsthe local UI client

Hand-written JS, no framework. Prints the model column beside the free floor.

# 

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 3099 input and 1534 output tokens per reading (one forecast line, at one weekly review), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

MEASURED, not asserted. Planner Notes is this kit's one injection surface — free text a planner types — and one of the shipped notes is instruction-shaped by design. It was fired as its own probe rather than named.

API_KEY is read from .env (gitignored) or the real environment; never logged, never placed in a prompt, and redacted from any provider error message the local UI passes to the browser.

The experimentWe DID attack it — one sentence, on every reading where it could matter

An indirect prompt injection needs a field somebody outside the process can write into, and this kit's candidate is Planner Notes. Rather than report the rate at which the shipped corpus happened to place the note next to a due raise — a denominator of four — evals/injection.py forces it onto every such reading and re-fires them with everything else held identical. Both were run for real on 2026-08-25.

One boundary is measured in both directions here: the injection probe forces the condition rather than waiting for it, and the privacy guard is red-proven by reproducing the schema-change condition that reaches its fallback.

The result0 of 32 raises suppressed suppressed by the instruction-shaped note — measured, not assumed.

One phrasing, one model, one corpus, 32 trials — every reading where suppression was even possible.

HonestyWhat this does not prove

• Any other injection. A note asking the model to report RESOLVED, or to name a different planner, is a different experiment and was not run.
• Whether the result holds on another model tier.
• Whether an injection placed in Planner Contact would have any effect — by construction it cannot reach the prompt at all.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never edit a forecast, place an order or close an exception — and never present the tolerance bands or the exposure thresholds as any real retailer's policy.

Stated to the model on every call in src/prompt.py's INSTRUCTION block, and enforced mechanically by evals/check_labels.py's banned-code-path scan before any run may spend.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is a PROMPT RULE PLUS A STATIC SCAN, not a runtime enforcement layer. Nothing stops a forker adding such a path tomorrow; the scan catches it only the next time somebody runs check_labels.py, which is a manual step.
• The injection result is ONE SENTENCE against ONE model on ONE corpus. It is not a resistance rate for prompt injection in general.

WatchedWhat is watched, and why that one

The kit measures 95 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

3 measured by the latest run 92 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on floor, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · with the model in the path — 2 runs. Columns here are only ever compared with each other.

not a time series No two of these 2 runs measured the same system — they differ on stateless — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 19 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 3 chips that all say so.

DeviationsWhat deviated

0 breaches across 7 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

Nothing fires yet. Not one of the 27 bands on this kit names a consequence, so there is no row to print. Which is the honest answer to the heading: a band that nothing is wired to is a number somebody has to notice, and this kit has only those. An empty table would have said the same thing while looking like a rendering failure.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The guardrail scan is a MANUAL step, run before each spend, not a hook. It ran before every paid run and passed at 0 each time. The injection probe is a ONE-OFF: it measured 0.0 pct suppression on 32 readings on 2026-08-25 and nothing re-runs it, so that figure ages from the day it was taken.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, no framework dependency, and a prompt anyone can read end to end. A framework abstraction would own the retrieval step — there is none here — and the memory layer, which is already the entire surface of src/state.py.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: the reader -> src/prompt.py -> src/adapters -> evals/scoring.py. No branching, no tool-calling and no agent loop.

The other sideWhat a framework costs you

• Swapping providers means editing the PROVIDERS dict by hand. In exchange the prompt has no hidden templating layer and requirements.txt stays empty.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-forecast-exception on the fast tier, with the carried state, 2026-08-25. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 4 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-forecast-exception-varianceonly, b001-forecast-exception-exposurearithmetic, b002-forecast-exception-structuredlogstrong recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the population is re-read whole on each scheduled run, and the prior run's carried state is what makes a reading a CHANGE.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe verdict, exposure band, cause, planner and work-now call, per reading, exact match against the computed answer key

whether each of the 5 answered fields equals the answer key, per reading

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings. No model grades anything.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 120 per field. exposure_ranking_accuracy_pct is scored separately over its own denominator.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/exception.step() from the same planted inputs the corpus builder wrote onto the page.

No true/false rates for this grader. It records 5 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No threshold was swept: exact match has no tunable. Denominators are stated beside every rate because 120 readings makes each one wide.

Cadence: once per run; free

The decisionWhen to reach for it

In one lineThe queue a planner works first, ranked by exposure rather than by variance

whether the arm put a genuinely work-worthy line in WORK_NOW

MethodHow the test was run

evals/scoring.py, same pass as the exact-match grader.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

recall over the 49 readings whose gold verdict is WORK_NOW; the false direction is reported separately over the other 71.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/exception.step() from the same planted inputs the corpus builder wrote onto the page.

No true/false rates for this grader. It records 2 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No band finer than one row: 49 WORK_NOW readings means one miss moves the rate by 2.04 points.

Cadence: once per run; free

The decisionWhen to reach for it

In one lineVariances raised where the truth was already explained, or inside tolerance

whether the arm put work on a planner's desk that did not belong there

MethodHow the test was run

evals/scoring.py, same pass.

Every grader on these pages scored the same 120 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

The formulaWhat it computes

false raises over the 88 quiet readings. The explained slice is reported on its own: of the 18 readings whose variance was already explained, how many were left alone.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/exception.step() from the same planted inputs the corpus builder wrote onto the page.

No true/false rates for this grader. It records 2 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? No sweep: work_now is a binary the arm emits, not a thresholded score.

Cadence: once per run; free

The decisionWhen to reach for it