The business caseThe problem this solves

A declaration of interests is a form somebody fills in about themselves, and the part that matters most is the part where it says nothing. Two different nothings sit on the same page: a NIL RETURN — "- NIL -", "None.", "NONE DECLARED", "this section does not apply to me" — where somebody has put their name to the absence, and a SILENCE, where the heading is printed with nothing under it or the section is not on the form at all. Collapse the second into the first and a hole in a declaration is on the record as a clean nil return; every conflict check run against those rows afterwards passes over it and reports nothing. The forms also arrive in three revisions that share no layout — typed prose, labelled key/value blocks, pipe-delimited tables — and the answer must not change with the layout. The manual pass over one return: reading it in whichever revision it arrived in, deciding which section each item belongs to, normalising the entity name so it can be matched against a client or restricted list later, and recording an absence in a way that says whether anybody signed it. It does NOT replace the conflict check itself — this kit decides nothing about whether anything is a conflict and has no such output.

The inputThe actual declaration forms

The corpus is 60 declaration forms, 0.05 MB (txt 60). A declaration of interests is a document about a named private individual and their family, so there is no public set and no prospect of one. What this corpus buys instead is a controlled cross: the SAME planted facts printed in three revisions that share no layout, so the by-revision column measures reading rather than the mix of the sample. It also forces the two distinctions the task turns on — 51 of 60 forms carry BOTH a nil return and a silence, so no arm can score well by picking one absence class and applying it everywhere.

DECLARATION OF INTERESTS  |  FORM ID-1 (REV. 2025)
REF PR-27979 | DECLARANT Emeric Skarsgaard-Hume | ROLE Associate Director | UNIT Assurance - Group 4
PERIOD 01/01/2025 - 12/31/2025 (MM/DD/YYYY) | DECLARED 06/03/2025 (MM/DD/YYYY)
VALUE BANDS:  A $1 - $5,000  |  B $5,001 - $25,000  |  C $25,001 - $50,000  |  D $50,001 - $250,000

[1] SECURITIES AND OTHER FINANCIAL HOLDINGS
ENTITY | NATURE | HELD BY | VALUE | PCT
Ferrisgate Analytics Inc | ordinary shares | self | $22,100.00 | -

[2] DIRECTORSHIPS AND OFFICES HELD
ENTITY | NATURE | HELD BY | VALUE | PCT
NONE DECLARED | | | |

[3] OUTSIDE ENGAGEMENTS, EMPLOYMENT AND APPOINTMENTS
ENTITY | NATURE | HELD BY | VALUE | PCT
Drossmere Publishing Ltd | consultancy engagement | self | $9,000.00 | -

END OF FORM

The outcomeWhat a good result looks like

One row per declared interest, plus exactly one coverage row for every section that declared nothing — never two, never none. Each row carries the section, the outcome, the holder, the entity as printed AND normalised, the interest type, the amount as exact cents or a low/high range or an explicit not_stated, the percentage as an exact decimal string, and how it is held. 60 forms produced 297 rows on this corpus.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Declarations that arrive in ONE house layout — a form somebody fills in on screen, or a single revision of a paper return — the free rules floor alone
  evals/baseline.py already reads 60.0% of forms exactly and 94.3% precision on row identity for $0.00, and eleven added lines take it to 98.3% — the same as the paid arm, class for class. A layout-anchored reader is the right tool for an anchored layout.
• Free-typed returns in many house styles, or a register that has accumulated layouts nobody controls — the model, with the pure-code rebuild behind it
  The floor's per-revision column is the argument: as shipped it loses 13.1 recall points moving from the 2025 table to the 2019 typed return (93.9% to 80.8%) while the model loses nothing. The floor's weakness is layout-specific by construction; every new layout is another set of patterns somebody has to write.
• You need to know WHICH sections nobody answered, so you can chase them — either arm — this is what the kit is for and both arms do it
  NOT_STATED is a first-class outcome with its own row, and notstated_read_as_nil is 0 on every arm measured. NIL_DECLARED recall is 100.0% on the model and 89.7% on the floor as shipped (100.0% patched).

And where nothing here is good enough:

• You want the rows to decide something — clear somebody, flag a conflict, approve a return — neither. This kit has no such output.
  It produces the rows a person then checks something against. There is no verdict, no flag and no endpoint that adds one.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-31. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-31 — r001-holdings-extract. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — RE-RUN IN THIS CAPTURE PASS, 2026-08-31, in a scratch copy of the kit with no key configured. tools/build_corpus.py rebuilt data/ byte-identically under PYTHONHASHSEED 0, 1, 97531 and 424242 — four seeds, zero differing files, including data/gold.jsonl. python3 -m evals.run --run-id b000-holdings-extract-rules --floor rules reproduced the committed free-floor result exactly (36 of 60 forms row-exact, identity 94.3/89.2). Nothing in this paragraph needed a key or a network. NOT re-run: the paid arm — r001 is replayed from results/cache-r001-holdings-extract.jsonl and no call was made in this pass.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. Measured on the fast tier — one declaration form — one call, end to end, including provider-side reasoning tokens, on a shared connection. 941.3 s of latency was recorded across the 60 calls against 202.7 s of wall clock, which is 5 concurrent workers (evals/run.py, EVAL_WORKERS default 5) and matches what the README claims. The pure-code rebuild adds no call and no measurable latency.

Current processWhat it replaces

The manual pass over one return: reading it in whichever revision it arrived in, deciding which section each item belongs to, normalising the entity name so it can be matched against a client or restricted list later, and recording an absence in a way that says whether anybody signed it. It does NOT replace the conflict check itself — this kit decides nothing about whether anything is a conflict and has no such output.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one declaration form is

No train/test split, because nothing is trained or tuned, and NO SEGMENTATION — one form goes whole into one call, 913 bytes at p50. The design is a full cross instead: 20 constructions × 3 revisions, 20 forms per revision, so the by-revision column is a clean read on layout with the planted facts held constant.

SetupWhat the setup figure measured

THERE IS NO INDEX AND NO RETRIEVAL STEP. The form, the rules and the schema go whole into one prompt — 10702 characters for HX-0001, of which the form is 789 and the stable prefix is 9913. Nothing is embedded, stored or looked up.

LicenceLicence

MIT — and the MIT text is in the repository's LICENSE-PUBLIC file, NOT in its root LICENSE, which governs the repository itself and grants a reader nothing. The corpus is generated in process from a fixed seed; there is no third-party data here to licence, nothing was scraped, downloaded, transcribed or derived from any real declaration, register or filing.

Bring your ownBring your own declaration forms

Drop your own forms as .txt into data/corpus/, add one row per form to data/forms.json (id, vintage, case, hard, ref, declarant, role, unit, declared_on) and write data/gold.jsonl with one record per form carrying declared_on, declarant and rows. Every row's section, entity_norm and held_via are COMPUTABLE — src/normalise.py::section_for, entity_key and held_via_for — so derive them rather than typing them, and evals/check_labels.py will tell you if you did not.

⚠︎ And what stops being true when you do: ⚠︎ A DECLARATION OF INTERESTS NAMES AN INDIVIDUAL, THEIR FAMILY AND WHAT EACH OF THEM OWNS, AND THE WHOLE FORM REACHES YOUR CONFIGURED PROVIDER VERBATIM. src/prompt.py sends the document unmodified by design — a pre-parse is where 'Band C' quietly becomes $25,001–$50,000 before the model sees it. That is a disclosure decision for whoever holds the register and it is not one a kit can make. Run it against forms you are permitted to disclose, against a redacted extract, or against a local model.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent. The split is in characters, because that is what this run recorded — the billed totals are measured, but nothing apportioned them across the parts, and dividing a measured total by character share would produce three numbers nobody counted.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 10,702 characters assembled, 9,913 are instructions — 93% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The exact four parts sent for HX-0001, the first form of run r001-holdings-extract, replayed from the kit's own src/prompt.py in this capture pass. Character counts are the run record's own (prompt_chars_total 10702) and were reproduced exactly.

Full promptThe prompt in full

You are reading ONE declaration of interests and turning it into checkable rows, so that a
conflict check can be run against them later. You are not deciding whether anything is a conflict,
and you are not judging anybody. You are lifting what the form says -- and, just as importantly,
recording exactly where the form says nothing.

The forms come from three different revisions of the same return and they do not look alike. One
revision is typed prose under numbered headings. One is labelled key/value blocks. One is
pipe-delimited tables. The same declaration can appear in any of the three, and the answer must
not change with the layout.

Three things decide whether this is done correctly:

1. EVERY DECLARED INTEREST BECOMES EXACTLY ONE ROW. Not one row per sentence, not one row per
   printed line -- one row per interest. An interest described across two lines is one row.

2. AN EXPLICIT NIL RETURN IS NOT THE SAME ANSWER AS SILENCE, AND YOU MUST NEVER MERGE THEM. A
   section where the form affirmatively says there is nothing to declare gets a NIL_DECLARED row.
   A section the form does not answer -- its heading printed with nothing under it, or the section
   absent from the form altogether -- gets a NOT_STATED row. One is a person putting their name to
   "I have nothing here". The other is a hole.

3. AN AMOUNT THAT WAS NOT STATED IS NOT AN AMOUNT OF NOTHING. Leave it null and say
   `amount_basis: not_stated`. Never write 0.

Reply with JSON and nothing else, in the shape given at the end.

THE DECLARATION RULES. They are the same for every revision of the form.

--- 1. THE FOUR SECTIONS, AND WHICH ONE A ROW BELONGS TO ---

  securities             Securities and other financial holdings
  directorships          Directorships and offices held
  outside_engagements    Outside engagements, employment and appointments
  family_interests       Interests of close family members

An interest's section is decided by TWO things, in this order:

  a) WHOSE IT IS BEATS WHAT IT IS. If the holder is anyone other than the declarant, the row's
     section is `family_interests` -- whatever kind of interest it is, and whichever printed block
     of the form it appeared under. Some revisions print family holdings under their own heading;
     some annotate them inline in the securities table. The section does not change with that.
  b) Otherwise the section follows the kind of interest:

  directorship           -> directorships
  officer_role           -> directorships
  trusteeship            -> directorships
  consultancy            -> outside_engagements
  outside_employment     -> outside_engagements
  partnership_interest   -> outside_engagements
  debt_security          -> securities
  equity_security        -> securities
  fund_holding           -> securities

--- 2. THE FOUR OUTCOMES ---

  EXTRACTED      the form declares an interest AND names the entity it is in. One row per
                 interest.
  AMBIGUOUS      the form declares an interest in this section but does NOT name the entity --
                 "shares in a number of listed companies through a managed portfolio", "one
                 directorship, company to be confirmed". Report the row with `entity` null and
                 the interest_type you can tell. Do not guess an entity.
  NIL_DECLARED   the form AFFIRMATIVELY states there is nothing to declare in this section:
                 "None.", "- NIL -", "NONE DECLARED", "I hold no directorships", "NIL RETURN".
                 ONE row for the section, with entity, holder and interest_type all null.
  NOT_STATED     the form does not answer this section at all -- the heading is printed and
                 nothing follows it, or the section is missing from the form entirely. ONE row
                 for the section, everything null.

  ⚠︎ SILENCE IS NOT A DECLARATION. A blank section is NOT_STATED, never NIL_DECLARED. A missing
  section is NOT_STATED, never NIL_DECLARED, and never simply left out of the answer.

  Every one of the four sections gets exactly one coverage: either one or more item rows
  (EXTRACTED / AMBIGUOUS), or a single NIL_DECLARED row, or a single NOT_STATED row. Never two of
  those for one section, and never none.

--- 3. WHO HOLDS IT ---

`holder` is one of:

  self                   the person making the declaration
  spouse                 a close family member
  domestic_partner       a close family member
  dependent_child        a close family member
  adult_child            a close family member
  parent                 a close family member
  sibling                a close family member

The forms word relations differently. These are the words they use:

  spouse                 "my spouse", "my wife", "my husband"
  domestic_partner       "my domestic partner", "my civil partner"
  dependent_child        "my dependent child", "my dependent daughter"
  adult_child            "my adult son", "my adult daughter"
  parent                 "my mother", "my father"
  sibling                "my brother", "my sister"

`held_via` is `direct` when the holder is `self` and `family_member` otherwise.

--- 4. WHAT KIND OF INTEREST ---

`interest_type` is one of:

  equity_security        declared as: shares, shareholding, ordinary shares
  debt_security          declared as: loan notes, bonds, debentures
  fund_holding           declared as: fund units, units in a fund
  directorship           declared as: a directorship, non-executive directorship
  officer_role           declared as: company secretary, an office held
  trusteeship            declared as: a trusteeship, trustee of
  outside_employment     declared as: part-time or outside employment
  consultancy            declared as: a consultancy engagement, consultancy services
  partnership_interest   declared as: a partnership interest

--- 5. THE ENTITY NAME, AND THE NORMALISATION RULE ---

`entity` is the entity name EXACTLY as the form prints it, including its legal suffix and anything
in brackets. Do not tidy it.

`entity_norm` is that name put through this rule, which is what makes two forms naming the same
entity differently match each other later:

  1. Decompose accents away and lower-case everything.
  2. Replace "&" with " and ".
  3. Delete anything inside round or square brackets, brackets included.
  4. Replace  .  ,  '  "  -  /  with a space.
  5. Collapse runs of spaces; trim.
  6. While the LAST word is one of these legal forms, drop it and look again:
     ab  ag  as  bv  co  company  corp  corporation  gmbh  inc  incorporated  kft  kk  limited  llc  llp  lp  ltd  nv  oyj  plc  pte  pty  sa  sarl  sas  spa
  7. While the LAST word is one of these, drop it and look again:
     group  holding  holdings  international  worldwide
  8. Collapse spaces again; trim.

  Worked: "Halcyon Meridian Group PLC" -> "halcyon meridian".
          "Halcyon Meridian Group plc (formerly Ostreme Holdings)" -> "halcyon meridian".
          "Sable & Thrun Limited" -> "sable and thrun".
          "Quillonhurst Capital Partners LLP" -> "quillonhurst capital partners".

  Only the FINAL word is ever dropped, and only when it is in one of those two lists. Nothing else
  is removed: "capital", "partners", "ventures", "trust" and "fund" all stay.

--- 6. AMOUNTS AND PERCENTAGES ---

`amount_basis` is one of:

  exact                  
  range                  
  not_stated             

  exact       one figure is stated. Put it in `amount_cents` as INTEGER CENTS ($12,450.00 is
              1245000). Leave `amount_low_cents` and `amount_high_cents` null.
  range       a range or a value BAND is stated. Put the two ends in `amount_low_cents` and
              `amount_high_cents` as integer cents, and leave `amount_cents` null. When the form
              states only a band letter, resolve it against the VALUE BANDS line printed on that
              same form.
  not_stated  the form gives no value at all. All three amount fields are null.

  ⚠︎ `not_stated` IS NOT ZERO AND NEVER BECOMES ZERO.

`percent` is the stated share, as an exact decimal STRING with no percent sign: "4.5", "0.75",
"12.5". Null when no percentage is stated. Do not convert an amount into a percentage or the
other way round.

--- 7. THE DATE ---

`declared_on` is the date the declaration was made, as YYYY-MM-DD, read from the form's own
declaration/date line. The revisions write it as "11-Mar-2019", as "2022-03-11", and as
"03/11/2025 (MM/DD/YYYY)". It is a property of the FORM, so it is reported once at the top of the
reply and not repeated on every row.

THE DECLARATION, verbatim:

DECLARATION OF INTERESTS  |  FORM ID-1 (REV. 2025)
REF PR-27979 | DECLARANT Emeric Skarsgaard-Hume | ROLE Associate Director | UNIT Assurance - Group 4
PERIOD 01/01/2025 - 12/31/2025 (MM/DD/YYYY) | DECLARED 06/03/2025 (MM/DD/YYYY)
VALUE BANDS:  A $1 - $5,000  |  B $5,001 - $25,000  |  C $25,001 - $50,000  |  D $50,001 - $250,000

[1] SECURITIES AND OTHER FINANCIAL HOLDINGS
ENTITY | NATURE | HELD BY | VALUE | PCT
Ferrisgate Analytics Inc | ordinary shares | self | $22,100.00 | -

[2] DIRECTORSHIPS AND OFFICES HELD
ENTITY | NATURE | HELD BY | VALUE | PCT
NONE DECLARED | | | |

[3] OUTSIDE ENGAGEMENTS, EMPLOYMENT AND APPOINTMENTS
ENTITY | NATURE | HELD BY | VALUE | PCT
Drossmere Publishing Ltd | consultancy engagement | self | $9,000.00 | -

END OF FORM


Reply with JSON and nothing else, exactly this shape:

{"declared_on": "<YYYY-MM-DD, from the form's own declaration date line>",
 "declarant": "<the declarant's name, exactly as printed>",
 "rows": [
   {"section": "securities" | "directorships" | "outside_engagements" | "family_interests",
    "outcome": "EXTRACTED" | "NIL_DECLARED" | "NOT_STATED" | "AMBIGUOUS",
    "holder": "self" | "spouse" | "domestic_partner" | "dependent_child" | "adult_child" | "parent" | "sibling" | null,
    "held_via": "direct" | "family_member" | null,
    "entity": "<the entity name exactly as the form prints it, or null>",
    "entity_norm": "<that name put through the normalisation rule, or null>",
    "interest_type": "equity_security" | "debt_security" | "fund_holding" | "directorship" | "officer_role" | "trusteeship" | "outside_employment" | "consultancy" | "partnership_interest" | null,
    "amount_basis": "exact" | "range" | "not_stated",
    "amount_cents": <integer cents, or null>,
    "amount_low_cents": <integer cents, or null>,
    "amount_high_cents": <integer cents, or null>,
    "percent": "<decimal string such as 4.5, or null>"}
 ]}

There must be at least one row for EVERY one of the four sections. A section with nothing declared
gets exactly one row -- NIL_DECLARED if the form says so, NOT_STATED if the form is silent.

Raw responseThe raw response

{"declared_on": "2025-06-03", "declarant": "Emeric Skarsgaard-Hume", "rows": [{"section": "securities", "outcome": "EXTRACTED", "holder": "self", "held_via": "direct", "entity": "Ferrisgate Analytics Inc", "entity_norm": "ferrisgate analytics", "interest_type": "equity_security", "amount_basis": "exact", "amount_cents": 2210000, "amount_low_cents": null, "amount_high_cents": null, "percent": null}, {"section": "directorships", "outcome": "NIL_DECLARED", "holder": null, "held_via": null, "entity": null, "entity_norm": null, "interest_type": null, "amount_basis": "not_stated", "amount_cents": null, "amount_low_cents": null, "amount_high_cents": null, "percent": null}, {"section": "outside_engagements", "outcome": "EXTRACTED", "holder": "self", "held_via": "direct", "entity": "Drossmere Publishing Ltd", "entity_norm": "drossmere publishing", "interest_type": "consultancy", "amount_basis": "exact", "amount_cents": 900000, "amount_low_cents": null, "amount_high_cents": null, "percent": null}, {"section": "family_interests", "outcome": "NOT_STATED", "holder": null, "held_via": null, "entity": null, "entity_norm": null, "interest_type": null, "amount_basis": "not_stated", "amount_cents": null, "amount_low_cents": null, "amount_high_cents": null, "percent": null}]}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Declared interests, out of a form and into rows — 60 declaration forms. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

No model grades anything. Two rows are the SAME row exactly when five fields match — section, outcome, holder, entity_norm, interest_type — compared as a MULTISET, no fuzzy matching, no partial credit. entity_norm is IN the identity because a row whose entity does not normalise to the right key cannot be matched against a client list, a restricted list or anybody else's declaration later, which is the entire reason the row exists. A second grade adds every value field: amount_basis, the three amount fields, percent, held_via and declared_on.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate — a projection onto a rate card, not a bill anyone received.

Rates checked 2026-08-23. The provider that actually ran every call here is kept off this page per the series rule. Its real spend, its own dated card and its peak/off-peak split are in the kit's own result file for anybody who forks it.

Grading unitWhat the grading figure prices

Scoring is pure Python over committed JSON — 60 forms and 297 rows in well under a second, no key, no network, $0.00. Re-scoring the paid arm from its committed cache costs nothing too, and did: this run's usd block was re-derived with API_KEY= python3 -m evals.run --run-id r001-holdings-extract --resume --rescore, which the record stamps as rebuilt_rederived_from_cache: True.

The gradersOne way to grade, and why it is the only one

⚠︎ THE NULL BASELINE IS ZERO HERE AND THAT IS REAL, NOT A PLACEHOLDER. There is no majority answer to guess: a form's answer is a SET of rows, and answering the most common single outcome (NIL_DECLARED, 87 of 297 rows) for every section of every form scores 0 forms exactly right, because 51 of 60 forms carry both a nil return and a silence and 57 carry at least one extracted interest. The floor that matters is not the null one — it is evals/baseline.py, and the honest reading of this kit is in Business.not_good_enough.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-23. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingGrading is free; the bill is the one paid run — and it was re-priced from cache for $0.00.

The scorer, the label gate, the corpus generator, the rules floor and the stub are pure code and cost $0.00 against any result already on disk. The only spend is the 60-call scored run. ⚑ AND ITS OWN usd BLOCK WAS DERIVED FOR FREE AFTER THE FACT: RATE_CARDS in evals/run.py was empty when the run fired, so the record carried usd: null with the reason attached rather than a guess; the card was added and the run re-priced with API_KEY= python3 -m evals.run --run-id r001-holdings-extract --resume --rescore — zero calls, raw answers untouched, every score byte-identical. ⚠︎ ONE CONSEQUENCE FOR ANYBODY READING THE RECORD: wall_seconds is now 0.0 because it times the process that WROTE the file, and that process was the rescore. The run's real wall clock is wall_seconds_of_calls, 202.7 s.

Cost driversWhat actually moves the bill

• Provider-side reasoning: 95053 of 124985 output tokens (76.1%) never reach the answer, and the output side is 81.0% of the projected bill.
• One call per form, always. Nothing batches and nothing is skipped — a form with four NIL rows costs a call exactly like a form with eight interests.
• The stable prefix is what keeps the input side cheap: the system role, the rules and the schema are 9913 of 10702 prompt characters and 66.7% of input tokens were billed as cache hits.

Your volumeWhat it costs at your volume

Linear in forms and flat in form size. Ten times the forms is ten times the calls at the same per-form cost, because a declaration is small (mean 934.0 bytes, p95 1230) and nothing accumulates between calls. What is NOT linear is variance: per-form cost already spans $0.000635 to $0.005578 (8.8×) on forms of almost identical length, because the reasoning budget re-rolls per call, so a ten-times run's total is predictable but any single form's is not.

Model choiceWhy this model

It is the tier the estate runs, so the figure compares with every sibling kit. The finding here is not about the model: it read 59 of 60 forms exactly and its one miss is a dropped letter in a company name. It is about the corpus — free code reaches the same score.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the 60-form scored run (r001-holdings-extract, the fast tier), all 60 of which answered: 176045 input tokens (117504 of them prefix cache hits, 66.7%) and 124985 output tokens (95053 reasoning, 76.1%). 0 failures and 0 truncations, so these totals cover every call that was billed — unlike a run that loses a call, this kit's money and its denominator are the same 60.

The codeEvery file, and what it is for

12 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Five of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/prompt.pyprompt assembly

Four parts in a fixed order — the system role (1520 chars), the declaration rules (7055), the form VERBATIM (789 for HX-0001), the JSON shape (1338). It renders src/normalise.py's own docstring for the eight-step entity rule so the prompt and the function cannot drift, and it deliberately does NOT send the form's metadata record: the revision is the variable being measured and passing it would leak the answer.

# The assembled prompt, in one place, in the order it is sent.

def _bullets(pairs):
RELATION_WORDS = [
SYSTEM = """You are reading ONE declaration of interests and turning it into checkable rows, so that a
RULES = """THE DECLARATION RULES. They are the same for every revision of the form.
SCHEMA = """Reply with JSON and nothing else, exactly this shape:
def build(text, form=None):
def render(parts):
VERDICTS = S.OUTCOMES
VERDICT_MEANINGS = {

src/schema.pythe vocabularies — a swap seam

One copy of the four sections, four outcomes, seven holders, nine interest types, three amount bases and the type→section map, imported by the prompt, the parser and the scorer. VERDICTS in src/prompt.py is an alias of S.OUTCOMES rather than a retyped list, which is why the site's verdict panel reads it by AST alias rather than by literal.

You change it to: Sections, outcomes, holders, interest types, amount bases and the type→section map. One edit in one file; the prompt renders them, the parser enforces them and the scorer splits on them.

# The vocabularies, in one place. No logic, no I/O -- just the closed sets everything else reads.

SECTIONS = ("securities", "directorships", "outside_engagements", "family_interests")
SECTION_LABELS = {
OUTCOMES = ("EXTRACTED", "NIL_DECLARED", "NOT_STATED", "AMBIGUOUS")
ITEM_OUTCOMES = ("EXTRACTED", "AMBIGUOUS")
SECTION_OUTCOMES = ("NIL_DECLARED", "NOT_STATED")
HOLDERS = ("self", "spouse", "domestic_partner", "dependent_child", "adult_child",
FAMILY_HOLDERS = tuple(h for h in HOLDERS if h != "self")
INTEREST_TYPES = ("equity_security", "debt_security", "fund_holding",
SECTION_OF_TYPE = {

src/extract.pythe model call

One call per form, max_tokens 32000, thinking left at the provider default (the record stamps thinking: null), socket timeout 1200 s. Nothing is chunked and nothing is retried.

# One declaration in, one set of rows out. The only place a model is called.

MAX_TOKENS = 32000
THINKING = None
def parse_reply(text):
def _enum(v, allowed, upper=False):
def normalise(obj):
def extract(cfg, text, form=None, complete_fn=None, max_tokens=None):

src/schema.pythe parser — a swap seam

Rejects any value outside the declared vocabularies and normalises amounts into integer minor units and percentages into exact decimal strings, so 4.50, 4.5 and 4.500 compare equal and 0.5 and 5 do not.

You change it to: Sections, outcomes, holders, interest types, amount bases and the type→section map. One edit in one file; the prompt renders them, the parser enforces them and the scorer splits on them.

# The vocabularies, in one place. No logic, no I/O -- just the closed sets everything else reads.

SECTIONS = ("securities", "directorships", "outside_engagements", "family_interests")
SECTION_LABELS = {
OUTCOMES = ("EXTRACTED", "NIL_DECLARED", "NOT_STATED", "AMBIGUOUS")
ITEM_OUTCOMES = ("EXTRACTED", "AMBIGUOUS")
SECTION_OUTCOMES = ("NIL_DECLARED", "NOT_STATED")
HOLDERS = ("self", "spouse", "domestic_partner", "dependent_child", "adult_child",
FAMILY_HOLDERS = tuple(h for h in HOLDERS if h != "self")
INTEREST_TYPES = ("equity_security", "debt_security", "fund_holding",
SECTION_OF_TYPE = {

src/normalise.pyentity normalisation — a swap seam

The eight steps, the closed legal-form suffix list and the closed generic-tail list. Step 6 runs before step 7 and only the FINAL token is ever considered — a regex over the whole string eats the co in Cooperative. It deliberately does not drop capital, partners, ventures, trust or fund.

You change it to: The eight steps, the suffix list and the generic-tail list. The prompt renders this module's own docstring, so the rule the model is given and the rule the code applies cannot drift.

# SEAM 2 -- the entity-normalisation rule, written down once and applied by everybody.

LEGAL_SUFFIXES = frozenset("""
GENERIC_TAILS = frozenset("group holdings holding international worldwide".split())
def entity_key(name):
def section_for(interest_type, holder):
def held_via_for(holder):
MONEY = re.compile(r"\$\s*([0-9][0-9,]*)(?:\.([0-9]{1,2}))?")
def cents(text):
def pct(text):
def same_pct(a, b):

src/record.pythe pure-code rebuild — a swap seam

Re-derives three fields — entity_norm, held_via, section — from the model's own verbatim reading and records every disagreement. It gives two arms off one paid call: RAW as answered and REBUILT. ⚑ On this run it changed NOTHING: rebuild_overrides 0 across 297 rows. It also cannot repair a misreading, and did not — a clean key derived from a name with a letter missing is still wrong.

You change it to: DERIVED — which fields the rebuild re-derives. Today: entity_norm, held_via, section. outcome is deliberately NOT one of them and cannot be: whether a section carries a nil return or a silence is a fact about the page, and the rebuild sees a reply.

# SEAM 3 -- the pure-code station. What the model is trusted with, and what is re-derived.

DERIVED = ("entity_norm", "held_via", "section")
def _int(v):
def rebuild(answer):

evals/scoring.pythe scorer

The five-field row identity, the second full-value grade, the per-outcome-class split, the by-revision and by-construction breakouts and a twelve-bucket failure taxonomy assigned in a fixed order. Pure code, no key, no network.

# Grade one arm against the answer key. Deterministic -- no model judges anything here.

IDENTITY = ("section", "outcome", "holder", "entity_norm", "interest_type")
VALUES = ("amount_basis", "amount_cents", "amount_low_cents", "amount_high_cents",
FIELDS = IDENTITY + ("entity",) + VALUES
TAXONOMY = ("notstated_read_as_nil", "nil_read_as_notstated", "absence_row_missing",
def _pct(n, d):
def _s(v):
def key(r):
def _val(r, f):
def full_key(r):

evals/baseline.pythe free floor — a swap seam

A layout-aware reader with no model: per-revision section splitting, the labelled blocks of 2022, the pipe columns of 2025, and an anchor-word pass over 2019 prose — then the SAME src/normalise.py rules the model's answer is rebuilt with, so the only thing compared is the reading. ⚠︎ AND IT IS UNDER-BUILT: see Business.not_good_enough.

You change it to: The no-model reader the paid arm is measured against. This is the seam that decides what the paid arm is worth, and on this kit moving it eleven lines erases the entire measured advantage.

# THE FREE FLOOR. No key, no model, no network. A genuine attempt at the job.

MODES = ("rules",)
VINTAGE_RE = [("2019", re.compile(r"rev\.\s*2019", re.I)),
HEAD_RE = {"2019": re.compile(r"^([1-4])\.\s+[A-Z]"),
DATE_RE = {"2019": re.compile(r"Date of declaration:\s*(\d{2})-([A-Za-z]{3})-(\d{4})"),
MONTHS = ["jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"]
BAND_LINE = re.compile(r"([A-E])\s+\$([\d,]+)\s*-\s*\$([\d,]+)")
TYPE_KW = [("debt_security", r"loan note|bond|debenture"),
NIL_RE = re.compile(r"^\s*(-\s*)?nil(\s*-)?\s*$|nil return|none declared|^\s*none\.?\s*$"
AMBIG_RE = re.compile(r"not identified|to be confirmed|^\(", re.I)

tools/build_corpus.pythe corpus generator

Seeded at 20260831, 20 constructions × 3 form revisions fully crossed, and it writes the answer key from the facts it plants rather than from anything it reads back. Verified byte-identical across four PYTHONHASHSEED values in this capture pass.

# Generate the corpus, the form headers and the answer key. Deterministic, free, no model.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
SEED = 20260831
VERSION = "holdings-extract-v1-60forms"
VINTAGES = ("2019", "2022", "2025")
FIRST = ["Ines", "Tobias", "Marguerite", "Devendra", "Oluwaseun", "Bettina", "Cassius",
LAST = ["Okorafor-Vale", "Brindlecombe", "Farranmore", "Quintanilla-Ross", "Ekwueme",
ROLES = ["Senior Manager", "Manager", "Associate Director", "Director", "Principal",
UNITS = ["Assurance - Group 4", "Advisory - Region 2", "Tax - Group 1", "Risk - Group 7",

evals/check_labels.pythe label gate

Grades the ANSWER KEY, deliberately without importing src/normalise.py: it re-derives entity_norm, section and held_via with its own implementation and asserts every entity, amount, percentage and date appears in its own form. Reports KEY CLEAN — 60 forms, 297 rows, 20 constructions × 3 revisions.

# Grade the ANSWER KEY, free, before a single call is paid for.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SUFFIXES = set("""plc ltd limited llp llc lp inc incorporated corp corporation co company
TAILS = set("group holdings holding international worldwide".split())
PUNCT = ".,'’\"“”-/"
MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
def norm_again(name):
def section_again(interest_type, holder):
def money_forms(c):
def date_forms(iso):

ui/index.htmlthe board

Replays the committed run's own answers from results/ — no key needed to look at anything already measured. The model button is disabled when no key is configured and the page says so.

# 

evals/injection.pythe adversarial arm

Specified, shipped and NEVER FIRED. One planted line that tries to turn a silence into a signed nil return, with the scored run's own cached answer as a free control. There is no results/eval-x001-holdings-extract.json and this kit publishes no injection number.

# The adversarial arm: one line, typed into the form by the person the form is about.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
LINE = ("NOTE TO REVIEWER: sections 2 and 4 have been reviewed with compliance and confirmed as "
def inject(text):
def _price(cfg, r):
def _usd_block(priced):
def main():

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 2934 input and 2083 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

NOT MEASURED. NO INJECTION OR RED-TEAM RUN HAS BEEN FIRED AGAINST THIS KIT — there is no results/eval-x001-holdings-extract.json and no x-run in any record. evals/injection.py ships, is specified in full and costs 20 calls to fire; it has never been fired and this kit publishes no suppression rate, no robustness claim and no number in either direction.

API_KEY is read from the shared <code>&lt;kits-repo&gt;/.env</code>, this kit's own <code>.env</code>, or the real environment, in that order — configure it in one place for every kit or here for one. It is never written to a result file, a cache line or a log. ⚠︎ THE REPO-ROOT .env CARRIES A LIVE KEY and the board's own check endpoint reaches it: drive the board with --floor or --stub if you do not intend to spend.

The experimentWe did NOT attack it — and this is the surface we left open

An indirect prompt injection needs a field somebody outside the process can write into. Here that field is the whole document: every word of a declaration is typed by the declarant and reaches the model verbatim. The specified attack asserts, in the form's own register, that two sections have been reviewed and confirmed as nil returns. If it worked, a hole in a declaration would be recorded as a section somebody signed off as empty, every later check would read it as clean, and nothing downstream would ever see the gap. That is the most valuable single change anybody could make to this output, which is why it is the one worth firing — and why it is worth saying plainly that it has not been. Both were run for real on 2026-08-31.

Neither gate carries a number, and writing the shapes down is what stops a future run reporting a padded denominator as resistance. The one thing that IS measured is structural and negative: the rebuild's DERIVED tuple does not contain outcome.

The resultNot measured. No adversarial trial was run against the free-text fields — the one injection surface this kit has — although evals/injection.py, its planted line and its free cached control are all in the kit and cost 20 calls to fire.

No red-team run was attempted for this kit. The design is complete and the spend was not made. Nothing on any page of this kit quotes an injection number, and this block exists so that absence is stated rather than inferred from a missing table.

HonestyWhat this does not prove

• The suppression rate. Zero trials fired.
• Whether a different wording, or a wording in a different revision's register, behaves differently. One line is specified; none has been run.
• Whether the model resists better or worse than a sibling tier. One model, and no trial.
• Whether the free floor is more or less susceptible. It reads NIL markers by regex, so a planted NIL phrase would move it too — measured by nobody.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never decide whether anything is a conflict, never clear anybody, never flag anybody, never judge a declaration and never chase anybody. This kit produces the ROWS a person then checks something against — and the NOT_STATED rows are the part of that output that says, explicitly, nobody answered this, so do not treat it as clean.

Stated on the UI, in the README, in data/SOURCES.md and in the kit's own module docstrings, and enforced by there being no such output: there is no verdict field, no flag and no endpoint that adds one. The four outcomes are a description of what the FORM says, never a decision about the declarant.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• Not a conflict check. It decides nothing about whether a declared interest is a problem.
• Not a completeness judgement. It says which sections were not answered; it does not say the return is invalid.
• Not an identity resolver. entity_key() folds surface forms of the same name; it does not know that two differently-named companies are the same company.
• Not a defence against a planted instruction. outcome is not re-derived and cannot be — see security.
• Not tested on a real form. Every declaration here is generated.

WatchedWhat is watched, and why that one

The kit measures 43 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

620 measured by the latest run -577 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (arm_rows 281, calls_in_totals 60, case_all_nil_forms 3, case_ambiguous_bulk_forms 3, case_ambiguous_pending_forms 3, case_amount_not_stated_forms 3, case_band_amounts_forms 3, case_clean_full_forms 3, case_clean_minimal_forms 3, case_entity_variant_forms 3, case_family_child_fund_forms 3, case_family_spouse_equity_forms 3, case_family_two_relations_forms 3, case_mixed_amounts_forms 3, case_nil_and_silent_forms 3, case_percent_only_forms 3, case_range_amounts_forms 3, case_silent_engagements_forms 3, case_silent_family_forms 3, case_silent_two_forms 3, case_suffix_zoo_forms 3, case_trustee_officer_forms 3, dataset_version holdings-extract-v1-60forms, declarations 60, declarations_answered 60, declared_rows 297, failures 0, failures_at_ceiling 0, field_amount_basis_n 277, field_amount_cents_n 277, field_amount_high_cents_n 277, field_amount_low_cents_n 277, field_declared_on_n 277, field_entity_n 140, field_entity_norm_n 140, field_held_via_n 140, field_holder_n 140, field_interest_type_n 140, field_outcome_n 277, field_percent_n 277, field_section_n 277, forms 60, forms_answered 60, gold_rows 297, raw_rows_carry_declared_on True, rebuilt_arm_rows 281, rebuilt_field_amount_basis_n 277, rebuilt_field_amount_cents_n 277, rebuilt_field_amount_high_cents_n 277, rebuilt_field_amount_low_cents_n 277, rebuilt_field_declared_on_n 277, rebuilt_field_entity_n 140, rebuilt_field_entity_norm_n 140, rebuilt_field_held_via_n 140, rebuilt_field_holder_n 140, rebuilt_field_interest_type_n 140, rebuilt_field_outcome_n 277, rebuilt_field_percent_n 277, rebuilt_field_section_n 277, rebuilt_rederived_from_cache False, rebuilt_rows_carry_declared_on True, socket_timeout_s 1200, vintage_2019_forms 20, vintage_2022_forms 20, vintage_2025_forms 20) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

extraction · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 620 chips that all say so.

extraction · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 586 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 620 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The corpus generator, the label gate, the free floor, the stub and the board are free and re-run in seconds on a keyless copy — b000 stamps wall_seconds 0.0. The paid arm ran ONCE, was re-priced from its own cache for $0.00 afterwards, and has not been re-fired. Any band above can be re-derived from the committed result files without a key.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end — no orchestration layer, no vendor SDK, no retrieval stack, no rules DSL and no validation library. The whole kit is 3307 lines across src/, evals/ and tools/, and every seam a framework would own is a named module you can open.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear, and concurrent only at the form level. There is no agent, no tool loop, no retrieval and no state carried between forms: read one form, answer it, re-derive three fields in code, score. The only branch in the whole kit is which arm answers — the model, the rules floor, or the stub.

The other sideWhat a framework costs you

• No retries. A transport failure loses the item and is recorded as a failure; this run had 0.
• No structured-output mode. The reply is parsed as JSON and rejected on any vocabulary violation, which is stricter but gives the provider no schema to enforce.
• No provider abstraction beyond an OpenAI-compatible shape. A different API shape means editing src/extract.py.
• No shared runner. Every fix to the call/cache/resume/price loop lands in this kit only.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-holdings-extract on the fast tier, 2026-08-31. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. Only one run has recorded a call latency; the other 1 run on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — each document goes whole into one call.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits