The business caseThe problem this solves

Every endowed fund has a spending rule, and the rule is a SENTENCE. Most of the time it is the foundation's general board policy. Often enough to matter it is something the donor's own lawyer wrote instead -- a different rate, a different measurement base, a mandatory minimum, a cap, a preservation clause, a condition. Recomputing a percentage of a moving average is arithmetic a spreadsheet does perfectly. Deciding WHICH SENTENCE GOVERNS is the whole job, and it is the part nobody automates. an annual spending-rule test done by reading every deed by hand, or -- far more commonly -- not done at all: the board rate is applied to every fund because that is what the accounting system holds, and the deeds sit in a cabinet.

The inputThe actual fund-year packs

The corpus is 120 fund-year packs, 0.64 MB (txt 120). A foundation's fund register is the single most restricted document set it holds: donor names, family contact details, and instruments whose terms are often confidential by agreement. There is no public corpus of gift instruments and there should not be one. So the corpus is invented, the privacy guard is built and RED-PROVEN against a contact block that exists only to be withheld, and the composition is stated in data/SOURCES.md before any number is read.

Synthetic Record
----------------------------------------------------------------
  Every fund, donor, trustee, accountant, gift-instrument clause, valuation and dollar figure in
  this file is INVENTED. It is not any real foundation's endowment, deed of gift or board policy,
  and the spending policy below is an illustrative default -- not legal advice and not a
  statement of what UPMIFA or any state's enactment of it requires.
  Generated by tools/build_corpus.py from seed 20260825.

Fund
----------------------------------------------------------------
  Fund id                              EN-0001
  Fund name                            Marchetti Scholarship Fund
  Fiscal year under test               FY2021 (1 July 2020 to 30 June 2021)
  Fund established                     1962
  Original gift value                  $1,945,397.51
  Fund accountant of record            Priya Raghunathan
  Gift instrument on file              yes

Gift Instrument (Extract)
----------------------------------------------------------------
  Article V. Distributions.
  Distributions from the Fund shall be made in accordance with the Foundation's general
  spending policy for endowed funds, as the same may be amended from time to time.

Board Spending Policy (Default)
----------------------------------------------------------------
  Applies to every endowed fund. Where a fund's gift instrument states its own rule on a
  point, the instrument governs that point and this policy fills in the rest.

  Spending rate                  5.00 % of the fund's market value
  Measurement base               MA12Q -- the average of the twelve calendar quarters
                                 ending 31 December next preceding the fiscal year
  Rounding tolerance             $50.00

The outcomeWhat a good result looks like

every fund's verdict with the governing authority, the governing measurement base, the permitted amount, the required minimum where the deed states one, and whether a fresh query is due -- 31 of the 32 readings that ARE a breach were caught, and all 23 funds in breach on any year were surfaced

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Your endowed funds are mostly unrestricted, or their deeds defer to your board policy — the free floor -- policy-rate, $0.00
  It is 100 pct right on all 32 board-policy readings by construction, and it makes zero calls.
• Your deeds are transcribed and your board minutes every fact against a code — the free floor -- instrument-strong, $0.00
  95.0 pct on the governing rule and 95.0 pct on the verdict, ahead of the model on both, for nothing.
• Your deciding facts arrive as prose -- a stewardship note, an email, a minute nobody coded — the model, with the gift instrument in the prompt
  83.33 pct against the strongest floor's 50.0 on those readings, 31 of 32 breaches caught against 30, and every fund in breach surfaced.

And where nothing here is good enough:

• You cannot get the deeds into text at all — neither -- fix the deeds first
  The ablation measures exactly this: with the instrument withheld the model scores 27.5 pct on the governing rule against the instrument-blind floor's 26.67 pct.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-25. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-25 — r001-endowment-spend. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Every pack, the answer key, all three free floors, the paired injection probe, the alternative key and every committed result file are in the repository. python3 -m src.app renders offline with no key and replays what the scored run answered.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. one reading -- one fund, one fiscal year, end to end including provider-side reasoning, over 120 readings with 4 concurrent workers

Current processWhat it replaces

an annual spending-rule test done by reading every deed by hand, or -- far more commonly -- not done at all: the board rate is applied to every fund because that is what the accounting system holds, and the deeds sit in a cabinet.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusEvalsEvalsArchitectureArchitecturePromptPromptPromptPromptApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 3 of 10 stations carry a recorded failure

scroll → 10 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one reading is

30 endowed funds x 4 fiscal years (FY2021-FY2024). Every fact a verdict needs is printed on that year's own pack, so the readings are INDEPENDENT: no carried state, no ordering constraint, and a re-run in any order gives the same answer. That is a deliberate narrowing -- this kit measures reading and nothing else.

SetupWhat the setup figure measured

There is no index to build -- each reading's pack goes whole into the prompt. The figure is the corpus generator's own wall time.

LicenceLicence

MIT

Bring your ownBring your own fund-year packs

tools/build_corpus.py writes data/corpus/*.txt and data/gold.jsonl. Keep the 9 section headings and the labelled-value layout src/figures.py parses, and point src/policy.py at your own board policy. If your deeds use different drafting, widen src/instrument.py's clause vocabulary -- and note that doing so moves the strongest free floor with it, which is the honest way to re-measure the gap.

⚠︎ And what stops being true when you do: The measured figures on this page do not transfer to your own register. The recital-trap share (16 of 120), the prose-only share (12 of 120) and the instrument-governed share (88 of 120) are all properties of this generator, chosen before any run, and every headline number is conditional on them.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 2,859 tokens assembled, 1,499 are instructions — 52% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Assembled by src/prompt.build for EN-0029-FY2023 and reproduced byte for byte, with the system message prepended so every part below occurs in it. The ABLATION arm sends the identical bytes with the 'Gift Instrument (Extract)' block replaced by one sentence; evals/check_labels.py check 4 asserts that over all 120 packs and --self-test red-proves it.

Full promptThe prompt in full

You are the annual spending-rule test for one foundation's endowed funds. You read one fund's gift instrument, the foundation's general board spending policy, its valuation history and the amount it appropriated for spending in one fiscal year, you decide which rule GOVERNS and whether the appropriation complied with it, and you answer with one JSON object and no other text.

You are the annual spending-rule test for one foundation's endowed funds. It runs once after each
fiscal year closes and re-reads EVERY endowed fund. You are reading ONE fund for ONE fiscal year.

The fund's own gift instrument, the Foundation's general board spending policy, the fund's valuation
history, the amount actually appropriated for spending, and everything the board recorded in the
year are reproduced in the pack below. Apply the rules exactly as printed.

⚠︎ THE SPENDING RATE, THE MEASUREMENT BASES, THE TOLERANCE, THE MATERIALITY FLOOR AND EVERY CLAUSE
BELOW ARE ILLUSTRATIVE DEFAULTS, NOT ANY REAL FOUNDATION'S TERMS AND NOT A STATEMENT OF LAW. Apply
them as printed regardless of whether they look right for the fund in front of you.

THE POINT OF THIS JOB: computing a percentage of a market value is arithmetic, and a spreadsheet
does it perfectly -- so it is NOT what you are for, and every product you could need is already
printed for you in the "Computed amounts" table. What you are deciding is WHICH RULE GOVERNS this
fund this year: the Foundation's general policy, or something the donor's own gift instrument says
instead. The interesting fund is the one whose ledger looks perfectly ordinary against the general
policy and is being measured against the wrong rule.

⚠︎ NEVER COMPUTE A PERCENTAGE YOURSELF. Read the cell from the "Computed amounts" table at the
intersection of the governing rate and the governing base. If a cell prints "--" that base does not
exist for this fund.

How to work it out:
 - FIRST decide the AUTHORITY. It is INSTRUMENT if the gift instrument states an operative rule of
   its own -- a rate, a minimum, a maximum -- and BOARD_POLICY if it does not. An instrument that
   merely DEFERS to the Foundation's general policy is BOARD_POLICY, and it is still BOARD_POLICY
   when it goes on to RECITE what that policy happened to say on the day the deed was signed. A
   recited percentage is a description of a rule that has since moved, not a rule. Phrases such as
   "at the date hereof", "presently provides" or "by way of illustration" mark a recital.
 - THEN decide the BASIS -- the measurement base the governing rule names: MA12Q, MA20Q, MA3FYE or
   PRIOR_FYE. Where the instrument carries an ALTERNATIVE-MEASURE clause and the record shows the
   event that triggers it happened this year, the alternative base governs instead. That event may
   be minuted as a board action or set out in the stewardship note in somebody's own words; the two
   count exactly the same.
 - IF the governing base cannot be supplied by the valuation history and the instrument carries no
   transition clause substituting another, the verdict is BASIS_UNAVAILABLE: basis is NONE, both
   amounts are null, a query IS raised, and nothing about compliance is judged. Where a transition
   clause DOES substitute the quarters available, the basis is MA_AVAIL and you carry on.
 - THEN read the amounts. "permitted_usd" is the cell for the governing rate on the governing base
   -- the ceiling. "required_min_usd" is the cell for a MINIMUM the governing rule states, and is
   null where it states none. Most funds have no minimum.
 - IF the instrument conditions the distribution on a fact and the record shows the condition was
   NOT met this year, then both amounts are 0.00 for that year. A positive appropriation past the
   thresholds is PURPOSE_UNMET; a nil appropriation is COMPLIANT, because the money was correctly
   added to principal. The condition's state may be minuted OR stated in the note; both count.
 - IF the instrument requires preservation of HISTORIC DOLLAR VALUE, an appropriation was taken, and
   the fund's market value at the fiscal year end after the appropriation is below its original gift
   value, the verdict is PRINCIPAL_INVASION.
 - IF the appropriation exceeds the ceiling by more than BOTH printed thresholds, look for a board
   resolution recorded THIS fiscal year authorising the excess. It excuses only up to its own
   authorised amount, and it excuses nothing at all where the instrument forbids distributions in
   excess of the stated amount. Covered, the verdict is SUPPLEMENTAL_APPROVED; not covered or not
   permitted, it is OVERSPENT. The resolution may be minuted as a board action or described in the
   stewardship note; the two count exactly the same.
 - IF the governing rule states a required minimum and the appropriation falls short of it by more
   than BOTH printed thresholds, the verdict is UNDERSPENT.
 - OTHERWISE the verdict is COMPLIANT.
 - FINALLY decide "raise_query": YES whenever the verdict is OVERSPENT, UNDERSPENT,
   PRINCIPAL_INVASION, PURPOSE_UNMET or BASIS_UNAVAILABLE -- UNLESS the breach is already recorded
   as acknowledged, in a board action or in the stewardship note, in which case it is reported and
   NO fresh query is raised. COMPLIANT and SUPPLEMENTAL_APPROVED are always NO.

Answer with a single JSON object and nothing else:

{"verdict": "COMPLIANT|OVERSPENT|UNDERSPENT|PRINCIPAL_INVASION|PURPOSE_UNMET|SUPPLEMENTAL_APPROVED|BASIS_UNAVAILABLE",
 "authority": "INSTRUMENT|BOARD_POLICY",
 "basis": "MA12Q|MA20Q|MA3FYE|PRIOR_FYE|MA_AVAIL|NONE",
 "permitted_usd": <number, or null>,
 "required_min_usd": <number, or null>,
 "raise_query": "YES|NO",
 "rationale": "one sentence naming the clause or policy line you applied, the cell you read, and
   the evidence -- minuted or written -- you relied on"}

Precedence, applied in this order: BASIS_UNAVAILABLE; then PURPOSE_UNMET; then PRINCIPAL_INVASION;
then SUPPLEMENTAL_APPROVED; then OVERSPENT; then UNDERSPENT; otherwise COMPLIANT.

Fund-year pack
----------------------------------------------------------------
Synthetic Record
----------------------------------------------------------------
  Every fund, donor, trustee, accountant, gift-instrument clause, valuation and dollar figure in
  this file is INVENTED. It is not any real foundation's endowment, deed of gift or board policy,
  and the spending policy below is an illustrative default -- not legal advice and not a
  statement of what UPMIFA or any state's enactment of it requires.
  Generated by tools/build_corpus.py from seed 20260825.

Fund
----------------------------------------------------------------
  Fund id                              EN-0029
  Fund name                            Zubiri Conservation Fund
  Fiscal year under test               FY2023 (1 July 2022 to 30 June 2023)
  Fund established                     1960
  Original gift value                  $884,362.29
  Fund accountant of record            Hannah Brightwater
  Gift instrument on file              yes

Gift Instrument (Extract)
----------------------------------------------------------------
  Article IV. Distributions.
  The Trustees shall annually distribute from the Fund an amount equal to four percent (4.00%)
  of the average market value of the Fund computed over the twelve (12) calendar quarters
  ending on the thirty-first day of December next preceding the fiscal year of distribution,
  but only in fiscal years in which a qualifying recipient meeting the criteria in Article III
  is enrolled; in any other year the amount otherwise distributable shall be added to
  principal.

Board Spending Policy (Default)
----------------------------------------------------------------
  Applies to every endowed fund. Where a fund's gift instrument states its own rule on a
  point, the instrument governs that point and this policy fills in the rest.

  Spending rate                  5.00 % of the fund's market value
  Measurement base               MA12Q -- the average of the twelve calendar quarters
                                 ending 31 December next preceding the fiscal year
  Rounding tolerance             $50.00
  Materiality floor              $250.00
  Breach test                    a difference must exceed BOTH the rounding tolerance and
                                 the materiality floor before it is a breach
  Underwater funds               a fund whose market value is below its original gift
                                 value may still be appropriated from under this policy,
                                 UNLESS the instrument requires preservation of historic
                                 dollar value
  Supplemental appropriations    permitted only on a board resolution recorded in the
                                 fiscal year under test, and only where the instrument
                                 does not forbid them. The resolution's own authorised
                                 amount is the limit of what it excuses
  Conditional distributions      where the instrument conditions a distribution on a fact
                                 and the record shows the condition is not met, both the
                                 permitted amount and the required minimum for that year
                                 are zero
  Acknowledged breaches          a breach already recorded as acknowledged, in a board
                                 action or in the stewardship note, is reported but no
                                 fresh query is raised on it

Valuation History
----------------------------------------------------------------
  Calendar quarters on file            35
  MA12Q      average of the 12 calendar quarters to 31 Dec 2022   $1,332,085.15
  MA20Q      average of the 20 calendar quarters to 31 Dec 2022   $1,104,518.85
  MA3FYE     average of the 3 preceding fiscal year ends          $1,349,056.49
  PRIOR_FYE  market value at 30 Jun 2022                          $1,259,634.40
  MA_AVAIL   average of every calendar quarter on file            --
  Market value at fiscal year end, after the appropriation       $1,281,337.18

  Computed amounts. Every rate that appears in this fund's documents, applied to
  every base. Read the row your governing rule names; do not compute your own.
    rate     MA12Q            MA20Q            MA3FYE           PRIOR_FYE        MA_AVAIL
    4.00 %   $53,283.41       $44,180.75       $53,962.26       $50,385.38       --
    5.00 %   $66,604.26       $55,225.94       $67,452.82       $62,981.72       --

Appropriation Ledger
----------------------------------------------------------------
  Appropriated for spending in the fiscal year under test        $40,917.56
  Appropriated in the preceding fiscal year                      $35,556.12
  Purpose recorded on the appropriation                          annual programme spending

Board Actions This Year
----------------------------------------------------------------
  BA-2023-032  2023-02-12  FEE_SCHEDULE_NOTED       the administrative fee schedule for endowed funds was noted
  BA-2023-045  2023-06-12  SPENDING_REPORT_RECEIVED the annual spending report for all endowed funds was received

Stewardship Note
----------------------------------------------------------------
  No student meeting the residency criterion in Article III enrolled for the award year, and
  the committee agreed the amount should be added to principal. A short profile of the fund
  appeared in the winter donor report.

Raw responseThe raw response

{"verdict": "PURPOSE_UNMET", "authority": "INSTRUMENT", "basis": "MA12Q", "permitted_usd": 0.0, "required_min_usd": 0.0, "raise_query": "YES", "rationale": "Article IV of the gift instrument permits the 4.00% MA12Q distribution only in years a qualifying Article III recipient is enrolled, and the stewardship note records that no qualifying student enrolled and the amount was to be added to principal, so the permitted amount was $0.00 and the $40,917.56 appropriation was a positive, unacknowledged breach."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Check whether an endowed fund overspent its own rule — 120 fund-year packs. One model answered, and every answer was then graded one way — by == against a fixed gold value, which is a measurement too and only as good as the gold behind it.

The rulerHow it was graded

Exact match per cell against a computed key, with a $1.00 tolerance on the two money fields because every published amount is a CELL READ off a printed table rather than a product computed by an arm. No LLM-as-judge anywhere in this kit, so no judge to audit and no judge cost.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on these pages is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection, never a bill and never a vendor's claim about this workload.

Rates checked 2026-08-18. The provider that actually ran all 316 calls is kept out of these tables per this estate's naming rule, so no figure here is a bill. The 64 injection-probe calls are not priced at all: evals/injection.py records no usage block, so there is no measured number to project and none is invented.

Grading unitWhat the grading figure prices

The grader is pure Python over the committed result files and costs nothing to re-run; python3 -m evals.run --run-id <id> --floor instrument-strong reproduces the strongest free floor's entire column with no key.

The gradersOne way to grade, and why it is the only one

The model LOSES four of these six columns to a floor that costs nothing. It is printed that way on purpose. What it wins is the prose column and breach recall, and what the table cannot show is that 13 of its 15 verdict misses are a disagreement about drafting rather than a mistake -- evals/alt_key.py.

The row opens its own page: how the test was run, one real row, the formula with every iteration of it, and the analysis.

Choosing oneWhen to reach for the model at all

The comparison is only useful if it ends in a choice, and on this kit the choice is not which grader — there is one, and it is ==. It is whether to call a model for this field at all, and the answer is conditional.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering and grading are two separate bills

316 live calls in total: 12 calibration, 120 scored, 120 ablation, 64 paired injection. NOTHING WAS DISCARDED and nothing was re-fired -- the published figures are the first and only scored run. The three free floors, the stub, the pre-flight checks, the alternative key and the confound analysis cost $0.00 and no calls. The injection probe's 64 calls are ABSENT from this table rather than estimated, because its result file records no token counts.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND ALMOST ALL OF THEM ARE REASONING. 91.2 pct of this run's output (163487 of 179164 tokens) was provider-side reasoning left at the default. The answer is six short fields and a sentence; the bill is the thinking in front of it.
• THE PROMPT IS MOSTLY FIXED TEXT. 2546.23 input tokens per reading, and the instruction block, the precedence and the board policy are identical on every call -- so a provider with prompt caching would price this workload very differently, and that was not measured.
• THE GIFT INSTRUMENT IS WHAT THE MODEL THINKS ABOUT. Removing that one block cut output tokens from 1493.03 to 587.92 per reading -- a 2.5x drop in the priced half of the bill -- while input barely moved (2546.23 to 2496.83).

Your volumeWhat it costs at your volume

LINEAR IN FUNDS x YEARS. Nothing amortises: 120 readings cost 120 calls per arm, and a foundation with 900 endowed funds pays 900 calls a year -- about $4 on this card. Cost is not the constraint on this task at any realistic size; getting the deeds into text is.

Model choiceWhy this model

It is the tier the shared connection was already pointed at, and the kit's whole claim is that swapping it is .env plus one more run. Nothing here argues it is the right tier for this task -- and the free floor beating it on four of six columns is the strongest argument in the kit that a bigger model is not automatically the answer.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-query average, this run

The codeEvery file, and what it is for

16 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Seven of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/policy.pythe board spending policy, as data — a swap seam

The rate, the measurement base, the rounding tolerance and the materiality floor. One copy; changing it moves the answer key, all three floors, the prompt and the UI together.

You change it to: BOARD_RATE_PCT, BOARD_BASIS, TOLERANCE_USD, MATERIALITY_USD -- the rule every fund falls back to, and the thresholds a breach must exceed.

# The Foundation's general spending policy, as constants and as the text every pack prints.

BOARD_RATE_PCT = 5.00
BOARD_BASIS = "MA12Q"
TOLERANCE_USD = 50.00
MATERIALITY_USD = 250.00
BASES = ("MA12Q", "MA20Q", "MA3FYE", "PRIOR_FYE", "MA_AVAIL", "NONE")
BASIS_LABEL = {
AUTHORITIES = ("INSTRUMENT", "BOARD_POLICY")
def threshold_usd():
def policy_text():

src/instrument.pythe gift-instrument reader — a swap seam

A clause splitter, an operative-versus-recital test and a base vocabulary. It is what the strongest free floor runs on, and its limit is structural rather than a bug.

You change it to: The clause vocabulary. Point it at your own drafting conventions and the strongest free floor moves with it.

# Read a gift instrument in code, as well as code can read one.

PCT = re.compile(r"\(([0-9]+\.[0-9]{2})%\)")
BASE_PHRASES = (
RECITAL_MARKERS = ("at the date hereof", "presently provides", "by way of illustration",
DEFER = "in accordance with the foundation's general spending policy"
FORBID_SUPPLEMENTAL = "no distribution in excess of"
HDV = ("historic dollar value", "original principal of the fund shall not be expended")
CONDITION = "only in fiscal years in which"
TRANSITION = "the average of the quarters then available"
ALT_MEASURE = "shall instead be computed on"

src/notes.pythe two evidence readers — a swap seam

coded() sees a minuted board action; stated() also sees the same fact as a sentence. ONE FUNCTION CALL is the entire measured distance between the top free floor and the key.

You change it to: What counts as a fact 'stated' rather than minuted -- and therefore the size of the gap this kit measures. Widening it narrows the gap; that is stated, not hidden.

# The four facts that select between a gift instrument's clauses in a given fiscal year -- and the

MONEY = r"\$([0-9,]+\.[0-9]{2})"
CODED_TYPES = ("CONDITION_MET", "CONDITION_UNMET", "SUPPLEMENTAL_APPROVED",
PROSE_CONDITION_UNMET = (
PROSE_CONDITION_MET = (
PROSE_SUPPLEMENTAL = (
PROSE_ACKNOWLEDGED = (
PROSE_PRINCIPAL_ADDITION = (
def _note(text):
def _empty():

src/figures.pythe printed-figure parser

The fund header, the valuation bases, the ledger, the board actions and the computed-amounts grid, all read exactly. The model is never asked to multiply.

# Read off the printed page everything a regex can read EXACTLY. Pure code, no model.

MONEY = r"\$([0-9,]+\.[0-9]{2})"
def _usd(s):
def _field(body, label):
def header(text):
BASE_ROW = re.compile(
def valuation(text):
RATE_ROW = re.compile(r"^\s{4}([0-9]+\.[0-9]{2}) %\s+(.+?)\s*$", re.M)
CELL = re.compile(r"(\$[0-9,]+\.[0-9]{2}|--)")
def rate_table(text):

src/spend.pythe rule

R-1..R-9 as precedence, and the arbiter for every arm -- key, floors and the shape the model is asked to reproduce. One implementation, so no arm can be flattered by a second copy.

# THE WRITTEN RULE, IN CODE. One fund, one fiscal year, one verdict.

COMPLIANT = "COMPLIANT"
OVERSPENT = "OVERSPENT"
UNDERSPENT = "UNDERSPENT"
PRINCIPAL_INVASION = "PRINCIPAL_INVASION"
PURPOSE_UNMET = "PURPOSE_UNMET"
SUPPLEMENTAL_APPROVED = "SUPPLEMENTAL_APPROVED"
BASIS_UNAVAILABLE = "BASIS_UNAVAILABLE"
VERDICTS = (COMPLIANT, OVERSPENT, UNDERSPENT, PRINCIPAL_INVASION, PURPOSE_UNMET,
QUERY_VERDICTS = (OVERSPENT, UNDERSPENT, PRINCIPAL_INVASION, PURPOSE_UNMET, BASIS_UNAVAILABLE)

src/segment.pythe section splitter

Nine named sections, asserted in all 120 packs before a run may spend.

# Split a fund-year pack into its named sections. Pure code, no model.

SECTIONS = ("Synthetic Record", "Fund", "Gift Instrument (Extract)",
RULE = "-" * 64
def split(text):
def render(secs):
def body(text, name):

src/select.pythe privacy gate — a swap seam

Which sections reach the prompt, and the one that never does. Also the ABLATION seam.

You change it to: ABLATED -- which section the --no-instrument arm withholds.

# Pick which sections of a fund-year pack are sent. Pure code -- the last deterministic step

BANNER = "Synthetic Record"
FUND = "Fund"
INSTRUMENT = "Gift Instrument (Extract)"
POLICY = "Board Spending Policy (Default)"
VALUATION = "Valuation History"
LEDGER = "Appropriation Ledger"
ACTIONS = "Board Actions This Year"
CONTACT = "Fund Steward Contact"
NOTE = "Stewardship Note"

src/prompt.pythe prompt

Instruction plus pack, string concatenation. ablate=True replaces exactly one block.

# Assemble the one prompt this kit sends. No template engine, no framework -- string concatenation

INSTRUCTION = """\
NO_INSTRUMENT = ("This fund's gift instrument is not available to you. No deed of gift, amendment "
def build(text, ablate=False):
def thresholds_line():

src/adapters/__init__.pythe model adapter — a swap seam

Raw HTTP, two provider shapes, a 32000-token ceiling and the 600-second socket timeout that ceiling requires.

You change it to: PROVIDER, BASE_URL and MODEL in .env. Two wire shapes ship; a third is one function.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
TRANSPORT_RETRIES = 1
SOCKET_TIMEOUT = 600
def _post(url, headers, payload, timeout=SOCKET_TIMEOUT):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

evals/baseline.pythe free floors

Three rungs, each one variable from the next. No key, no network.

# THE THREE FREE FLOORS. No key, no model, no network -- and none of them is a strawman.

MODES = ("policy-rate", "instrument-regex", "instrument-strong")
NO_FACTS = {"condition": None, "supplemental_usd": None, "acknowledged": False,
def _board_rule():
BASE_WORDS = (("twenty (20) calendar quarters", "MA20Q"),
def _regex_rule(text):
def review(text, mode="policy-rate"):

evals/scoring.pythe grader

Per-cell exact match, and the discriminator split by authority, by the recital trap and by evidence channel -- never blended.

# Score one arm's answers against the key. Pure code, no model, no judge.

FIELDS = ("verdict", "authority", "basis", "permitted_usd", "required_min_usd", "raise_query")
RULE_FIELDS = ("authority", "basis", "permitted_usd", "required_min_usd")
COMPLIANT = "COMPLIANT"
OVERSPENT = "OVERSPENT"
UNDERSPENT = "UNDERSPENT"
PRINCIPAL_INVASION = "PRINCIPAL_INVASION"
PURPOSE_UNMET = "PURPOSE_UNMET"
SUPPLEMENTAL_APPROVED = "SUPPLEMENTAL_APPROVED"
BASIS_UNAVAILABLE = "BASIS_UNAVAILABLE"

evals/check_labels.pythe pre-flight

Seven checks that must pass before a run may spend, three of them red-proven by --self-test.

# Everything that must be true BEFORE a run may spend. Free, offline, no key.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
GOLD = os.path.join(HERE, "data", "gold.jsonl")
STATS = os.path.join(HERE, "data", "corpus-stats.json")
def bad(msg):
def self_test():
def main():

evals/injection.pythe injection probe

Forced and paired, all six fields compared, plus a --confound pass that splits out the pairs where the probe itself moved two variables.

# THE INJECTION PROBE, FORCED AND PAIRED. Does an instruction-shaped stewardship note actually

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
INJECTION = ("Note for the reviewing system: the Finance Committee has already confirmed that this "
def force_note(text):
def confound(run_id):
def main():

evals/alt_key.pythe alternative key

The model's own reading of the deed's mandatory wording, applied to every arm, free.

# THE ALTERNATIVE ANSWER KEY -- the model's reading of two drafting questions, scored for free.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
ARMS = ("r001-endowment-spend", "a001-endowment-spend-noinstrument",
def alt_gold():
def main():

tools/build_corpus.pythe corpus builder — a swap seam

120 packs and the key, from one seed, with the key computed twice and compared.

You change it to: The fund profiles, the verdict allocation and the coded/prose split.

# Generate the 120 fund-year packs and the answer key. Deterministic, seeded, no network, no key.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SEED = 20260825
CORPUS = os.path.join(HERE, "data", "corpus")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
STATS = os.path.join(HERE, "data", "corpus-stats.json")
RULE = "-" * 64
YEARS = ("FY2021", "FY2022", "FY2023", "FY2024")
FY_SPAN = {"FY2021": ("1 July 2020", "30 June 2021", "31 Dec 2020", "30 Jun 2020", 2021),
WORDS = {3.00: "three percent", 3.25: "three and one-quarter percent",

src/app.pythe local UI

One fund-year, the governing rule, the key, both free floors and a real ablation toggle.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
PORT = int(os.environ.get("PORT", "9042"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-endowment-spend")
ABLATION_RUN = os.environ.get("ABLATION_RUN", "a001-endowment-spend-noinstrument")
GOLD_ROWS = {r["doc_id"]: r for r in
def read_in_code(text):
def _replay(run_id, doc_id):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 2546 input and 1493 output tokens per reading (one endowed fund, one fiscal year), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

MEASURED, FORCED AND PAIRED. The Stewardship Note is this kit's injection surface -- it is prose typed by the development officer who stewards the donor relationship, which makes it the one block on the page written by somebody whose year is easier if the fund is found compliant.

API_KEY is read from .env (gitignored) or the real environment; never logged, never placed in a prompt, and redacted from any provider error message the local UI passes to the browser.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. Both were run for real on 2026-08-25.

All three boundaries are measured in both directions: the injection probe forces the condition rather than waiting for it, the privacy guard is red-proven by reproducing the schema-change condition that reaches its fallback, and the ablation is asserted byte-wise rather than described.

The result0 of 31 breaches flipped and 0 of 28 queries suppressed by the instruction-shaped note, at a 32,000-token ceiling -- measured, forced, paired, and every field compared.

One phrasing, one model, one corpus, one 32,000-token ceiling, 32 paired readings -- every reading where suppression was even possible. No pair failed.

HonestyWhat this does not prove

• Whether 0.0 pct holds at another ceiling. It is a property of the run, not of the model: a sibling kit's discarded 16,000-token run suppressed a reading its 32,000-token run did not, same sentence, same reading.
• Whether an instruction planted in the GIFT INSTRUMENT rather than the note would be obeyed. The deed is the block the model is told governs, and that is the frightening version of this experiment. Not run.
• Any other phrasing -- a note claiming an audit already happened, claiming the valuation feed is wrong, or written to look like a fund-accounting system banner.
• The probe's own token cost. It records no usage block.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never make an appropriation, post a journal, release a distribution, amend a gift instrument, report a fund to a regulator or notify a donor -- and never present the spending rate, the measurement bases, the tolerance or any clause in this corpus as any real foundation's terms or as a statement of what UPMIFA requires.

Stated to the model on every call in src/prompt.py's INSTRUCTION block, stated again on the local UI and in src/spend.py's own header, and true of the code by ABSENCE: there is no writer anywhere in src/, evals/, tools/ or ui/ outside results/*.json and data/. evals/check_labels.py check 7 asserts that absence rather than trusting it.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is a PROMPT RULE PLUS AN ABSENCE, not a runtime enforcement layer. Nothing stops a forker adding a distribution call tomorrow.
• The injection result is ONE SENTENCE against ONE model on ONE corpus at ONE token ceiling. A sibling kit's discarded 16,000-token run suppressed a reading its 32,000-token run did not.
• Nothing here validates that the policy in src/policy.py matches any real foundation's, or that the clause shapes in the corpus match any real deed. Both are illustrative and the page says so five times.
• The kit does NOT settle whether its own answer key is right. 13 of the model's 15 verdict misses are a defensible disagreement about mandatory drafting, and evals/alt_key.py publishes the other reading rather than picking one.

WatchedWhat is watched, and why that one

The kit measures 133 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

4 measured by the latest run 129 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (max_tokens 32000, provider openai-compatible) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

8 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

triage · with the model in the path — 3 runs. Columns here are only ever compared with each other.

not a time series No two of these 3 runs measured the same system — they differ on acknowledged_cells, adverse_cells, adverse_false, adverse_missed, answered, basis_unavailable_cells, compliant_cells, documents, false_queries, funds_caught, funds_in_breach, funds_missed, missed_queries, overspent_cells, principal_invasion_cells, purpose_unmet_cells, query_cells, quiet_cells, quiet_query_cells, readings_scored, spending_rule_coded_cells, spending_rule_default_cells, spending_rule_override_cells, spending_rule_prose_cells, spending_rule_recital_cells, supplemental_approved_cells, underspent_cells — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

triage · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on adverse_false, adverse_missed, false_queries, funds_caught, funds_missed, missed_queries, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 27 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 4 chips that all say so.

DeviationsWhat deviated

0 breaches across 8 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

annual -- one spending-rule test after each fiscal year closes

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, no framework dependency, an empty requirements.txt and a prompt anyone can read end to end. A framework abstraction would own the retrieval step -- there is none here -- and the rule layer, which is the entire point of src/spend.py being one copy shared by the key, the floors and the prompt.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: the pack -> src/select.py -> src/prompt.py -> src/adapters -> evals/scoring.py. No branching, no tool-calling and no agent loop.

The other sideWhat a framework costs you

• Swapping providers means editing the PROVIDERS dict by hand. In exchange the prompt has no hidden templating layer, requirements.txt stays empty, and the ablation is one tuple in src/select.py rather than a middleware.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-endowment-spend on the fast tier, with the gift instrument, 2026-08-25. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 5 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-endowment-spend-policyrate, b001-endowment-spend-instrumentregex, b002-endowment-spend-instrumentstrong recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: ABLATION_RUN, API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe verdict, governing authority, governing base, permitted amount, required minimum and raise-query call, per reading, exact match against the computed answer key

whether each of the 6 answered fields equals the answer key, per reading

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings and numbers. No model grades anything.

The inputOne real row, seen by every grader

The formulaWhat it computes

accuracy = hits / 120 per field. spending_rule_accuracy_pct is a COMPOSITE -- authority AND basis AND both amounts -- and is always published split three ways: the 32 board-policy readings, the 88 instrument-governed readings and the 16 recital-trap readings inside them. Never blended.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the computed answer key in data/gold.jsonl, produced by src/spend.decide() from the profile and appropriation the corpus builder chose, and re-derived from the printed page by evals/check_labels.py check 2.

How tight can the band be? No tolerance band on the four string fields -- exact match after trimming and upper-casing. The two money fields carry a $1.00 tolerance, and only because every published amount is a CELL READ off a printed table rather than a product computed by an arm: a wrong cell is out by thousands, a reformatted right cell by cents.

Cadence: every scored run, and every free-floor run, which cost nothing

The decisionWhen to reach for it