The business caseThe problem this solves

A most-favoured-nation clause promises that nobody inside a group the clause itself defines is on better terms, on the terms it lists, subject to carve-outs it writes down. Checking it looks like a spreadsheet job: put the counterparty's other agreements in columns, put the protected one beside them, flag anything smaller. Today that is what gets built, and on this corpus 59.7 pct of everything it raises is not a breach -- 56.9 pct once the same free, deterministic recheck this kit ships is run over it, which is still more than half -- because almost none of the work is comparing numbers. It is deciding WHICH numbers the clause is even about. Building the MFN comparison by hand or in a spreadsheet -- reading which of the counterparty's agreements the clause actually reaches, which side letters change an effective value, which carve-out covers what, and then computing the remedy the clause requires.

The inputThe actual review packs

The corpus is 60 review packs, 0.38 MB (txt 60). The three judgements an MFN review actually turns on, planted deliberately, because no public dataset can carry them: what a counterparty pays its OTHER distributors is a negotiating position and is never published. 19 rows put the better number OUTSIDE the group the clause defines (8 wrong territory, 5 wrong tier, 3 another service, 3 outside the look-back); 14 put it inside the group and behind a written carve-out; 8 turn on the clause's own netting rule, 4 where it neutralises the differential entirely and 4 where it survives partially; and 7 riders move an effective value so that the breach is invisible on the printed face. Those are the 33-plus-7 rows a two-box checker cannot express in either direction, and they are the whole reason the kit has four verdicts rather than two.

MOST-FAVOURED-NATION REVIEW
  File reference        MFN-0001
  Review date           31 August 2026
  Protected party       Northgate Cable Partners
  Counterparty          Meridian Media Group
  Agreement under review  AG-0100, executed 1 January 2024
  Covered terms         rate, committed term, exclusivity, payment timing

THE CLAUSE  (AG-0100, clause MFN-1, as executed)
----------------------------------------------------------------------------------------------------------------------
  MFN-1.1  Comparison group
     Agreements of the Counterparty for the Service "Meridian Sports Network", with distributors in Tier 1,
     in the territory United Kingdom, executed within 18 months of the review date. An agreement
     failing any one of those attributes is not a Comparison Agreement.

  MFN-1.2  Covered terms, and what each one means
     Rate             the recurring per-subscriber fee, net of all RECURRING credits,
                      rebates and support payments. A one-off or signing payment is NOT
                      netted, whatever its size.
     Committed term   the months during which the distributor may not terminate for
                      convenience. A unilateral termination right SHORTENS it; an option
                      to renew does not change it.
     Exclusivity      the number of exclusivity holdbacks the agreement imposes.
     Payment timing   the days allowed to settle in the ordinary course. An extended
                      settlement rider LENGTHENS it; a suspension of the clock during a
                      formal dispute does not, being unavailable in the ordinary course.

  MFN-1.3  Carve-outs
     CO-1   A rate granted within the first 6 months following that distributor's launch

The outcomeWhat a good result looks like

One verdict per covered term that a person can confirm as written: 240 term rows over 60 review packs, each CARVED_OUT, NOT_COMPARABLE, BREACH or COMPLIANT, with the offending agreement named and the new rate, the arrears months and the credit computed to the cent in pure Python. Measured: 238 of 240 verdicts right (99.2 pct) and 58 of 60 packs right on all four terms (96.7 pct), against the same free code put through the same free recheck at 200 and 26. THE CALL BUYS 38 TERM ROWS AND 32 PACKS. (Until 2026-08-31 the free column read 196 and 23, because the floor arm never ran the recheck every model arm ran; nothing about the paid arm changed.)

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• A counterparty whose paper is uniform and whose clause has no carve-outs and no netting step -- a flat parity undertaking over one tier in one territory — the free rules floor alone
  On the 21 CLEAN packs of this corpus the floor already scores 19 of 21 and the paid call scores 20. One pack. Where nothing sits outside the group, nothing is carved out and nothing is netted, a column comparison is the answer and it costs $0.00 in 0.097 s for all sixty.
• A clause that defines its comparison group by attributes and writes down carve-outs -- which is what an MFN clause normally does — the paid reading step, with the pure-code recheck behind it
  This is where all the money is. On the 39 HARD packs free code through the same recheck scores 7 and the paid arm 38 (the raw floor, this kit's free column until 2026-08-31, scores 4). ALL 33 of free code's rechecked false breaches are rows a two-box checker cannot express at all -- 19 NOT_COMPARABLE and 14 CARVED_OUT -- and no amount of threshold tuning creates a verdict the vocabulary does not have. Raw it raised 37, the extra 4 being the clause's netting rule, which is the one thing the recheck hands back.
• Any clause whose remedy is a figure that goes in a letter — the pure-code station, always, whatever reads the pack
  Both arms hand their reading to the same engine and the arithmetic is then exact -- and free code is handed the same engine, so this is not a line the paid call buys. Rechecked, free code still gets the money wrong on 4 of the 25 breaches it finds, by $16,214,300.00, and all four name an agreement outside the comparison group entirely: a reading error the arithmetic cannot reach. Raw, before the recheck, it was 8 rows and $24,729,000.00, the other 4 being the netting rule the recheck hands back. A claim that is right with an indefensible number is the version that gets settled at a discount.

And where nothing here is good enough:

• A pack assembled from paper the counterparty controls, going to a provider verbatim — nothing on this page -- the measurement does not exist yet
  The adversarial arm is written, paired and unfired. There is no suppression rate here in either direction, and the recheck's trust in carve_outs means the pure-code station is inside the blast radius rather than outside it.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-31. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-31 — r001-mfn-check. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Measured on a copy of the kit with no .env and no API key: tools/build_corpus.py rebuilt all 60 packs, the answer key and the stats in 0.072 s and the result was byte-identical to the committed corpus; evals/check_labels.py re-graded the key with an independent parser and independent arithmetic in 0.050 s and reported no disagreement on any of the 240 rows; the free rules floor scored all 60 packs in 0.097 s and its result file matched the committed b000 record in every field except its own timestamp; the wiring stub ran in 0.090 s. The board and the corpus render with no key, and /api/review returns 200 saying so rather than failing. Nothing was bought and nothing was reached over a network.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. END-TO-END per review pack on run r001-mfn-check: one provider call, wall clock around it, from prompt assembly to a parsed reply. There is no retrieval, no index and no second call, so this figure and Cost.cost_by_model's are the same measurement -- unusually for this estate, they are not two different things wearing one field name. 3,541.6 s of summed call time ran in 735.5 s of wall clock at 5 workers (4.82x), so the wall figure is a concurrency measurement and the p50/p95 are the per-pack truth.

Current processWhat it replaces

Building the MFN comparison by hand or in a spreadsheet -- reading which of the counterparty's agreements the clause actually reaches, which side letters change an effective value, which carve-out covers what, and then computing the remedy the clause requires.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one review packs -- one MFN clause and its comparison agreements each is

No train/test split, because nothing is trained or tuned, and no segmentation, because the pack goes whole into one call (the sizes here are each pack's own bytes). The unit every RATE stands on is the covered TERM -- 240 rows -- and the unit every all-correct figure stands on is the PACK -- 60. Both denominators are printed beside every number rather than averaged into one. 21 packs are clean and 39 are hard, across 16 planted case families: plain_breach_rate 8, clean_compliance 8, carve_out_promotional 5, carve_out_volume 4, outside_group_tier 4, net_off_insufficient 4, net_off_neutralises 4, effective_rate_breach 4, outside_group_stale 3, outside_group_territory 3, carve_out_bundle 3, plain_breach_term_length 3, carve_out_payment_early 2, effective_term_breach 2, plain_breach_payment 2, effective_payment_breach 1.

SetupWhat the setup figure measured

There is no index to build and no retrieval step: the pack goes whole into one prompt at 4,818.8 input tokens on average. The 0.072 s is the corpus generator rebuilding all 60 packs, the 390 comparison agreements, the clause and the 240-row answer key -- measured on a copy of the kit with no .env and no key. Beside it the independent key checker re-graded that key in 0.050 s, the free rules floor scored all 60 packs in 0.097 s including writing its result file, and the wiring stub ran in 0.090 s. That is what the entire free half of this kit costs: 0.31 s and $0.00.

LicenceLicence

MIT -- and the MIT text is in the repository's LICENSE-PUBLIC file, NOT in its root LICENSE, which governs the repository itself and grants a reader nothing. The KIT is MIT; the repository is not, and this line says which because it is the only claim in a kit a reader is entitled to rely on before forking. There is no third-party data here to licence at all: the packs, the clause, the printed pages and the answer key are generated in process from a fixed seed. ⚠︎ data/SOURCES.md states the licence as 'MIT, with the corpus generated in process' and does not name LICENSE-PUBLIC; it makes no false claim, but it leaves a reader to open LICENSE and find the opposite. That file belongs to the kit repo and was not edited by this capture pass.

Bring your ownBring your own review packs

Drop your own review packs into data/corpus/ in the printed layout this kit uses -- the clause as executed, the protected agreement, a COMPARISON AGREEMENTS table, a TERMS AS PRINTED ON THE FACE table and a RIDERS AND SIDE LETTERS block -- and mirror their structured form in data/agreements.json, which is what the engine and the free floor read. The clause's own shape is the seam: the group's four attributes, the carve-out kinds, the netting step and the arrears cap all live per pack, and data/clause.md is rendered into the prompt verbatim so changing what 'more favourable' means changes what the model is told and what the code enforces in one edit. Grading needs a gold row per pack in data/gold.jsonl; without one the free floor, the UI and the recheck all still run and only the scoreboard goes quiet.

⚠︎ And what stops being true when you do: ⚠︎ THE PACK REACHES YOUR CONFIGURED PROVIDER VERBATIM, AND IT IS ASSEMBLED FROM THE COUNTERPARTY'S OWN PAPER. There is no redaction step and no pre-digest: the rates, terms and subscriber counts of a counterparty's other agreements -- which is commercially sensitive material somebody else wrote -- go whole into one call. That is a deliberate design choice measured here (a shortlist is where the decisive agreement gets dropped before the model sees it) and it is a disclosure decision on your material, not ours. ⚑ AND NONE OF THE MEASURED NUMBERS TRAVEL. Every figure on this board was measured on an invented, templated, generated corpus with a planted defect mix; the reading was near-perfect on everything except one boundary precisely because a side letter here is one labelled line. On real paper the reading is the part that degrades, and the whole design rests on it.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — but they are the input total apportioned by each part’s share of the characters, not a per-part reading from the provider’s counter. The total is measured; the split is arithmetic, and a part carrying denser text really costs more of it than this says.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 4,750 tokens assembled, 1,947 are rules — 41% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

The exact four parts sent for MFN-0001, the first pack of run r001-mfn-check, replayed from the kit's own src/prompt.py build(). Trustworthy as the prompt that was sent because the replayed part names and character counts (4,260 / 8,753 / 6,488 / 1,857 = 21,358) match the decomposition the run itself recorded, field for field, and the section list matches too. ⚠︎ THE PER-PART TOKEN COUNTS ARE APPORTIONED, NOT TOKENISED: the provider reported 4,750 input tokens for this call and the four parts carry it in proportion to their characters. The total is measured; the split between the four is a derivation and is stated as one.

Full promptThe prompt in full

You are checking one most-favoured-nation clause against the counterparty's other
agreements, for the party the clause protects.

An MFN clause is not a promise of the best terms in the market. It is a promise about a group the
clause itself defines, on terms the clause itself lists, subject to carve-outs the clause itself
writes down. So the question is never "who has the best deal in this file". It is, once per covered
term: does any agreement THIS CLAUSE ADMITS carry a term THIS CLAUSE COVERS on which somebody else
is treated better, in a way THIS CLAUSE does not excuse?

THE BEST TERMS IN A FILE USUALLY BELONG TO SOMEBODY THE CLAUSE WAS NEVER WRITTEN ABOUT. Treat a
better number as a question, not as a finding.

Read the file and do three things, and only these three.

1. THE COMPARISON GROUP. Decide which comparison agreements the clause's own group definition
   admits. Every attribute the clause names must match; one failure puts an agreement outside,
   however good its terms are. List the ids that are IN.

2. THE RIDERS. The file carries riders and side letters. Each says which agreement it belongs to,
   which term it moves, and by how much. The clause's definitions decide which ones count -- a
   recurring credit changes a rate and a one-off payment does not; a unilateral termination right
   shortens a committed term and a renewal option does not; an extended-settlement rider lengthens
   payment terms and a dispute-clock suspension does not. List the rider ids that COUNT.
   DO NOT APPLY THEM. Do not compute an effective rate, a differential or a credit. Say which
   riders count and the arithmetic is done from the printed figures afterwards, in code.

3. THE CARVE-OUTS. For each comparison agreement that would otherwise be more favourable, say
   whether a carve-out in this clause covers it, and which one. A carve-out is per agreement AND
   per term: an agreement carved out on rate may still be a comparator on payment timing.

Then give one verdict for each covered term. The covered terms, and the direction "more favourable"
runs in each:

  rate            (per subscriber per month) a LOWER value is more favourable
  term_length     (months)  a LOWER value is more favourable
  exclusivity     (holdbacks) a LOWER value is more favourable
  payment_timing  (days)    a HIGHER value is more favourable

The four verdicts:

  CARVED_OUT       An agreement INSIDE the comparison group really is more favourable on this
                   term, and a carve-out written into the clause covers every one that is --
                   a launch-window rate, a rate conditioned on a volume floor the protected
                   party does not meet, a fee bundling several subjects, or payment terms
                   bought with an early-settlement discount (MFN-R.5).
  NOT_COMPARABLE   Something in the file is more favourable on this term, but every agreement
                   that is sits OUTSIDE the comparison group the clause defines -- a
                   different tier, a different territory, a different subject, or executed
                   before the look-back window opens (MFN-R.2).
  BREACH           An agreement inside the group is more favourable on this term, no
                   carve-out covers it, and the clause's netting rule leaves a differential
                   standing (MFN-R.7). The remedy in MFN-R.8 applies to the NETTED
                   differential, not the headline one.
  COMPLIANT        Nothing inside the comparison group is more favourable on this term.
                   INCLUDING the case where a comparator is visibly cheaper and the clause's
                   own netting rule reduces the differential to zero or less, because it
                   bought the discount with a longer committed term (MFN-R.6).

Where more than one looks arguable at once, the precedence is: a live breach outranks a carve-out,
a carve-out outranks an out-of-group finding, and an out-of-group finding outranks silence.

NAME ONLY IDENTIFIERS PRINTED IN THE FILE IN FRONT OF YOU -- agreement ids, rider ids, carve-out
ids. An invented citation is worse than none; it is the first thing the other side checks.

Reply with JSON and nothing else, in the shape given at the end.

THE READING STANDARD, as written:

# MFN-R — how a most-favoured-nation clause is checked

This standard says how one agreement's most-favoured-nation clause is checked against the
counterparty's other agreements. It is deliberately narrow. It decides how to read a clause that is
already written; it decides nothing about whether the clause was a good idea, what either party
should have agreed, or what anybody should now do commercially.

**The clause in front of you is the authority.** This standard names the moving parts every such
clause has — a comparison group, a set of covered terms, carve-outs, a netting rule and a remedy —
and says how they interact. It does not fix their contents. Every file carries its own clause, and
its group definition, its carve-outs, its netting step and its remedy cap are whatever that clause
says they are.

## MFN-R.1 — What is being asked

One party (the **protected party**) holds a most-favoured-nation promise from a **counterparty**.
The promise is that the counterparty will not give better terms to somebody else — but only to
somebody else *inside a group the clause itself defines*, only on *terms the clause itself lists*,
and only *subject to carve-outs the clause itself writes down*.

The question, asked once per covered term, is therefore never "who has the best deal". It is:
**does any agreement the clause admits carry a term the clause covers, on which that counterparty
is treated more favourably than the protected party, in a way the clause does not excuse?**

## MFN-R.2 — The comparison group

MFN-1.1 of each clause names the attributes that define the group — typically a class or tier, a
territory, the subject of the agreement, and a look-back window measured from the review date.
An agreement is **in the group** only if it matches every attribute the clause names. An agreement
that fails any one of them is outside, however favourable its terms are.

**An agreement outside the group is not evidence of anything.** It is the single commonest way a
most-favoured-nation review produces a claim that cannot be sustained: the best terms in the file
usually belong to somebody the clause was never written about.

## MFN-R.3 — The covered terms, and which direction "more favourable" runs

| term | what it is | more favourable means |
|---|---|---|
| `rate` | the recurring per-subscriber fee | a **lower** rate |
| `term_length` | the committed term, in months, during which the counterparty may not terminate for convenience | a **shorter** committed term |
| `exclusivity` | the number of exclusivity holdbacks the agreement imposes | **fewer** holdbacks |
| `payment_timing` | the days allowed to settle an invoice | **more** days |

A term is more favourable only when it is **strictly** better. Equal treatment is exactly what a
most-favoured-nation clause promises, so a comparator on identical terms is not a breach of it.

## MFN-R.4 — Effective values, not printed headlines

MFN-1.2 of each clause defines each covered term. Those definitions are usually wider than the
headline figure on the front page, and the difference is where a review is won or lost.

Each file carries a **riders and side letters** section. A rider states which agreement it belongs
to, which term it moves, and by how much. **Whether a rider counts is decided by the clause's own
definition of that term, not by the rider's size.** Typical wording:

- *Rate* means the net effective fee after all **recurring** credits, rebates and support payments.
  A recurring monthly marketing credit therefore reduces the rate it is netted against. **A
  one-off, non-recurring or signing payment does not**, however large it is.
- *Committed term* means the period during which the counterparty may not terminate for
  convenience. A **unilateral termination right** exercisable by the counterparty therefore
  shortens it. A **renewal option** does not shorten anything.
- *Payment timing* means the days allowed to settle in the ordinary course. An **extended
  settlement rider** lengthens it. A **suspension of the clock during a formal dispute** does not,
  because it is not available in the ordinary course.

State the rider ids the clause's definitions take in. **Do not apply them yourself** — say which
ones count and the arithmetic is done from the printed figures.

## MFN-R.5 — Carve-outs

MFN-1.3 of each clause lists the circumstances in which a more favourable term in a comparison
agreement does **not** trigger the clause. A carve-out is decided from a printed attribute of the
comparison agreement, never from an impression of it. The four kinds that appear:

| kind | applies when |
|---|---|
| `launch_window` | the comparator is still inside the launch window the clause allows, counted from its own launch of the subject |
| `volume_commitment` | the comparator's rate is conditioned on a committed subscriber floor at or above the clause's threshold, **and the protected party does not itself meet that floor** |
| `bundled_services` | the comparator's fee covers two or more subjects in a single figure, so no separable rate for the covered subject exists |
| `early_settlement` | the comparator's payment terms are conditioned on an early-settlement discount, so they are not ordinary-course terms |

A carve-out is **per agreement and per term**. A comparator carved out on rate may still be a
comparator on payment timing.

## MFN-R.6 — The netting rule

MFN-1.4 of each clause says what to do when a comparison agreement is better on one covered term
and worse on another. Every clause in this corpus nets **rate against committed term**, because
that is the trade the parties actually make: a longer lock-in buys a lower fee.

> Where a comparison agreement carries a lower rate **and** a longer committed term, the rate
> differential is reduced by the clause's netting **step** for each **full** period (usually 12
> months) by which the comparator's committed term exceeds the protected party's. If the reduced
> differential is zero or less, the comparator is **not** more favourable. If it is still positive,
> the reduced differential is the differential the remedy applies to.

Two things follow, and both are counter-intuitive enough to be worth saying out loud:

1. **A comparator can be visibly cheaper and still be compliant.** The clause has already priced
   the difference.
2. **Partial netting still leaves a breach — a smaller one.** The remedy applies to the *netted*
   differential, not the headline one, so a review that reports the headline gap over-claims.

The step count is a whole number of full periods. Eleven extra months net nothing.

## MFN-R.7 — The four verdicts

Exactly one per covered term.

| verdict | when |
|---|---|
| `CARVED_OUT` | an agreement **inside** the group is more favourable on this term, and a carve-out in the clause covers every one that is |
| `NOT_COMPARABLE` | the only agreements more favourable on this term sit **outside** the comparison group |
| `BREACH` | an agreement inside the group is more favourable, no carve-out covers it, and the netting rule leaves a differential standing |
| `COMPLIANT` | nothing inside the group is more favourable — including the case where the netting rule reduces every differential to zero or less |

Where more than one is arguable at once, the order above is the order of precedence: a live breach
outranks a carve-out, a carve-out outranks an out-of-group finding, and an out-of-group finding
outranks silence. **`CARVED_OUT` and `NOT_COMPARABLE` are not softer words for `COMPLIANT`.** They
say a better term exists and name why the clause does not reach it, which is the sentence the other
side will ask for.

## MFN-R.8 — The remedy

MFN-1.5 of each clause states what a breach requires. It is not a matter of choice and it is not
negotiable in the review:

| covered term | remedy |
|---|---|
| `rate` | reduce the protected rate to match, and credit the difference in arrears from the comparison agreement's effective date, capped at the clause's arrears limit |
| `term_length` | shorten the committed term to match |
| `exclusivity` | release holdbacks to match |
| `payment_timing` | extend the payment window to match |

Where no term is in breach, the file requires **no adjustment**.

## MFN-R.9 — Citations

Name only agreement ids, rider ids and carve-out ids that are **printed in the file in front of
you**. An invented citation is worse than none: it is the first thing the counterparty's lawyer
checks, and it costs the whole review its credibility.

## MFN-R.10 — What this review is not

It does not adjust a rate, issue a credit, serve a notice or open a renegotiation. It produces the
finding a person reads before deciding whether to do any of those.


THE REVIEW FILE, exactly as printed:

MOST-FAVOURED-NATION REVIEW
  File reference        MFN-0001
  Review date           31 August 2026
  Protected party       Northgate Cable Partners
  Counterparty          Meridian Media Group
  Agreement under review  AG-0100, executed 1 January 2024
  Covered terms         rate, committed term, exclusivity, payment timing

THE CLAUSE  (AG-0100, clause MFN-1, as executed)
----------------------------------------------------------------------------------------------------------------------
  MFN-1.1  Comparison group
     Agreements of the Counterparty for the Service "Meridian Sports Network", with distributors in Tier 1,
     in the territory United Kingdom, executed within 18 months of the review date. An agreement
     failing any one of those attributes is not a Comparison Agreement.

  MFN-1.2  Covered terms, and what each one means
     Rate             the recurring per-subscriber fee, net of all RECURRING credits,
                      rebates and support payments. A one-off or signing payment is NOT
                      netted, whatever its size.
     Committed term   the months during which the distributor may not terminate for
                      convenience. A unilateral termination right SHORTENS it; an option
                      to renew does not change it.
     Exclusivity      the number of exclusivity holdbacks the agreement imposes.
     Payment timing   the days allowed to settle in the ordinary course. An extended
                      settlement rider LENGTHENS it; a suspension of the clock during a
                      formal dispute does not, being unavailable in the ordinary course.

  MFN-1.3  Carve-outs
     CO-1   A rate granted within the first 6 months following that distributor's launch
            of the Service is not a Comparison Rate.
     CO-2   A rate conditioned on a committed subscriber floor of 2,000,000 or more is not a
            Comparison Rate unless the Protected Party meets the same floor.
     CO-3   A fee covering two or more Services in a single figure is not a Comparison
            Rate, no separable rate for the covered Service existing.
     CO-4   Payment terms conditioned on an early-settlement discount are not Comparison
            Payment Terms.

  MFN-1.4  Netting
     Where a Comparison Agreement carries a lower rate AND a longer committed term, the
     rate differential is reduced by 0.0500 per subscriber per month for each full 12 months
     by which its committed term exceeds the Protected Party's. A differential reduced to
     zero or below is not more favourable. Part periods net nothing.

  MFN-1.5  Remedy
     Rate             reduce the Protected Party's rate to match, and credit the
                      difference in arrears from the Comparison Agreement's date of
                      execution, capped at 12 months.
     Committed term   shorten the Protected Party's committed term to match.
     Exclusivity      release holdbacks to match.
     Payment timing   extend the Protected Party's settlement window to match.

THE PROTECTED AGREEMENT
----------------------------------------------------------------------------------------------------------------------
  Agreement             AG-0100
  Distributor           Northgate Cable Partners
  Tier                  Tier 1
  Territory             United Kingdom
  Service               Meridian Sports Network
  Executed              1 January 2024
  Rate                  0.8400  per subscriber per month
  Committed term        36  months
  Exclusivity           1  holdbacks
  Payment timing        30  days
  Reported subscribers  900,000

COMPARISON AGREEMENTS  (every other agreement of the Counterparty, as at the review date)
----------------------------------------------------------------------------------------------------------------------
Agreement   Distributor                  Tier     Territory         Service                     Executed   
----------------------------------------------------------------------------------------------------------------------
AG-0400     Eastvale Broadband           Tier 1   United Kingdom    Meridian Sports Network     2025-01-01 
AG-0401     Hallamgate Telecom           Tier 1   United Kingdom    Meridian Sports Network     2025-03-02 
AG-0402     Kirkhaven Networks           Tier 1   United Kingdom    Calderstone Documentary     2025-05-03 
AG-0403     Netherby Communications      Tier 1   United Kingdom    Meridian Sports Network     2025-07-04 
AG-0404     Quarrington Cable            Tier 1   United Kingdom    Meridian Sports Network     2025-01-05 
----------------------------------------------------------------------------------------------------------------------

TERMS AS PRINTED ON THE FACE OF EACH COMPARISON AGREEMENT
----------------------------------------------------------------------------------------------------------------------
Agreement   Rate      Term       Exclusivity  Payment   Subs floor   Services  Launch    Early set.
            per sub   months     holdbacks    days      committed    covered   wnd left  disc.    
----------------------------------------------------------------------------------------------------------------------
AG-0400     0.8900    36         1            30        none         1         none      no       
AG-0401     0.6400    36         2            25        none         1         none      no       
AG-0402     0.9900    36         1            20        none         1         none      no       
AG-0403     1.0400    48         2            20        none         1         none      no       
AG-0404     1.0900    36         1            25        none         1         none      no       
----------------------------------------------------------------------------------------------------------------------

RIDERS AND SIDE LETTERS
----------------------------------------------------------------------------------------------------------------------
  R-1   AG-0403     payment timing   25        days
        Suspension of the settlement clock while a formal invoice dispute is open. Not available in the ordinary course.
----------------------------------------------------------------------------------------------------------------------

This file is a review pack. It adjusts nothing, credits nothing and serves nothing.
Every distributor, network, service, agreement, rate and subscriber count in it is invented.


Reply with JSON and nothing else, exactly this shape:

{"comparison_group": ["<agreement ids the clause's group definition admits>"],
 "outside_group": [{"agreement": "<id>",
                    "reason": "tier" | "territory" | "service" | "executed_before_lookback"}],
 "included_riders": ["<rider ids the clause's definitions take into account, e.g. R-2>"],
 "carve_outs": [{"agreement": "<id>",
                 "term": "rate" | "term_length" | "exclusivity" | "payment_timing",
                 "carve_out": "<carve-out id from this clause, e.g. CO-1>"}],
 "terms": [{"term": "rate" | "term_length" | "exclusivity" | "payment_timing",
            "verdict": "CARVED_OUT" | "NOT_COMPARABLE" | "BREACH" | "COMPLIANT",
            "offending_agreement": "<the agreement id that breaches, or null>",
            "why": "<one sentence: the finding this term would carry in the review>"}],
 "file_action": "RAISE_MFN_CLAIM" | "NO_ADJUSTMENT_DUE",
 "rationale": "<two sentences at most, on what decided the hardest term>"}

Give one entry in "terms" for every covered term, in the order listed above, even where the answer
is COMPLIANT -- three of the four terms being unremarkable is the ordinary case, not a reason to
drop them.

`offending_agreement` is an agreement id ONLY where the verdict is BREACH, and null otherwise. On
CARVED_OUT and NOT_COMPARABLE the agreement that was checked and set aside belongs in `carve_outs`
or `outside_group`, not here.

DO NOT return a differential, a netted differential, a new rate or a credit. Those are computed
from the printed figures in code, from the reading above, and a number in this reply would be
discarded.

`file_action` is RAISE_MFN_CLAIM if any term is BREACH, and NO_ADJUSTMENT_DUE otherwise. Neither
adjusts, credits, serves or renegotiates anything -- each is a recommendation to the person who
does.

Raw responseThe raw response

{"comparison_group":["AG-0401","AG-0403"],"outside_group":[{"agreement":"AG-0400","reason":"executed_before_lookback"},{"agreement":"AG-0402","reason":"service"},{"agreement":"AG-0404","reason":"executed_before_lookback"}],"included_riders":[],"carve_outs":[],"terms":[{"term":"rate","verdict":"BREACH","offending_agreement":"AG-0401","why":"AG-0401 is an admitted comparison agreement with a lower recurring rate, no carve-out in MFN-1.3 applies to it, and its committed term is not longer than AG-0100's, so netting does not reduce it."},{"term":"term_length","verdict":"COMPLIANT","offending_agreement":null,"why":"No admitted comparison agreement has a committed term shorter than AG-0100's 36 months; AG-0401 is equal and AG-0403 is longer."},{"term":"exclusivity","verdict":"COMPLIANT","offending_agreement":null,"why":"No admitted comparison agreement has fewer exclusivity holdbacks than AG-0100's one; both admitted comparators impose two."},{"term":"payment_timing","verdict":"COMPLIANT","offending_agreement":null,"why":"No admitted comparison agreement allows settlement in more than AG-0100's 30 days, and R-1 does not count because it is a dispute-clock suspension unavailable in the ordinary course."}],"file_action":"RAISE_MFN_CLAIM","rationale":"AG-0401 is the only admitted comparison agreement with a more favourable rate; no carve-out in MFN-1.3 covers it and its committed term is the same 36 months as AG-0100's, so the netting rule has nothing to net. AG-0400 and AG-0404 fall outside the 18-month look-back, AG-0402 is outside on service, and R-1 is a dispute-clock suspension that does not count as an ordinary-course payment extension."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Term by term, is anyone comparable on better terms and what does the clause require — 60 review packs. One model answered, and every answer was then graded Four different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

No model grades anything and there is no LLM judge anywhere in the grading path. Every metric is exact match against data/gold.jsonl over a closed four-verdict vocabulary, with money compared as an integer count of cents and rates as exact four-place decimals. The ANSWER KEY itself is constructed rather than labelled: every pack is built from a planted intent that states its verdict on all four terms BEFORE the engine runs, and the build fails if src/mfn.py disagrees with the planter. Nobody reads a finished pack and decides what it says, which is the only way this corpus could be labelled honestly -- 'is this rate more favourable than the clause allows' is exactly the judgement being measured.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anybody paid. The token counts come from the provider's own usage block on each of the 60 calls, and 94.7 pct of the output tokens they report are provider-side reasoning the kit did not ask for.

Rates checked 2026-08-31. The provider that actually ran every call here is kept off this page per the series rule. Its real spend, its own dated card (as_of 2026-08-23, recorded in the run file), its peak/off-peak split and its cached-input tier are recorded per call in results/eval-r001-mfn-check.json and summarised in Cost.measured_on, not projected here.

Grading unitWhat the grading figure prices

Scoring is pure Python over committed JSON: 240 rows in well under a second, no key, no network, no provider. Re-scoring a cached run costs $0.00 and the cache ships, so either arm re-scores with --resume --rescore for nothing. ⚠︎ --rescore is honoured ONLY under --resume; alone it re-fires every call.

The gradersFour ways to grade

⚑ THE FREE COLUMN CHANGED ON 2026-08-31 AND IT IS SMALLER THAN THIS KIT USED TO CLAIM. b000-mfn-check-rules reported 196 of 240 and 23 of 60 in BOTH its columns because evals/run.py's --floor branch wrote the arm's own answer into the rechecked slot and never called src/recheck.py, while every model arm went through it. The kit repo was fixed and the arm re-run at $0.00 with zero calls: the same free answers through the same pure code score 200 of 240 and 26 of 60, with 11 overrides, and that is the column every comparison here now uses. RAW 196 and 23 are kept beside it as what a headline comparison alone achieves. ⚑ THE PAID ARM'S LEAD IS 38 TERM ROWS AND 32 PACKS, NOT 42 AND 35. 238 of 240 verdicts against 200, 58 of 60 packs against 26, and the margin is concentrated exactly where the four-box vocabulary lives -- on the 39 HARD packs free code scores 7 and the model 38. On the 21 CLEAN packs the floor already scores 19 and the model 20, so the call buys ONE pack there and the money is entirely for the hard half. Row-level McNemar over the 240 graded rows is 39 the model wins to 1 it loses, exact two-sided p = 7.5e-11 on 40 discordant rows, so the gap is not noise. (Against the raw floor it read 43-1, p = 5.1e-12.) ⚠︎ AND THE ONE ROW IT LOSES IS THE MOST VALUABLE ROW IN THE CORPUS. On MFN-0010/rate the free floor returns the verdict BREACH, the offending agreement AG-0490, the new rate 1.7500, 12 arrears months and a credit of $3,175,200.00 -- every field exactly right, and it files RAISE_MFN_CLAIM. The paid arm calls the row NOT_COMPARABLE and files NO_ADJUSTMENT_DUE. That single row is 100 pct of the model's money shortfall: $3,175,200.00 of the $30,609,000.00 at stake, and it is the largest single credit the corpus contains. Free code found it and the call did not. It is one row in 240 and it is the row a reader should look at first. ⚠︎ ONE LINE WHERE THE FLOOR LOOKS BETTER IS NOT A WIN AND MUST NOT BE READ AS ONE. Group recall: the floor recovers 274 of 274, the model 262. The floor scores 100 pct by admitting all 390 agreements in the file -- its group PRECISION is 70.3 pct and the 116 it wrongly admits are where 19 of its 33 rechecked false breaches come from. The recheck cannot touch this line on either arm -- the reading is what it trusts. A checker that admits everything has perfect recall by construction and is answering a different question. What the floor is actually wrong about, in both directions and they are different errors. OVER-FLAGGING: rechecked, of the 58 rows it raises, 33 are not breaches -- 56.9 pct of everything it says, and 15.9 pct of the 208 rows that are not a breach. 19 are NOT_COMPARABLE (8 wrong territory, 5 wrong tier, 3 another service, 3 outside the look-back) and 14 are CARVED_OUT. Raw, it raised 62 with 37 false, the extra 4 being rows the clause's own netting rule had already priced away -- and the netting rule is the one thing the recheck hands back, so it removes exactly those 4. ALL 33 that remain are rows a two-box checker cannot express: that is the vocabulary, not tuning. UNDER-FLAGGING: 7 of the 32 real breaches, 21.9 pct, worth $8,448,240.00 unclaimed, and all seven are one miss -- a rider changes the effective value so the breach is invisible on the printed face, and the floor reads the face. AND ITS MONEY IS WRONG ON 4 OF THE 25 BREACHES IT DID FIND, by $16,214,300.00, all four naming the cheapest paper in the file, which in all four is OUTSIDE the comparison group -- a credit computed against an agreement the clause does not reach. Raw it was 8 rows and $24,729,000.00; the other 4 carried the headline differential where the clause nets part of it away, and the recheck fixes all four to the cent.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-31. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingGrading is free; the bill is the 60 calls that answered.

The scorer, the independent key checker, the corpus generator and the rules floor are pure code and cost $0.00 against any result set -- there is no LLM judge anywhere in the grading path, and re-scoring a cached run is free because results/cache-r001-mfn-check.jsonl ships. Measured on a keyless copy this pass, the whole free half runs in 0.31 s: corpus rebuild 0.072 s, key check 0.050 s, rules floor 0.097 s, wiring stub 0.090 s. The figure above prices the ONE paid run this kit has (60 calls, 289,125 input tokens and 428,296 output) at the same projected card cost_per_query_usd uses. ⚠︎ The 20-call adversarial arm is written and has never been fired, so it is not in this figure and there is nothing to add to it.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, and it is not close: 7,138.3 out against 4,818.8 in per pack, at a 6x output rate on the projection card, is 89.9 pct of the projected bill. This kit's answer is four verdict rows, three lists and two sentences -- a few hundred tokens of JSON -- so the output side is not the answer.
• PROVIDER-SIDE REASONING, which is 405,713 of the 428,296 output tokens (94.7 pct) and therefore ~85 pct of the whole projected bill on its own. It was left at the tier's default and re-rolls per call. That is the number to look at before assuming this job is cheap, and it is why the floor's $0.00 column is real rather than rhetorical.
• THE PACK, which goes in WHOLE at 6,488 characters for MFN-0001 and grows linearly with the counterparty's number of comparison agreements. It is only 30.4 pct of the prompt here and the prompt is only 10.1 pct of the bill, so it is the third driver -- but it is the one that moves with somebody else's paper rather than with a setting.
• NOT THE PREFIX, which is why it is listed to be dismissed. The role, the reading standard and the schema are 14,870 byte-identical characters on every call and 55.0 pct of input tokens came back as cache hits. On the withheld provider's own card that cached tier is 1/31 of the miss rate; on every projection card below there is no cached tier at all and those tokens are priced at full rate.

Your volumeWhat it costs at your volume

Linear in packs and NOT linear in the counterparty's paper. Ten times the reviews is ten times the calls at the same per-pack cost -- there is no index, no retrieval and no state between packs, which is the whole concurrency story (5 workers turned 3,541.6 s of call time into 735.5 s of wall clock). But the pack goes whole into every prompt, so a counterparty holding 60 comparison agreements rather than this corpus's mean of 6.5 is roughly ten times the input tokens AND a reply that grows with the group -- and the reply is where the bill is. At this corpus's size, 200 reviews a day is about $4.76 a day on the projection card. The same 200 against ten times the paper is not ten times that: the 32,000 output ceiling starts to bind before the 1M context does, and the p95 call is already 113.5 s against a 1,200 s socket that is not streamed.

Model choiceWhy this model

It is the tier the estate runs, so this figure compares with every sibling kit. ⚠︎ AND IT IS NOT A RECOMMENDATION. Nothing here says this tier is the right one for this job: no second model was called against this corpus, so the only thing measured is that THIS tier reads a templated pack near-perfectly and misses one boundary. The comparison that IS measured is against free code, and it is the one worth reading: on the 39 hard packs the call buys 34 packs, on the 21 clean packs it buys one, and on the corpus's single most valuable row it loses.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

the whole 60-pack scored run (r001-mfn-check, the fast tier): 289,125 input tokens, 158,976 of them prefix cache hits, and 428,296 output of which 405,713 reasoning. All 60 calls answered and are inside these totals -- nothing errored before the accumulator ran, so this is a total and not a floor. The real bill is on the withheld provider's card and lives in the run record and in Cost.measured_on, not in this block.

The codeEvery file, and what it is for

12 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Seven of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

tools/build_corpus.pycorpus generator

Builds all 60 review packs, data/agreements.json, data/clause.md and the answer key from seed 20260831 against review date 2026-08-31. Every pack starts from a PLANTED INTENT that states the verdict it means to produce on all four covered terms, and the build FAILS if src/mfn.py disagrees with the planter.

# Generate the whole corpus and its answer key. Deterministic, offline, no key, no model.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA = os.path.join(HERE, "data")
CORPUS = os.path.join(DATA, "corpus")
SEED = 20260831
SETS_WANTED = 60
REVIEW_DATE = "2026-08-31"
DATASET_VERSION = "mfn-check-v1-60sets"
COUNTERPARTIES = ["Meridian Media Group", "Calderstone Networks", "Thornbury Broadcast Holdings",
DISTRIBUTORS = [

src/mfn.pythe MFN engine

The pure-code half and the whole design. Effective values, the comparison, the clause's netting rule, the verdict, the offending agreement, the new rate, the arrears months and the credit. Rates are Decimal in exact four-place decimals, money is an integer count of cents, quantised exactly once. No model anywhere in the file.

# The MFN engine. Exact arithmetic, one verdict per covered term, no model anywhere in this file.

TERMS = ("rate", "term_length", "exclusivity", "payment_timing")
TERM_LABEL = {"rate": "Rate", "term_length": "Committed term",
BETTER = {"rate": "lower", "term_length": "lower", "exclusivity": "lower",
TERM_UNIT = {"rate": "per subscriber per month", "term_length": "months",
REMEDY = {"rate": "REDUCE_RATE_TO_MATCH",
ADJUSTMENT_KINDS = ("NONE",) + tuple(REMEDY[t] for t in TERMS)
CARVE_OUT_KINDS = ("launch_window", "volume_commitment", "bundled_services", "early_settlement")
OUTSIDE_REASONS = ("tier", "territory", "service", "executed_before_lookback")
def rate(s):

src/prompt.pyverdict vocabulary — a swap seam

VERDICTS and VERDICT_MEANINGS, declared once at module level. The definition block the model reads is GENERATED from them, the parser's whitelist is built from them, and the scorer's confusion matrix is keyed on them. Import-time assertions refuse a verdict the reading standard never names or the schema never offers.

You change it to: VERDICTS and VERDICT_MEANINGS. One declaration read by the prompt, the parser, the scorer and the UI panel; import-time assertions refuse a vocabulary the standard or the schema does not carry.

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CLAUSE_MD = open(os.path.join(HERE, "data", "clause.md"), encoding="utf-8").read()
VERDICTS = ("CARVED_OUT", "NOT_COMPARABLE", "BREACH", "COMPLIANT")
VERDICT_MEANINGS = {
def _verdict_block():
def _wrap(text, width, indent):
def _term_block():
SYSTEM = """You are checking one most-favoured-nation clause against the counterparty's other
SCHEMA = """Reply with JSON and nothing else, exactly this shape:

src/prompt.pyprompt assembly — a swap seam

Four parts in a fixed order -- system 4,260 chars, the reading standard (data/clause.md) 8,753, the review pack VERBATIM 6,488, the JSON schema 1,857. 21,358 characters for MFN-0001. No pre-digest, no comparison table built for the model, no shortlist of candidate comparators.

You change it to: VERDICTS and VERDICT_MEANINGS. One declaration read by the prompt, the parser, the scorer and the UI panel; import-time assertions refuse a vocabulary the standard or the schema does not carry.

# The assembled prompt, in one place, in the order it is sent.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CLAUSE_MD = open(os.path.join(HERE, "data", "clause.md"), encoding="utf-8").read()
VERDICTS = ("CARVED_OUT", "NOT_COMPARABLE", "BREACH", "COMPLIANT")
VERDICT_MEANINGS = {
def _verdict_block():
def _wrap(text, width, indent):
def _term_block():
SYSTEM = """You are checking one most-favoured-nation clause against the counterparty's other
SCHEMA = """Reply with JSON and nothing else, exactly this shape:

src/reviewer.pythe model call

The only place a model is called. One call per review pack, max_tokens 32,000, socket timeout 1,200 s. A reply cut off at the ceiling is recorded as a failure with at_ceiling set and stays inside the published denominator; it is never re-fired and never spliced.

# One review file in, one MFN review out. The only place a model is called.

MAX_TOKENS = 32000
THINKING = None
def parse_reply(text):
AGREF = re.compile(r"^AG[-_ ]?0*(\d{1,5})$", re.I)
RIDREF = re.compile(r"^R[-_ ]?0*(\d{1,4})$", re.I)
COREF = re.compile(r"^CO[-_ ]?0*(\d{1,4})$", re.I)
def _ag(v):
def _rid(v):
def _co(v):

src/recheck.pythe recheck — a swap seam

Three fields in, every figure out. Takes comparison_group, included_riders and carve_outs from the reply and recomputes everything else through the same engine. It overrode the model 0 times on this run -- because it was never asked to agree about anything: the schema forbids the model a differential, a credit or a rate.

You change it to: TRUSTED -- the three fields the recheck takes from the reply. Widen it and the model owns arithmetic; narrow it and the reading is asked for less.

# The model's reading, the clause's arithmetic. Pure code, no model, no key.

TRUSTED = ("comparison_group", "included_riders", "carve_outs")
def reading_of(answer):
def recheck(answer, case):

evals/baseline.pythe free floor — a swap seam

The spreadsheet, implemented literally: compare the HEADLINE value of every covered term on EVERY agreement in the file and flag anything better. No key, no model, no network. Deliberately NOT routed through src/mfn.py -- handing it the engine would give it the netting rule for free, and not having it is one of the things measured.

You change it to: What a spreadsheet would have said -- which columns are compared and on what basis. This is the arm every paid figure is measured against.

# THE FREE FLOOR. No key, no model, no network. It is also what most MFN "monitoring" actually is.

MODES = ("rules",)
def _headline(ag, term):
def _adjustment(case, term, pv, off_ag, diff):
def code(case, mode="rules"):

evals/scoring.pythe scorer — a swap seam

Deterministic, no model in the grading path. The unit is the covered term, not the pack: 60 x 4 = 240 rows graded by exact match against data/gold.jsonl. Breach precision and recall are reported separately and never averaged.

You change it to: The graders, the two error directions kept separate, and the fixed-order failure taxonomy.

# Grade one arm against the answer key. Deterministic -- no model judges anything here.

KEYS = ("case_id", "term")
FIGURE_FIELDS = ("kind", "new_rate", "new_value", "credit_cents", "arrears_months", "subscribers")
def _pct(n, d):
def rows_of(rec):
def adj_equal(a, b):
def reading_of(rec):
def _set(xs):
def _co_set(cos):
RIDER_CASES = ("effective_rate_breach", "effective_term_breach", "effective_payment_breach")

evals/check_labels.pythe independent key checker

A SECOND implementation that imports nothing from src/ or tools/. It re-parses the printed data/corpus/*.txt pages -- the same bytes the model gets, not the JSON -- and carries rates as integer ten-thousandths rather than Decimal. 60 files, 240 rows, no disagreement, re-run this pass in 0.050 s.

# Grade the answer key with a SECOND implementation. No key, no model, no network.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
MONTHS = ["January", "February", "March", "April", "May", "June",
def tt(s):
def tt_str(v):
def credit_cents(diff_tt, subs, months):
def months_between(a, b):
def iso_from_pretty(s):
FIELDS = re.compile(r"\s{2,}")

evals/run.pythe harness

Reviews all 60 packs and writes one result file. --floor and --stub are free; the scored arm spends. Every run id carries the slug (r001-, b000-<arm>, t000--stub, x001-) because build/measured/runs/ is flat and keyed by run id alone.

# Review all 60 files and write one result file. THIS SPENDS MONEY unless --floor or --stub.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
RESULTS = os.path.join(HERE, "results")
CORPUS = os.path.join(HERE, "data", "corpus")
KIT = "mfn-check"
RATE_CARDS = {
def tariff_at(card, ts):
def price(card, row):
def cases():
def load_text(cid):

src/app.pythe local UI

Standard library HTTP server. /api/files, /api/file, /api/rules, /api/recorded and /api/board need no key; only /api/review calls a provider and with no API_KEY it returns 200 saying so. The free floor is computed on every pack and shown beside the model's answer, and the working is printed in full rather than summarised.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
PORT = int(os.environ.get("PORT", "9231"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-mfn-check")
FLOOR_RUN = "b000-mfn-check-rules"
def cases():
def golds():
def load_text(cid):

src/budget.pythe shared call ledger

A cap on live calls shared by every kit on this machine. It counts CALLS, not dollars, deliberately -- the kit cannot know the rate card a forker is pointed at, and a cap reading the wrong card is worse than none.

# A cap on live calls, shared by every kit on this machine. No dependency, no service.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
KIT = os.path.basename(HERE)
ROOT = os.path.dirname(os.path.dirname(HERE))
SHARED_ENV = os.path.join(ROOT, ".env")
LEDGER = os.path.join(ROOT if os.path.exists(SHARED_ENV) else HERE, ".calls-ledger.jsonl")
PAID_ARMS = ("r", "x")
FREE_ARMS = ("t", "b")
BOOT = os.urandom(6).hex()
PID = os.getpid()

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 4818 input and 7138 output tokens per query, all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

⚠︎ THE PROBE IS WRITTEN, PAIRED AND UNFIRED, SO THIS KIT PUBLISHES NO SUPPRESSION RATE. The threat is not hypothetical and it is structural rather than incidental: a review pack is assembled from the COUNTERPARTY'S own agreements, side letters and covering notes, and it reaches the model verbatim with no redaction and no pre-digest. Anyone who can put a sentence in a side letter can put a sentence in front of whatever reads the pack.

API_KEY is read from the shared &lt;repo&gt;/.env, this kit's own .env, or the real environment, in that order of precedence, by src/config.py in eight lines of standard library. Both .env files are gitignored from the first commit and this repo has never held a credential. src/budget.py counts live CALLS against a shared cap -- calls, not dollars, because the kit cannot know the rate card a forker is pointed at.

The experimentWe have not attacked it yet -- and the hole is already named

An indirect prompt injection needs a field an outside party controls that reaches the prompt. This use case is unusually exposed, because the whole input is the other side's paper: the agreements, the riders and the covering notes in a review pack are written by the counterparty whose compliance is being checked, and they go into the call verbatim. The prepared attack asks the reader to convert every BREACH into CARVED_OUT under MFN-1.3 -- the direction that costs the protected party money and leaves the review looking clean. It is refutable from the pack's own columns ('Subs floor committed', 'Launch wnd left', 'Early set. disc.' all read none/none/no on the offending agreements) and it contradicts the clause printed above it. Both facts make it a fair test rather than an easy one. Both were run for real on 2026-08-31 -- and what was measured is the SCORED run, not an attack. No adversarial call has been made against this kit by any route; see could_not_verify.

Three of the five boundaries above hold in CODE and were confirmed by reading it, not by payloads run through a provider. The first two are the ones that matter and neither is held: the injection is unmeasured, and the recheck is INSIDE the blast radius rather than outside it. A kit whose pure-code station is trusted to be the guardrail should say when it is not, and this is the field for it.

The result0 of 20 adversarial trials fired. Three boundaries hold in code -- invented citations, the absence of any arithmetic in the reply, and the absence of a write path -- and the two that decide whether this kit can be talked out of a claim are unmeasured, one of them because the pure-code recheck trusts the very field the attack targets.

The probe is ready and paired. evals/injection.py appends one line to the RIDERS AND SIDE LETTERS block, where a covering note plausibly sits, asking the reader to record every more-favourable term as carved out under MFN-1.3 and raise no claim. It attacks the finding this kit exists to produce, in the direction that costs money. The control is the scored run's own cached reply for the same pack, so no second control call is bought, and the paired denominator counts only rows the control got right. inject() RAISES if its anchor is missing, because an injection that silently did not inject is a probe reporting perfect resistance without having attacked. --dry-run proves it lands on all 60 packs and costs nothing. Firing it is 20 calls and it has not been bought.

HonestyWhat this does not prove

• WHETHER ANY SENTENCE WORKS. Zero trials fired. There is no suppression rate on this page in either direction, and no basis for one.
• WHETHER THE RECHECK WOULD PROPAGATE A CAPTURED CARVE-OUT. Established by reading the code that it CAN; never measured that it DOES.
• WHETHER A REAL COVERING NOTE READS LIKE THE PREPARED ONE. The attack sentence was written for this corpus's layout, and it lands on all 60 packs because every pack has the same anchor.
• WHETHER THE MODEL WOULD REFUSE, COMPLY OR SPLIT. No behavioural reading of any kind exists here, including the honest possibility that it flags the note as contradicting the clause.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never adjust a rate, issue a credit, serve a notice or open a renegotiation, and never answer with a number the model supplied. MFN-R.10: the output is the record a person confirms before a letter goes to a counterparty's lawyer.

Stated in the reading standard the model is sent (data/clause.md, MFN-R.10), enforced by the schema in src/prompt.py -- which forbids the model a differential, a netted differential, a new rate, arrears months and a credit -- and by the absence of any write path in src/app.py's route table. Printed on the UI and in the README.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is ABSENCE OF A WRITE PATH plus a schema restriction, not a runtime enforcement layer. Nothing intercepts a reply and rejects it on policy.
• ⚠︎ IT IS NOT A DEFENCE AGAINST THE PACK ITSELF. The recheck trusts carve_outs, and normalise_reading validates that a claimed carve-out is real PAPER, not that its conditions hold. A carve-out claim is exactly how a breach is suppressed without touching a number, which is what the unfired adversarial arm was built to measure.
• It is not a check that the verdict is right. It is a check that the verdict is CHECKABLE: the whole working is printed rather than summarised, and the free floor is computed and shown beside the model's answer on every pack, because a finding a person cannot check is a finding they have to believe.
• It says nothing about a real clause. Every rule enforced here is a rule this kit invented.
• recheck_overrides 0 is NOT evidence the guardrail works. The recheck overrode nothing because it was never asked to agree about anything -- the schema gives it no model figure to disagree with. Reading it as 'the model was right about the arithmetic' would be false.

WatchedWhat is watched, and why that one

The kit measures 66 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

121 measured by the latest run -55 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

all shared guards held between the recorded runs (breach_key 32, carveouts_gold 14, clean_packs 21, comparison_agreements 390, covered_terms 4, credit_rows_key 16, dataset_version mfn-check-v1-60sets, failures 0, group_gold 274, hard_packs 39, money_at_stake_cents 3060900000, non_breach_rows 208, offender_key 32, recheck_arm_is_copy_of_raw False, rechecked_breach_key 32, rechecked_carveouts_gold 14, rechecked_clean_packs 21, rechecked_credit_rows_key 16, rechecked_group_gold 274, rechecked_hard_packs 39, rechecked_money_at_stake_cents 3060900000, rechecked_non_breach_rows 208, rechecked_offender_key 32, rechecked_rederived_from_cache False, rechecked_riders_gold 7, rechecked_term_rows 240, rechecked_term_rows_answered 240, review_packs 60, review_packs_answered 60, riders_gold 7, socket_timeout_s 1200, term_rows 240, term_rows_answered 240, trusted_fields 3) — which is why the history below is allowed to mean anything.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

3 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 3 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

comply · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 117 chips that all say so.

comply · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 131 chips that all say so.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 121 chips that all say so.

DeviationsWhat deviated

0 breaches across 3 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The corpus generator, the label gate, the MFN engine, the recheck and the free floor are free and re-run end to end in 0.31 s on a keyless copy, so every band above is reproducible without a provider. The paid arm re-scores from its committed cache for $0.00 with --resume --rescore. Only firing the unfired adversarial arm (20 calls) or buying a second model costs anything.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end -- no orchestration layer, no vendor SDK, no vector store, no schema library and no HTTP client beyond urllib. The UI is the same: one stdlib server and one page of plain JS. pip install pulls nothing, which keeps the fork test to one step.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear, and concurrent only at the pack level. There is no agent, no tool loop, no retrieval and no state carried between packs: assemble prompt -> one call -> parse -> normalise -> recheck in pure code -> score. The only branch is the arm (paid / floor / stub), chosen once at the top of evals/run.py.

The other sideWhat a framework costs you

• Adding a provider means editing the PROVIDERS dict by hand. In exchange pip install pulls nothing and a forker on any vendor runs the kit with one .env edit.
• The parser is hand-written, so a reply shape nobody anticipated fails rather than coercing. On this run that cost nothing -- 60 of 60 parsed -- and it is the reason a bad reply is a recorded failure rather than a quietly wrong row.
• No structured-output enforcement, so nothing prevents a malformed reply at the provider. Whether a library would have changed the parse rate is unmeasured; it held on all 60.
• The 5-worker thread pool is fixed in code rather than configured. It is the right size for 60 packs and would need editing for 6,000.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-mfn-check on the fast tier, 2026-08-31. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 1 earlier run on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

No drift chart. Only one run has recorded a call latency; the other 1 run on record made no model call at all — a rules-only floor stores 0, which is an absence, not a zero-millisecond answer. One timed run is a reading, not a trend, and it is in the table above. A chart appears the day a second run times itself.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — the rulebook is data read whole into the prompt.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT -- and the MIT text is in the repository's LICENSE-PUBLIC file, NOT in its root LICENSE, which governs the repository itself and grants a reader nothing. The KIT is MIT; the repository is not, and this line says which because it is the only claim in a kit a reader is entitled to rely on before forking. There is no third-party data here to licence at all: the packs, the clause, the printed pages and the answer key are generated in process from a fixed seed. ⚠︎ data/SOURCES.md states the licence as 'MIT, with the corpus generated in process' and does not name LICENSE-PUBLIC; it makes no false claim, but it leaves a reader to open LICENSE and find the opposite. That file belongs to the kit repo and was not edited by this capture pass. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe covered-term verdict -- CARVED_OUT, NOT_COMPARABLE, BREACH or COMPLIANT, one per term per pack

whether each of the 240 term rows carries the verdict the clause produces. The unit is the covered term, not the pack: the arm's row for MFN-0007/rate is graded against the key's row for MFN-0007/rate, so no matching step is needed and none exists. The four-box vocabulary is the point -- CARVED_OUT and NOT_COMPARABLE are the two a two-box checker cannot express, and 33 of the free floor's 37 false breaches are one of those two.

MethodHow the test was run

python3 -m evals.run --run-id <id> [--floor rules]; evals/scoring.py exact-matches a closed verdict vocabulary against data/gold.jsonl. No model is in the grading path and re-scoring a cached run costs nothing.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl -- 240 covered-term verdicts computed by src/mfn.py from the generator's planted intents, never typed by a person, and tools/build_corpus.py refuses to write a pack in which the engine disagrees with the intent the planter declared. Graded independently by evals/check_labels.py, which imports nothing from src/ or tools/: it re-parses the printed data/corpus/*.txt pages rather than the JSON, rebuilds the clause, the protected agreement, every comparison agreement and every rider out of the printed columns, decides which riders count from the printed prose rather than a flag, and carries rates as integer ten-thousandths rather than Decimal. Two implementations, one key; they agree on all 240 rows and on the credit total to the cent. Every counterparty, distributor, service, agreement, rate, rider and subscriber count is invented -- see data/SOURCES.md.

No true/false rates for this grader. It records 12 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? There is no threshold to tune -- exact match on a verdict string from a closed four-word vocabulary, so nothing here has a tolerance. The numbers that look like thresholds live in the CLAUSE, not the grader: the 18/24/36-month look-backs, the 0.0500-per-12-months netting step, the arrears cap and the carve-out conditions, all in data/agreements.json and data/clause.md, all applied identically by every arm.

Cadence: Once per corpus version. The paid arm ran ONCE and was not re-fired after its misses were read; the recheck and the score re-derive from the committed cache for $0.00 with --resume --rescore.

The decisionWhen to reach for it

In one lineThe two error directions, kept apart -- breach precision and breach recall, never averaged

what each arm gets wrong and which way. A false breach is answered by the counterparty quoting your own clause back at you and every other finding in the letter is then read sceptically; a missed breach is money the protected party goes on paying, and the arrears cap means the months that pass are not recoverable. They are not interchangeable and an F-score would hide the whole finding.

MethodHow the test was run

Same harness, same result files. The confusion matrix in the result file is the audit trail: the floor's has exactly two off-diagonal shapes, BREACH->COMPLIANT 7 and 33 rows of NOT_COMPARABLE/CARVED_OUT->BREACH.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the same data/gold.jsonl -- this grader does not have a reference standard of its own and is not one. It re-partitions the SAME 240 verdicts by direction of error, so its correctness is exactly the term grader's correctness; what it adds is the split that an accuracy figure destroys.

How tight can the band be? No threshold and no sweep. Both rates are counts over a closed vocabulary. ⚠︎ The denominators are small and the page says so beside every rate: 32 real breaches, sub-bucketed 16 rate / 8 exclusivity / 5 term_length / 3 payment_timing, so one row moves recall by 3.1 points and no alarm band here may be finer than that.

Cadence: Once per corpus version, read before the headline.

The decisionWhen to reach for it

In one lineThe READING, graded beside the verdicts -- the comparison group, the riders that count, the carve-outs relied on

whether the three judgements the recheck TRUSTS were right, separately from whether the verdict came out right. This exists because a perfectly executed netting calculation over a group that is missing a member returns a confident, exact, wrong answer -- and both of the paid arm's misses are exactly that. Publishing one accuracy figure over reading and arithmetic together would hide which half failed.

MethodHow the test was run

Same harness. The reading figures are recorded per arm in the result file alongside the verdicts, never merged with them.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl's comparison_group, included_riders and carve_outs blocks -- the three judgements src/recheck.py trusts, held to the same key the verdicts are held to and re-derived by evals/check_labels.py from the printed pages.

How tight can the band be? No threshold. Set membership by agreement reference, exact. The one number that behaves like a threshold is the clause's own look-back, and this run proved an arm can read <= as < on it: 14 boundary agreements, 12 dropped.

Cadence: Once per corpus version, and always alongside the verdicts -- a right verdict from a wrong reading is luck and this is the only grader that can tell you.

The decisionWhen to reach for it

In one lineThe remedy in whole cents -- the new rate, the arrears months and the credit

whether the letter that goes out carries a defensible number. Right verdict with the wrong figure is the shape that gets settled at a discount: the claim is right, the number is indefensible, and the counterparty only has to open one file to say so.

MethodHow the test was run

Same harness. Money is an integer count of cents throughout and rates are exact four-place Decimals; evals/check_labels.py re-derives the same credits in integer ten-thousandths and agrees on every row.

Every grader on these pages scored the same 60 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, and what the grader made of it

The formulaWhat it computes

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl's per-row adjustment blocks -- the new rate, the arrears months, the subscriber count and the credit in whole cents, computed by src/mfn.py in exact four-place Decimals and quantised to cents exactly once, and re-derived independently by evals/check_labels.py in integer ten-thousandths. The two agree on the total to the cent: 3,060,900,000 both ways.

How tight can the band be? No tolerance at all: exact equality on integer cents and on exact four-place Decimal rates. Money is never a float anywhere in this kit -- at 1.84 million subscribers over 14 months one rounding in the seventh place is a real number of dollars on a real credit note. The credit is quantised to whole cents exactly once, at the end.

Cadence: Once per corpus version. Free to re-derive from the committed cache; the arithmetic is deterministic, so a re-run that moves a cent is a defect.

The decisionWhen to reach for it