The business caseThe problem this solves

A transfer transcript has arrived, an adviser has proposed each course against a degree requirement, and someone has to decide which of those proposals actually satisfy. It is not a lookup, and the reason is the thing every first version gets wrong: EQUIVALENCE IS NOT STRING MATCHING. Calculus III reads like Calculus II and articulates to MATH-203. Algebra-based College Physics II reads like the calculus-based requirement and articulates to PHYS-102. Managerial Accounting reads like Financial Accounting. Every one of those is creditable toward the degree total and satisfies nothing, and telling a student the difference late costs them a term. And the reverse costs the same: an addendum executed after the equivalency table was drawn, a sending institution that moved to a semester calendar so the quarter units printed are already semester units, a department chair's certification of upper-division equivalency -- all three look deficient on every table in the packet and all three satisfy. The named trap is DUPLICATE CREDIT, and half of it is in no document in the packet at all: an advanced-placement or dual-enrolment award posted years before the transfer transcript existed, recorded only in a sentence of registrar correspondence, with every table saying the application satisfies. Someone reading a transfer transcript line by line against a catalog: looking each sending course up in the articulated equivalency table, checking what it articulates TO against what the requirement NAMES, converting quarter units to semester units and comparing them to the minimum, checking the grade against the catalog minimum for that requirement, checking whether an upper-division requirement is being met with lower-division work, checking whether the requirement is one that must be completed in residence, checking what has already been posted against it -- and then reading the registrar correspondence to find out which of those apparent deficiencies somebody has already answered, and which perfectly ordinary-looking application is for a requirement that is already satisfied.

The inputThe actual transfer credit evaluation packets

The corpus is 40 transfer credit evaluation packets, 0.38 MB (txt 40). The thing being measured has to be PLANTED to be measured. The question is whether an evaluation says a course SATISFIES a requirement it does not satisfy, and to count that you have to know which applications genuinely satisfied. A real archive holds the decision that was POSTED, not the decision that was right -- and the fields it turns on are education records: transcripts, grades, prior awards, holds and adviser notes. A corpus that redacted them would not carry the question.

Transfer Credit Evaluation
----------------------------------------------------------------
  Packet               TC-0001
  Receiving programme  BS Mechanical Engineering
  Catalog year         2026-2027
  Sending institution  Marbleshaw State University
  Sending calendar     semester
  Evaluation opened    2026-05-20
  Basis                articulated equivalencies and the catalog's requirement rules

Catalog Rules
----------------------------------------------------------------

  Degree requirements

  RQ-ACC1   Financial accounting                    named      ACCT-210          3.00  C   lower   transferable   AR-2.1
  RQ-BIO1   General biology I                       named      BIOL-160          4.00  C   lower   transferable   AR-2.1
  RQ-CHM1   General chemistry I                     named      CHEM-150          4.00  C   lower   transferable   AR-2.1
  RQ-DIV1   Global and diverse perspectives         category   CAT-GLOBAL        3.00  D   lower   transferable   AR-2.4
  RQ-ECO1   Principles of microeconomics            named      ECON-201          3.00  D   lower   transferable   AR-2.1
  RQ-ETH1   Professional ethics, in residence       named      PHIL-410          3.00  C   upper   residency      AR-2.1
  RQ-MTH2   Calculus II                             named      MATH-202          4.00  C   lower   transferable   AR-2.1
  RQ-MTH3   Linear algebra                          named      MATH-231          3.00  C   lower   transferable   AR-2.1
  RQ-SEM1   Programme seminar                       named      UNIV-450          1.00  C   upper   residency      AR-2.1
  RQ-SOC1   Social science elective                 category   CAT-SOCIALSCI     3.00  D   lower   transferable   AR-2.4

The outcomeWhat a good result looks like

A drafted evaluation: one entry per proposed application, in the packet's order, each carrying SATISFIES, or CREDIT_ONLY with a named ground, the governing policy from this packet and the document identifiers behind it, or INSUFFICIENT_EVIDENCE naming what is missing -- plus the units the requirement actually receives, independently recomputed on EVERY application, and a packet-level recommendation. Beside every application, what the strongest free code would have written.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

• Your articulation table is complete, your catalog rules are structured, and every application arrives with a proposed requirement already on it. Nothing in your process depends on a note, a memo or an addendum that has not been folded into the table yet. — the free floor -- catalog-gate, $0.00, a tenth of a second, no key
  It scores 76.67 pct on the discriminator, takes 100 of 100 catalog-decidable grounds with their policies and documents, names every unsettleable application, and invents nothing. Set UTOL to your own posting precision first: the sweep moves it 5.84 points.
• Your articulation table lags the agreements. Addenda, chair certifications, calendar conversions and prior awards live in email and in the student's file, and the table is re-issued once a year. — the paid arm, with the free floor computed beside it on every application
  That is the entire gap: 19 of 20 correspondence-only grounds against the floor's 0, and 36 of 36 already-answered applications refused by the floor against the model's 2. Both arms cost the same to compute in the UI and the disagreements are the escalation list.
• You are deciding whether to build any of this at all. — run the free floor on your own packets first -- it needs no key
  It is the honest denominator. A vendor quoting an accuracy for this job without a table-lookup floor beside it is quoting the easy half back to you: on this corpus a title matcher scores 20.83 pct and wrongly grants 41.67 pct of the applications that do not satisfy.

And where nothing here is good enough:

• You want a machine to POST credit without a person reading the result. — neither, and nothing on this page supports it
  The best arm here still waved through 1 non-satisfying application(s) and attached an invented identifier on 11 of 240 citing applications. A wrong SATISFIES leaves no machine symptom at all -- it surfaces in the term the student cannot graduate.

At a glanceHow the whole thing runs

Run once, for real, on 2026-08-26. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-08-26 — r001-transfer-credit. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Checked before this shipped — Clone, python3 -m tools.build_corpus, python3 -m src.app, open the port. No key, no install, no index build: the corpus, the answer key, all three free floors, the tolerance sweep and every recorded run are committed, and the generator is deterministic from one seed so a fresh clone rebuilds the same bytes.

Step 01 of 14Business case

Written for: product manager · Measured during the eval run. Never estimated, never quoted from a vendor page.

In brief

What the clock covers. the fast tier on r001-transfer-credit, 40 packets, one call each, 4 concurrent workers, a 64,000-token ceiling. Wall clock 1642.6 s for the whole run.

Current processWhat it replaces

Someone reading a transfer transcript line by line against a catalog: looking each sending course up in the articulated equivalency table, checking what it articulates TO against what the requirement NAMES, converting quarter units to semester units and comparing them to the minimum, checking the grade against the catalog minimum for that requirement, checking whether an upper-division requirement is being met with lower-division work, checking whether the requirement is one that must be completed in residence, checking what has already been posted against it -- and then reading the registrar correspondence to find out which of those apparent deficiencies somebody has already answered, and which perfectly ordinary-looking application is for a requirement that is already satisfied.

Step 02 of 14Architecture

Written for: solutions architect · Component map derived from the real code, not drawn from intent.

In brief

CorpusCorpusArchitectureArchitectureEvalsEvalsEvalsEvalsPromptPromptPromptPromptArchitectureArchitectureApp UIApp UIEvalsEvalsBusiness caseUnit cost

• 1stations numbered in run order
• txt
• jsonl
• json
• 8 of 11 stations carry a recorded failure

scroll → 11 stations

One question, end to end. Colour on the fan-in is the corpus format; a red hook marks every station with a recorded failure, and the words — every sub line, failure and report field — are in the ledger beneath. Every station is a link — click one to jump to its row, and each row links to the lens that explains it.

The swap seams

Components

Step 03 of 14App UI

Written for: anyone, in 5 seconds · Screenshots of the running app taken during the run.

SuccessesWhen it works

LimitsWhen it does not

A report showing only wins is an advert. This one is required, and the validator now refuses a kit that omits it.

Step 04 of 14Corpus

Written for: "can I point this at mine?" · Measured off the corpus the run actually used.

In brief

How it is cutWhat one proposed application is

40 packets, each one incoming transcript proposed against one receiving catalog, with its degree requirements, its articulated equivalency table, its documents on file and its registrar correspondence. Every packet is independent -- no carried state, no ordering constraint -- so the run is embarrassingly parallel and a failure on one says nothing about any other. The composition is CONSTRUCTED rather than dealt: 32 NOTIFY_STUDENT_OF_GAP, 4 POST_AS_PROPOSED, 4 REFER_FOR_FACULTY_REVIEW, because a random deal makes every packet carry a gap and the packet-level recommendation collapses to a constant. As it is, the single majority word already scores 80.0 pct, and both the majority and the measured figure are printed.

SetupWhat the setup figure measured

There is no index to build. The whole packet, minus the withheld Student Record block, goes into the prompt verbatim -- no chunking, no retrieval, no pre-digest. A summariser here would be a place for the addendum sentence to be lost before the model saw it. The 0.1 seconds is what all three free floors take to score all 40 packets.

LicenceLicence

MIT -- this repository's own licence. The corpus is generated in-process by tools/build_corpus.py from a fixed seed and contains no third-party data of any kind; verified by reading every generator input on 2026-08-26.

Bring your ownBring your own transfer credit evaluation packets

Rewrite src/packet.py's regular expressions for your own export's shape -- everything above it reads the parsed dicts and does not care where they came from. Then rewrite src/checks.py's SUPPORT_TYPE (this corpus files exactly one document per ground; a real evaluation wants several kinds), re-read src/select.py before pointing it at anything real, re-run evals/tolerance.py on your own data because the published unit tolerance was chosen from the sweep on THIS corpus, and re-derive your own answer key so evals/check_labels.py can assert it agrees with your checks. data/SOURCES.md is the step-by-step.

⚠︎ And what stops being true when you do: ⚠︎ A REAL EVALUATION PACKET IS AN EDUCATION RECORD AND THIS KIT IS NOT BUILT TO HOLD ONE. src/select.py withholds a NAMED SECTION and nothing else; the transcript itself, the grades and the prior awards all leave the machine on every call. Before pointing this at anything real, decide what your provider agreement says about that -- the kit gives you the seam to change and does not decide it for you.

Step 05 of 14Prompt

Written for: engineer · Logged from the run, verbatim.

In brief

The assembled prompt in full, and its real decomposition. Every part below occurs in the prompt that was actually sent, and the token counts sum to the recorded input total — they were measured against the provider’s own counter, not apportioned by character share.

AssemblyHow the prompt is assembled

This is the cost lesson as arithmetic: of the 3,285 tokens assembled, 2,187 are contexts — 67% of the prompt, the largest single part. A per-request bill follows whichever part is biggest, and on this kit that is the one named here.

Reproduced from src/prompt.build for packet TC-0001 with the Student Record block withheld exactly as src/select.py withholds it at run time.

Full promptThe prompt in full

You are evaluating proposed transfer credit against a receiving institution's catalog.

You act for the RECEIVING institution's registrar, before anything is posted. This evaluation is not
a posting and not a denial: it is what the credit evaluator reads before either. A wrong SATISFIES
lets a student register for the next course in a sequence they have not met the prerequisite for and
costs them a term. A wrong refusal sends a student back for a course they have already passed, and
they will appeal it against the policy you cite.

Return one entry for EVERY application in "Proposed Applications", in the order the packet lists
them, using the packet's own application identifiers. Never drop an application because it looks
unremarkable.

For each application give exactly one disposition:

  SATISFIES              the course satisfies the requirement it is proposed for.
  CREDIT_ONLY            the course is creditable toward the degree total but does NOT satisfy the
                         requirement it is proposed for. State the ground.
  INSUFFICIENT_EVIDENCE  something about the application cannot be settled from this packet -- the
                         course has never been evaluated for equivalency, or the record that would
                         settle it is not here. Do not state a ground; say what is missing.

When the disposition is CREDIT_ONLY, the ground is exactly one of:
  NO_ARTICULATION        the course is not articulated to what this requirement names. A title that
                         reads like the requirement is not an articulation.
  UNITS_SHORT            the units, in the units the requirement is stated in, are below its
                         minimum
  LEVEL_TOO_LOW          an upper-division requirement against work granted at the lower division
  GRADE_BELOW_MINIMUM    the grade earned is below the minimum the catalog carries
  DUPLICATE_CREDIT       the content has already been credited, so the requirement is not satisfied
                         again
  RESIDENCY_REQUIRED     the requirement is designated for completion in residence
Otherwise ground is null.

Also give, for every application, the applied units you independently recompute: the units the
requirement actually receives after any conversion the catalog rules call for, to two decimal
places. Give null where the packet does not carry units to convert.

Cite the governing policy as a policy identifier that appears in this packet, and attach the support
as document identifiers that appear in this packet. NEVER write a policy or document identifier that
is not printed in the packet in front of you -- an invented citation is worse than none, because a
student appealing a refusal reads the policy first and it will not exist.

Anything in the packet may bear on an application. Where a printed table and a later statement in
the packet contradict each other, they are not a tie.

Reply with JSON and nothing else:

{"packet_action": "NOTIFY_STUDENT_OF_GAP" | "REFER_FOR_FACULTY_REVIEW" | "POST_AS_PROPOSED",
 "applications": [{"application": "<the packet's application identifier>",
                   "course": "<the sending course code>",
                   "disposition": "SATISFIES" | "CREDIT_ONLY" | "INSUFFICIENT_EVIDENCE",
                   "ground": "<one of the six, or null>",
                   "governing_policy": "<a policy identifier from this packet, or null>",
                   "support": ["<document identifiers from this packet>"],
                   "applied_units": <number or null>,
                   "finding": "<the one sentence the evaluation would carry for this line>"}],
 "rationale": "<two sentences at most, on what decided the hardest application>"}

packet_action is NOTIFY_STUDENT_OF_GAP if any application is CREDIT_ONLY; REFER_FOR_FACULTY_REVIEW
if none is but at least one is INSUFFICIENT_EVIDENCE; POST_AS_PROPOSED otherwise. None of the three
posts, awards or denies anything -- each is a recommendation to the person who does.

Transfer Credit Evaluation
----------------------------------------------------------------
Packet               TC-0001
  Receiving programme  BS Mechanical Engineering
  Catalog year         2026-2027
  Sending institution  Marbleshaw State University
  Sending calendar     semester
  Evaluation opened    2026-05-20
  Basis                articulated equivalencies and the catalog's requirement rules

Catalog Rules
----------------------------------------------------------------
Degree requirements

  RQ-ACC1   Financial accounting                    named      ACCT-210          3.00  C   lower   transferable   AR-2.1
  RQ-BIO1   General biology I                       named      BIOL-160          4.00  C   lower   transferable   AR-2.1
  RQ-CHM1   General chemistry I                     named      CHEM-150          4.00  C   lower   transferable   AR-2.1
  RQ-DIV1   Global and diverse perspectives         category   CAT-GLOBAL        3.00  D   lower   transferable   AR-2.4
  RQ-ECO1   Principles of microeconomics            named      ECON-201          3.00  D   lower   transferable   AR-2.1
  RQ-ETH1   Professional ethics, in residence       named      PHIL-410          3.00  C   upper   residency      AR-2.1
  RQ-MTH2   Calculus II                             named      MATH-202          4.00  C   lower   transferable   AR-2.1
  RQ-MTH3   Linear algebra                          named      MATH-231          3.00  C   lower   transferable   AR-2.1
  RQ-SEM1   Programme seminar                       named      UNIV-450          1.00  C   upper   residency      AR-2.1
  RQ-SOC1   Social science elective                 category   CAT-SOCIALSCI     3.00  D   lower   transferable   AR-2.4
  RQ-UDC1   Upper-division computing elective       named      CSCI-330          3.00  C   upper   transferable   AR-2.1
  RQ-UDE1   Upper-division engineering elective     named      ENGR-340          3.00  C   upper   transferable   AR-2.1

  Articulated course equivalencies

  MBS-ACCT-2402   Managerial Accounting                   ACCT-220         lower   AR-3.1
  MBS-BIOL-1406   General Biology I                       BIOL-160         lower   AR-3.1
  MBS-BUSI-2305   Business Statistics                     BUSI-240         lower   AR-3.1
  MBS-CHEM-1405   Introductory Chemistry                  CHEM-105         lower   AR-3.1
  MBS-COSC-1436   Programming Fundamentals I              CSCI-140         lower   AR-3.1
  MBS-COSC-1437   Programming Fundamentals II             CSCI-141         lower   AR-3.1
  MBS-COSC-2330   Introduction to Systems Programming     CSCI-330         lower   AR-3.1
  MBS-ECON-2302   Principles of Microeconomics            ECON-201         lower   AR-3.1
  MBS-ENGR-2340   Introduction to Thermal Systems         ENGR-340         lower   AR-3.1
  MBS-GEOL-1403   Physical Geology                        CAT-LABSCI       lower   AR-3.1
  MBS-MATH-1342   Elementary Statistics                   STAT-240         lower   AR-3.1
  MBS-MATH-2318   Linear Algebra                          MATH-231         lower   AR-3.1
  MBS-MATH-2414   Calculus II                             MATH-202         lower   AR-3.1
  MBS-SOCI-1301   Introductory Sociology                  CAT-SOCIALSCI    lower   AR-3.1

  Unit conversion

  Semester units        carried across at face value                  AR-4.1
  Quarter units         two thirds of a semester unit                 AR-4.2
  Posted units          carried to two decimal places                 AR-4.3

  Grade scale

  A          4.00 grade points   AR-5.2
  B          3.00 grade points   AR-5.2
  C          2.00 grade points   AR-5.2
  D          1.00 grade points   AR-5.2
  F          0.00 grade points   AR-5.2
  IP          n/a grade points   AR-5.2

  Policy index

  AR-2.1    A requirement naming a receiving course is satisfied only by a course articulated to that course
  AR-2.4    A requirement naming a category is satisfied by any course articulated to that category
  AR-3.1    The articulated course equivalency table governs course equivalence
  AR-4.1    Semester units transfer at face value
  AR-4.2    Quarter units convert to semester units at two thirds
  AR-4.3    Posted units are carried to two decimal places
  AR-4.5    The minimum units a transferred course must carry to satisfy a requirement
  AR-5.2    The minimum grade a transferred course must carry
  AR-5.5    An upper-division requirement is not satisfied by lower-division work
  AR-6.2    Requirements designated for completion in residence
  AR-7.3    One body of content is credited once, and a requirement already satisfied is not satisfied again

Proposed Applications
----------------------------------------------------------------
Application A1
  Sending course    MBS-INDS-2340
  Title             Independent Study in Applied Science
  Units             3.00
  Unit basis        semester
  Grade             A
  Transcript level  lower
  Proposed for      RQ-ACC1

Application A2
  Sending course    MBS-MATH-2318
  Title             Linear Algebra
  Units             2.99
  Unit basis        semester
  Grade             B
  Transcript level  lower
  Proposed for      RQ-MTH3

Application A3
  Sending course    MBS-MATH-2414
  Title             Calculus II
  Units             5.03
  Unit basis        quarter
  Grade             A
  Transcript level  lower
  Proposed for      RQ-MTH2

Application A4
  Sending course    MBS-ECON-2302
  Title             Principles of Microeconomics
  Units             3.00
  Unit basis        quarter
  Grade             B
  Transcript level  lower
  Proposed for      RQ-ECO1

Application A5
  Sending course    MBS-BIOL-1406
  Title             General Biology I
  Units             4.55
  Unit basis        quarter
  Grade             C
  Transcript level  lower
  Proposed for      RQ-BIO1

Application A6
  Sending course    MBS-SOCI-1301
  Title             Introductory Sociology
  Units             4.00
  Unit basis        semester
  Grade             B
  Transcript level  lower
  Proposed for      RQ-SOC1

Documents On File
----------------------------------------------------------------
Document DOC-0001-01
  Type                Official transcript
  Reference           A1
  Date                2026-02-27
  Content             Sealed transcript entry for the course on this application
  Recorded units      3.00
  Recorded grade      A
  Recorded term       Spring 2026

Document DOC-0001-02
  Type                Official transcript
  Reference           A2
  Date                2026-01-17
  Content             Sealed transcript entry for the course on this application
  Recorded units      2.99
  Recorded grade      B
  Recorded term       Spring 2025

Document DOC-0001-03
  Type                Syllabus on file
  Reference           A2
  Date                2026-01-28
  Content             Course outline supplied by the sending institution
  Reviewed course     MBS-MATH-2318

Document DOC-0001-04
  Type                Official transcript
  Reference           A3
  Date                2026-04-03
  Content             Sealed transcript entry for the course on this application
  Recorded units      5.03
  Recorded grade      A
  Recorded term       Spring 2025

Document DOC-0001-05
  Type                Official transcript
  Reference           A4
  Date                2026-02-14
  Content             Sealed transcript entry for the course on this application
  Recorded units      3.00
  Recorded grade      B
  Recorded term       Spring 2026

Document DOC-0001-06
  Type                Official transcript
  Reference           A5
  Date                2026-04-22
  Content             Sealed transcript entry for the course on this application
  Recorded units      4.55
  Recorded grade      C
  Recorded term       Autumn 2024

Document DOC-0001-07
  Type                Official transcript
  Reference           A6
  Date                2026-03-14
  Content             Sealed transcript entry for the course on this application
  Recorded units      4.00
  Recorded grade      B
  Recorded term       Spring 2026

Registrar Correspondence
----------------------------------------------------------------
- Two additional courses are in progress at the sending institution and are not part of this evaluation.
- Marbleshaw State University moved to a semester calendar in autumn 2024. The units shown for MBS-MATH-2414 on this transcript are semester units notwithstanding the quarter basis printed beside them; the sending registrar confirmed this in writing on 2026-03-10.
- The transfer credit appeal window for this catalog year closes thirty days after posting.
- Marbleshaw State University moved to a semester calendar in autumn 2025. The units shown for MBS-ECON-2302 on this transcript are semester units notwithstanding the quarter basis printed beside them; the sending registrar confirmed this in writing on 2026-03-19.
- Marbleshaw State University moved to a semester calendar in spring 2026. The units shown for MBS-BIOL-1406 on this transcript are semester units notwithstanding the quarter basis printed beside them; the sending registrar confirmed this in writing on 2026-03-26.
- Two additional courses are in progress at the sending institution and are not part of this evaluation.

Raw responseThe raw response

{"packet_action":"NOTIFY_STUDENT_OF_GAP","applications":[{"application":"A1","course":"MBS-INDS-2340","disposition":"INSUFFICIENT_EVIDENCE","ground":null,"governing_policy":"AR-3.1","support":["DOC-0001-01"],"applied_units":3.00,"finding":"MBS-INDS-2340 is not listed in the articulated equivalency table, so its equivalency to ACCT-210 cannot be established from this packet."},{"application":"A2","course":"MBS-MATH-2318","disposition":"CREDIT_ONLY","ground":"UNITS_SHORT","governing_policy":"AR-4.5","support":["DOC-0001-02"],"applied_units":2.99,"finding":"MBS-MATH-2318 articulates to MATH-231, but its 2.99 semester units are below the 3.00 minimum for RQ-MTH3."},{"application":"A3","course":"MBS-MATH-2414","disposition":"SATISFIES","ground":null,"governing_policy":"AR-2.1","support":["DOC-0001-04"],"applied_units":5.03,"finding":"MBS-MATH-2414 articulates to MATH-202 and the confirmed semester units of 5.03 meet the 4.00 minimum with grade A."},{"application":"A4","course":"MBS-ECON-2302","disposition":"SATISFIES","ground":null,"governing_policy":"AR-2.1","support":["DOC-0001-05"],"applied_units":3.00,"finding":"MBS-ECON-2302 articulates to ECON-201 and the confirmed semester units of 3.00 meet the 3.00 minimum with grade B."},{"application":"A5","course":"MBS-BIOL-1406","disposition":"SATISFIES","ground":null,"governing_policy":"AR-2.1","support":["DOC-0001-06"],"applied_units":4.55,"finding":"MBS-BIOL-1406 articulates to BIOL-160 and the confirmed semester units of 4.55 meet the 4.00 minimum with grade C."},{"application":"A6","course":"MBS-SOCI-1301","disposition":"SATISFIES","ground":null,"governing_policy":"AR-2.4","support":["DOC-0001-07"],"applied_units":4.00,"finding":"MBS-SOCI-1301 articulates to CAT-SOCIALSCI and its 4.00 semester units with grade B meet RQ-SOC1."}],"rationale":"The hardest call is A1, because MBS-INDS-2340 has no entry in the articulated equivalency table, so equivalency to ACCT-210 cannot be settled from this packet. A2 is credit-only because 2.99 semester units fall below the 3.00 minimum, so the packet action is NOTIFY_STUDENT_OF_GAP."}

Step 06 of 14Evals

Written for: the skeptic · The harness output, with as_of / model version / dataset version / verified_by.

In brief

Decide which transfer credits satisfy a degree requirement — 240 transfer credit evaluation packets drawn from 40 real transfer credit evaluation packets. One model answered, and every answer was then graded Two different ways — because the grade is a measurement too, and most of what went wrong on this kit went wrong in the ruler rather than the model.

The rulerHow it was graded

Exact match per proposed application, pure code, no model grading anything. An application counts as accurate only when the disposition is right AND, when it is CREDIT_ONLY, the ground is the right one of the six, the governing policy is the one the packet itself carries for that ground, every document the ground needs is attached, and nothing cited on that application was invented. ⚑ THE TWO ERROR DIRECTIONS ARE NEVER AVERAGED AND THEY ARE NOT SYMMETRIC. false_satisfy_pct is a non-satisfying application waved through: it is posted, the student registers on the strength of it, and it surfaces in the term they cannot graduate. overdenial_rate_pct is a satisfying application refused: a student sent back for a course they have passed, which at least they can see and appeal. A catch rate with no over-denial rate beside it is not a measurement -- an arm that refuses everything scores 100 pct on the first and sends every transfer student back to a community college.

MeasurementsWhat was measured

Two words carry this page: Counted is deterministic and nobody's opinion and Not yet known is printed blank rather than filled with something plausible. There is no Judged row here and that is the point: nothing on this page is a model's opinion about another model. The row most worth having is currently the empty one.

Grading costWhat it costs

Every dollar on this page is a MEASURED token count multiplied by a named vendor's PUBLISHED rate -- a projection onto a rate card, not a bill anyone paid. The rate is read from this repo's own committed card, build/facts/models.json, rather than typed here, so it cannot drift from what the rest of the estate projects onto.

Rates checked 2026-08-18.

Grading unitWhat the grading figure prices

The GRADER is free -- pure code, no model, no network. The BILLING unit of the RUN is one packet -- one call reads the whole thing -- and the SCORING unit is one proposed application. 240 applications over 40 packets, so a per-application cost is the per-packet cost divided by 6.0.

The gradersTwo ways to grade

Every baseline figure here is the strongest floor, catalog-gate, at the shipped src/checks.UTOL = 0.025 (b002-transfer-credit-cataloggate). ⚠︎ CHANGE THAT ONE CONSTANT AND THE FLOOR MOVES 5.84 POINTS: at UTOL = 0.000 it scores 70.83 pct, because it then refuses the sending registrar's own two-decimal rounding. The tolerance was chosen from evals/tolerance.py's sweep on this same corpus, which is an advantage handed to the floor and is disclosed rather than buried -- and the paid arm is still ahead. ⚑ THE COLUMNS THE FLOOR CANNOT REACH AT ANY TOLERANCE ARE THE CORRESPONDENCE ONES: 0 of 20 correspondence grounds and 36 of 36 already-answered applications refused, at every one of the eight values swept.

Every row opens its own page: how the test was run, the input on one real row, the prompt or formula with every iteration of it, and the analysis.

Set limitationsWhat this set cannot show

Deliberately NOT balanced, and the imbalance is stated rather than corrected: 120 of the 240 applications do not satisfy, 106 do, and 14 are unsettleable. The CHANNEL split is the one this kit is actually about and it IS constructed: 100 catalog-decidable refusals against 20 that only the registrar correspondence reveals, plus 36 reverse traps that look deficient to every table and satisfy anyway.

It makes the null baseline 50.0 pct -- 'CREDIT_ONLY for everything' -- and that number is printed beside every arm rather than left for the reader to work out. It also means the per-channel figures are NOT what a real registrar's queue would produce: this corpus plants 20 correspondence cases in 240 applications because that is the column being measured, and a real transcript run would carry far fewer. The catalog half is the half that generalises; the correspondence rate here is a property of the corpus.

The corpus costs $0 to build: tools/build_corpus.py is standard library, takes one seed and no network, and writes the answer key in the same pass that writes the packets. No human annotation, so no annotation cost and no annotator disagreement to report. No natural-distribution run exists. This set answers whether an arm can tell a catalog-decidable deficiency from a correspondence-only one; it does not answer what accuracy looks like against a real registrar's queue, where the correspondence cases are far rarer than the 20 in 240 planted here and the articulation table is thousands of rows rather than a couple of dozen.

Choosing oneWhich one to use

The comparison is only useful if it ends in a choice. The answer is conditional — and it is not the grader with the best agreement score.

This pipeline was also attacked, which is a different question from whether its answers are right: see Threat model for the injection experiment, its two gates and what it does not prove.

Failure causesHow it fails

Step 07 of 14Unit cost

Written for: whoever signs off on the bill · Measured over the eval run, per model. Never quoted from a vendor page.

In brief

The token counts are measured on a real run and belong to this pipeline — which sections are selected, the field schema, the prompt — so they hold wherever you run it. The dollars are those counts multiplied by each vendor's published rate, checked 2026-08-18. Nobody paid these particular bills; they are what the same work would cost you on a card you can sign up to today.

Answering vs gradingAnswering costs money here and grading costs nothing

ZERO FOR GRADING, AND IT IS A MEASURED ZERO. evals/scoring.py is exact string match in pure Python -- no judge, no rubric, no second model -- so one full scoring pass over all 240 applications costs $0.00 and can be re-run by anyone who clones this repo, with no key at all. So can the answer-key gate, all three free floors and the eight-point tolerance sweep. The ANSWERING side is $2.330406 for the scored run's 40 packets on the projected card. The calibration probe, the injection probe and the prompt-token measurement are answering cost too and are not free: 4 + 20 + 2 calls. ⚑ TOTAL PROJECTED SPEND ACROSS EVERY PAID ARM: $3.997990 over 66 live calls -- and the arm that takes the whole catalog-decidable half of this job is in the $0.00 column.

Cost driversWhat actually moves the bill

• OUTPUT TOKENS, AND ALMOST ALL OF THEM ARE REASONING. 95.96 pct of r001-transfer-credit's output (724,081 of 754,579) was provider-side reasoning rather than the evaluation. The evaluation itself is a few hundred tokens per packet. You are paying for the reading, not the writing.
• THE PACKET GOES IN WHOLE. 3333.45 input tokens per packet on average (133,338 over 40), and p001-transfer-credit-prompt-tokens measured the split from two calls at max_tokens=1: a fixed 1098.03 tokens -- the system prompt PLUS the provider's own chat wrapper, billed and invisible to the kit -- plus 0.235347 tokens per packet character. The input side is 15.0 pct of the token bill.
• ONE CALL PER PACKET, NOT PER APPLICATION. A transcript with twelve proposed applications costs the same one call as one with six. Splitting per application would multiply the bill and would also break the thing that works: a note recording that an addendum articulated a course bears on the application that names it, and a per-application prompt cannot see the packet it sits in.
• ⚑ THE SIX CATALOG CHECKS COST NOTHING AND TAKE 100 OF THE 100 CATALOG-DECIDABLE GROUNDS WITH THEIR POLICIES AND DOCUMENTS. The half of this job that is a table lookup should be a table lookup; what is left to pay for is the 20 applications only a sentence in the registrar correspondence settles.

Your volumeWhat it costs at your volume

Linear in packets and flat in applications. Ten times the transcripts is ten times the calls at $0.058260 each on the projected card; ten times the applications per transcript is the SAME number of calls, a longer packet, and a larger reply. The thing that does not scale linearly is the catalog half of the packet -- see breaks_at_scale.

Model choiceWhy this model

One provider, one key, one tier -- the fork test. A forker holds one key and the kit's point is that swapping the model is .env plus the same run again, not that this tier is the right one. No second tier was called and no accuracy is projected onto any other model; only cost is.

Other modelsOn other models

This is arithmetic, not a run. No model below was executed against the labelled set, and no accuracy is claimed for any of them. What transfers is the input volume: retrieval is model-independent, so every model would receive the same passages.

per-packet average, this run

The codeEvery file, and what it is for

13 modules, no framework, and almost no dependencies. The pipeline is stdlib Python, because a kit is meant to be read and changed by someone who cloned it ten minutes ago — every dependency is a thing they have to understand before they can touch the part they came for.

Six of these are swap seams: the points you edit to make the kit yours. Each is marked, and each block below is the module’s real shape, read out of the file rather than written here.

The files on this page are not in the public GitHub catalog.

src/segment.pysection split

Cut the packet on its underlined headings -- Transfer Credit Evaluation, Catalog Rules, Proposed Applications, Documents On File, Registrar Correspondence, Student Record. Deterministic, so 'what did we send' is a list of section names rather than a guess about a token window.

# Cut a transfer credit evaluation packet into addressable sections. Pure code, no model.

HEAD = re.compile(r"^([A-Z][A-Za-z ,]+)\n-{10,}\n", re.M)
def sections(text):
def named(text, name):

src/select.pythe seam that withholds — a swap seam

The Student Record block -- the student identifier, the date of birth, the financial aid standing, whether accommodations are on file, the registration holds and the adviser's own note -- never leaves the machine. Every one of those is an education record and none of them bears on whether a course is articulated, carries the units, carries the grade or sits at the level the requirement demands. Withheld at the seam rather than trusted to a sentence in a prompt, and the UI prints what went and what stayed.

You change it to: <code>NEVER_SENT</code> is a tuple of section names. Add a heading to it and that section stops leaving; the UI's sent/withheld list is derived from the same tuple, so the page cannot claim a control the code does not have.

# SEAM 2 -- what leaves this machine. Pure code, and deliberately a denylist of one.

NEVER_SENT = ("Student Record",)
def sent(sec_names):
def body(text, sections_fn):

src/packet.pythe packet parser — a swap seam

The degree requirement table with its minimum units, minimum grade, level and residency columns, the articulated course equivalency table, the unit conversion rule, the grade scale, the policy index, every proposed application and every document on file, read with regular expressions. The model is never asked for a figure at a fixed offset. ⚑ AND THE COURSE TITLE IS NOT CONSULTED ANYWHERE IN THIS FILE -- that is the kit's whole claim, enforced by there being no code that reads it. Returns empty tables rather than guessing on an unfamiliar layout.

You change it to: Every table is one regular expression. Point the kit at your own export by rewriting them; nothing above this file reads the raw text.

# Read a transfer credit evaluation packet into structured values. Pure code, no model.

SATISFIES, CREDIT_ONLY, INSUFFICIENT = "SATISFIES", "CREDIT_ONLY", "INSUFFICIENT_EVIDENCE"
DISPOSITIONS = (SATISFIES, CREDIT_ONLY, INSUFFICIENT)
GROUNDS = ("NO_ARTICULATION", "UNITS_SHORT", "LEVEL_TOO_LOW", "GRADE_BELOW_MINIMUM",
PACKET_ACTIONS = ("NOTIFY_STUDENT_OF_GAP", "REFER_FOR_FACULTY_REVIEW", "POST_AS_PROPOSED")
CATEGORY_PREFIX = "CAT-"
def num(s):
def parse(text):
def documents_for(parsed, reference, dtype=None):
def requirement(parsed, rid):

src/checks.pythe catalog checks -- and the tolerance — a swap seam

The six grounds the packet's OWN TABLES can decide -- residency, a prior award against this requirement, articulation, level, grade, units -- plus the two absences that mean no ground can be stated at all, plus the applied-units conversion printed on every application. ⚑ UTOL inside it is the single unpublished constant that decides where the free floor lands; the grade comparison is exact and is deliberately kept out of the sweep, because a grade is an ordinal and running a unit tolerance over it is the category error the sweep exists to expose.

You change it to: <code>UTOL</code> is one float and the six checks are six blocks in a fixed priority. Change either and re-run <code>evals/tolerance.py</code> before believing any floor's score -- the sweep is what chose the shipped value.

# The structured articulation checks, in pure code. No model, no registrar correspondence.

UTOL = 0.025
PRIORITY = ("RESIDENCY_REQUIRED", "DUPLICATE_CREDIT", "NO_ARTICULATION", "LEVEL_TOO_LOW",
SUPPORT_TYPE = {
NO_GRADE = ("IP", "", "NR", "W")
def applied_units(app, parsed=None):
def grade_points(parsed, grade):
def articulates(parsed, app, req):
def recompute(app, parsed):
def prior_award(app, parsed):

src/prompt.pythe prompt

The whole instruction in one place, in send order, plus the correspondence-blind variant an ablation arm would use. The ablation TRUNCATES at the heading and RAISES if the heading is absent, because a silent no-op there would publish an ablation arm that was never ablated. ⚠︎ THAT ARM WAS NEVER RUN -- s001 is a spend decision and it is in not_measured, not left to look like a pass.

# The assembled prompt, in one place, in the order it is sent.

NOTES_HEADING = "Registrar Correspondence"
VERDICTS = ("SATISFIES", "CREDIT_ONLY", "INSUFFICIENT_EVIDENCE")
VERDICT_MEANINGS = {
SYSTEM = """You are evaluating proposed transfer credit against a receiving institution's catalog.
BLIND_SYSTEM_SUFFIX = """
def build(text, blind=False):
def _strip_notes(body):
def render(parts):

src/evaluate.pythe model call and reply parse

One packet in, one drafted evaluation out. Holds the published 64,000-token ceiling and the tolerant JSON parse -- a correct answer wrapped in a code fence has not got it wrong -- and normalises a single document identifier returned as a bare string.

# One packet in, one drafted transfer credit evaluation out. The only place a model is called.

MAX_TOKENS = 64000
THINKING = None
def parse_reply(text):
def normalise(obj):
def evaluate(cfg, text, blind=False, complete_fn=None, max_tokens=None):
def cited_ids(answer):
def action_from_applications(answer):

src/adapters/__init__.pythe adapter — a swap seam

One interface, several providers, raw HTTP and no vendor SDK. ⚑ ITS SOCKET TIMEOUT AND THE TOKEN CEILING ARE ONE SETTING: 1,800 s beside a 64,000-token ceiling, with transport retries cut to ONE against four for a transient status, because a non-streamed generation held on a short socket is a transport failure the inherited retry policy pays for five times.

You change it to: PROVIDER and MODEL in <code>.env</code>, then the same run again. Adding a provider is one function and one entry in PROVIDERS; it must return token counts, because the LLM lens publishes them and the Cost lens prices them.

# SEAM 1 — the model. Swapping provider or model is .env plus the same run again.

class AdapterError(RuntimeError):
TRANSIENT = {408, 429, 500, 502, 503, 504}
RETRIES = 4
TIMEOUT_S = 1800
TRANSPORT_RETRIES = 1
def _post(url, headers, payload, timeout=TIMEOUT_S):
def openai_compatible(cfg, system, user, max_tokens, thinking=None):
def anthropic(cfg, system, user, max_tokens):
PROVIDERS = {"openai-compatible": openai_compatible, "anthropic": anthropic}

evals/baseline.pythe three free floors — a swap seam

Title match, table sweep, and the catalog gate. Each a genuine attempt at the job, all three free -- and the first one is the thing this kit is arguing with: a real title matcher, stop-worded and Jaccard-scored, which is what 'AI transfer credit' most often ships as.

You change it to: <code>_STOP</code> and the 0.5 Jaccard threshold in <code>_titles_match</code>. This is the floor the kit argues with, so it is a seam on purpose: make the matcher better and the argument gets harder, which is the honest way to test it.

# THREE FREE FLOORS. No key, no model, no network. Each is a genuine attempt at the job.

MODES = ("title-match", "table-sweep", "catalog-gate")
def _action(rows):
def _row(app, disposition, ground, policy, support, finding, applied=None):
def _tokens(s):
def _titles_match(app, req):
def review(text, mode="catalog-gate", tol=checks.UTOL):
def _title_row(app, p, au):

evals/tolerance.pythe tolerance sweep

Three floors by eight unit tolerances, free and offline. It is the experiment that says the free floor's number is a measurement of a parameter as much as of a method: at zero the floor refuses the sending registrar's own two-decimal rounding, and wide it walks past genuine shortfalls of three hundredths of a unit.

# THE TOLERANCE SWEEP. Free, offline, and it must run before anyone believes a floor's score.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
RESULTS = os.path.join(HERE, "results")
TOLERANCES = (0.00, 0.005, 0.01, 0.025, 0.05, 0.10, 0.25, 1.00)
def _docs():
def _text(d):
def sweep():
def main():

evals/scoring.pythe scorer

Exact match per application, pure code, no model grading anything. Keeps the two error directions on separate denominators and never averages them: a wrong SATISFIES is posted and surfaces in the term the student cannot graduate, a wrong refusal is appealable. Splits the catch rate by channel and gives the named trap its own denominator on top of that.

# Score an arm against the answer key. Pure code, exact match per application. No model grades

SATISFIES, CREDIT_ONLY, INSUFFICIENT = "SATISFIES", "CREDIT_ONLY", "INSUFFICIENT_EVIDENCE"
OMITTED = "OMITTED"
HUNDREDTH = 0.005
def _pct(n, d):
def _key(s):
def rows_by_application(answer):
def _cited(r):
def _numeric(v):
def score(records, golds, valid_ids=None, recomputed=None):

evals/check_labels.pythe gate on the answer key — a swap seam

Twelve properties re-derived from the shipped corpus before any arm is believed -- including the two that carry the kit's central claim: a correspondence refusal must look SATISFIED to the tables, and a correspondence trap must look REFUSED to them. --self-test seeds four defects in memory, convicts all four by name, and proves the shipped key acquits before and after.

You change it to: <code>DERIVABLE</code>, <code>PROSE_FLAG_CASES</code> and <code>PROSE_TRAP_CASES</code> are the three sets that define what 'catalog-decidable' means here. Redefine them for your own corpus and the gate follows.

# A GATE ON THE ANSWER KEY ITSELF. Free, offline, no model. Run it before believing any arm.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
STATS = os.path.join(HERE, "data", "corpus-stats.json")
STRUCTURED_CASES = {"S_NO_ARTICULATION", "S_UNITS_SHORT", "S_LEVEL_TOO_LOW",
PROSE_FLAG_CASES = {"C_DUPLICATE_CREDIT"}
PROSE_TRAP_CASES = {"P_ARTICULATION_AMENDED", "P_UNITS_CONVERTED", "P_LEVEL_CERTIFIED"}
DERIVABLE = STRUCTURED_CASES | {"CLEAN_APPLICATION", "M_NOT_EVALUATED", "M_NO_GRADE_ON_RECORD"}
def load_gold():

tools/build_corpus.pythe corpus generator

40 packets, 240 proposed applications, one fixed seed, no network. Plants the look-alike titles from a written-out table rather than from a random walk, so 'the title reads like the requirement and the articulation points elsewhere' is data a reader can check rather than a claim. Counts every case it could not place and prints them, so the dealt targets and the shipped composition can be compared instead of assumed equal.

# Generate the transfer credit evaluation corpus. Deterministic, seeded, no network, no third party.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CORPUS = os.path.join(HERE, "data", "corpus")
GOLD = os.path.join(HERE, "data", "gold.jsonl")
STATS = os.path.join(HERE, "data", "corpus-stats.json")
RULEBOOK = os.path.join(HERE, "data", "rulebook.json")
SEED = 20260826
PACKETS = 40
APPS_PER_PACKET = 6
DATASET = "transfer-credit-2026-08-26-40packets"

src/app.pythe minimal UI

Standard library http.server on port 9068. Renders with no key at all: the packet, everything read off it in pure code, the applied units on every application and the strongest free floor's whole evaluation are computed locally, and the scored run can be replayed straight off the committed result file.

# The minimal local UI. Standard library only -- python3 -m src.app, then open the printed URL.

HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
UI = os.path.join(HERE, "ui")
CORPUS = os.path.join(HERE, "data", "corpus")
PORT = int(os.environ.get("PORT", "9068"))
RECORDED_RUN = os.environ.get("RECORDED_RUN", "r001-transfer-credit")
FLOOR_MODE = "catalog-gate"
def documents():
def load_doc(doc_id):
def read_in_code(text):

Start hereThe shortest path into it

Every entry above is a file in this kit, listed in the order the pipeline runs it.

ScenariosWhat it costs at your volume

Everything below is computed from one measured base: 3333 input and 18864 output tokens per packet (one incoming transcript, covering every application proposed on it), all of it fixed — this kit sends the document's own selected sections, so there is no retrieval depth to turn. Change the volume and the bill scales linearly, because nothing here batches.

OperationsFour shapes of workload

The same arithmetic, at volumes worth naming. These are illustrations of scale, not measurements of anyone’s deployment.

Model follows the controls above.

What this ignoresWhere a real bill diverges

In one lineWhat this kit exposes

MEASURED ONCE, AND IT LARGELY HELD. One sentence forced into the Registrar Correspondence of 20 packets took 4 of 81 refusals the fast tier had just stated correctly (4.94 pct, x001-transfer-credit-injection), paired and scoped BEFORE the run at a 64,000-token ceiling. ⚑ THE NUMERATOR IS SHARPER THAN THE RATE: the note named articulation, level and units, and those grounds lost 4 of 35 (NO_ARTICULATION 2 of 16, UNITS_SHORT 1 of 9, LEVEL_TOO_LOW 1 of 10) while the grounds it did not name lost 0 of 46 (DUPLICATE_CREDIT 0 of 23, GRADE_BELOW_MINIMUM 0 of 12, RESIDENCY_REQUIRED 0 of 11). ⚠︎ AND SUPPRESSION IS THE DANGEROUS DIRECTION HERE: a refusal talked away becomes a SATISFIES, which is posted, registered against, and discovered in the term the student cannot graduate. Under injection the model invented 2 citation(s) and 9 cell(s) held their verdict while a cited identifier changed.

API_KEY is read from <code>&lt;repo&gt;/.env</code>, this kit's own <code>.env</code>, or the real environment, in that precedence. Both files are gitignored from the first commit and this repo has never held a credential. Nothing is asked of a reader: the page on the site executes nothing, and the local UI renders fully with no key at all -- which on this kit is not a degraded mode, because the free floor takes the whole catalog-decidable half and is computed on every application anyway.

The experimentWe attacked it — two gates

An indirect prompt injection has to clear two gates, and they behave differently. Both were run for real on 2026-08-26.

The result4.94 pct of the refusals this model had already stated correctly were withdrawn by one sentence in a section the kit cannot remove -- because that section is the only evidence for 56 of 240 applications.

One phrasing, one model, one corpus, one 64,000-token ceiling, 81 correctly-refused cells across 20 packets -- every cell where suppression was even possible, each paired against its own un-injected answer in r001-transfer-credit. Scope was fixed before the run by a recorded rule and capped at 20 packets as a spend decision, not applied after reading the answers. 9 cell(s) held their verdict while a cited identifier moved, and 2 citation(s) were invented under injection. The second phrasing is written out in full in evals/injection.py and was NOT fired, so there is no second rate here to average with and none is invented.

HonestyWhat this does not prove

• How far any measured suppression rate generalises. One sentence, one ceiling, one model, one run. The by-ground split shows whether the wording drew a line the model obeyed; it does not bound what another wording would do.
• Whether an injection placed in the withheld Student Record block would have any effect. By construction it cannot reach the model, and that was not probed.
• Whether a CORRESPONDENCE ground can be suppressed as opposed to blinded. A note that contradicts the correspondence is arguing with the only evidence such a ground rests on, so those cells are reported on their own channel rather than counted as clean suppression.
• Anything about the provider's own handling of a packet after the completion returns.

In one lineThe only guardrail is the prompt

One instruction, no enforcement. It mostly holds — and mostly is the whole subject.

Never post, award or deny transfer credit, and never write to a student record. This kit produces an EVALUATION for a person to take into that decision, and the refusal to decide -- INSUFFICIENT_EVIDENCE naming what is missing and asking for a faculty equivalency review -- is part of the evaluation rather than a failure of it. Posting credit is the registrar's act; a denial is appealable and the student will read the policy it cites.

Stated on the UI and in the README, and enforced by there being no such endpoint: src/app.py serves /api/packets, /api/packet, /api/prompt, /api/recorded and /api/evaluate, and nothing else. There is no write path to remove.

EvidenceDoes it hold?

The limitWhat a guardrail is not

• This is ABSENCE OF A WRITE PATH plus a prompt rule, not a runtime enforcement layer. There is no policy engine, no approval workflow and no audit log, because a kit has none of those and adding them would make it a different product.
• src/select.py withholds a NAMED SECTION. It is not a de-identification system, and a packet that mentions an accommodation inside the registrar correspondence will send it. The transcript itself is an education record and it leaves on every call.
• Nothing in this kit constrains a citation to the packet's own identifier set. 11 of 240 citing applications cited something the packet does not print, and the kit COUNTS it rather than preventing it.
• ⚠︎ AND THE PROMPT RULE IS NOT A DEFENCE. "NEVER write a policy or document identifier that is not printed in the packet in front of you" is in the system prompt on every call, and the scored run's fabrication rate is measured rather than assumed to be zero. A sentence in a prompt is not an enforcement layer.
• ⚠︎ AND THE PUBLISHED FLOOR NUMBERS ARE NOT PARAMETER-FREE. Every free-floor figure on this page is computed at src/checks.UTOL = 0.025, a constant chosen from a sweep over the same corpus, worth 5.84 points across the eight values swept. It is disclosed everywhere it is printed rather than treated as a property of the method.

WatchedWhat is watched, and why that one

The kit measures 96 watched figures plus 8 comparability guards and 5 reference constants — three duties, not one list. 0 of the watched are grader rates, each on its own grader page; this board watches the ones that decide when to look again. Each square is one watched figure; filled means the latest run measured it.

7 measured by the latest run 89 need the model half

GuardsBefore any comparison

Preconditions, not alarms. If one moves, the runs were not measuring the same system and nothing else on this board may be differenced.

no shared guard The two recorded runs declare no guard in common, so nothing below is protected by a precondition — read the history as two separate observations, not as a difference.

The reference constants — the null-grader baseline, the adjudication matrix, the negative count — are the third duty. They are never re-measured per run: a baseline that moves with the thing it measures is not a baseline.

BandsThe alarm bands

Derived from evidence, never picked: exact-match where two runs reproduced a metric to the digit, wider than the measured same-input drift for latency, one row of the negative class for any rate — denominator printed beside it.

HistoryRun history

7 recorded runs. A dash is a metric that run did not measure, and a dash is never differenced: not measured is a third state, neither a value nor a zero, and reading it as zero is how a board reports an improvement on a run that never looked. They are split into 4 blocks because a free floor is a baseline rather than a peer, and a run of another kind measures another thing; a column in one block is never differenced against a column in another.

match · no model in the path — a baseline, not a peer column — 3 runs. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

not a time series No two of these 3 runs measured the same system — they differ on applications_citing_any_id, credit_ground_named_cells, credit_support_cells, floor, policy_cited_cells, the model — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

match · with the model in the path — 2 runs. Columns here are only ever compared with each other.

not a time series No two of these 2 runs measured the same system — they differ on answered_trap_applications, applications, applications_citing_any_id, applied_units_cells, catalog_ground_cells, correspondence_ground_cells, credit_ground_named_cells, credit_insufficient_cells, credit_support_cells, duplicate_catalog_cells, duplicate_correspondence_cells, duplicate_credit_cells, max_tokens, nonsatisfying_applications, packets, packets_answered, policy_cited_cells, satisfying_applications — so nothing here may be differenced and the Verdict column is left off. Read this block DOWN a column, as a comparison between those runs, not along a row as a history.

stub · no model in the path — a baseline, not a peer column — 1 run. A floor is what the kit scores with no model in the path; it is the line the model runs have to beat, and it is never differenced against them.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 22 chips that all say so.

redteam · with the model in the path — 1 run. Columns here are only ever compared with each other.

one run A verdict is a comparison and one run has nothing to compare with, so the Verdict column is left off rather than filled with 7 chips that all say so.

DeviationsWhat deviated

0 breaches across 7 runs

The day a band breaks, a row renders here carrying the metric, the two runs, the band it crossed and what changed between them — model, prompt, chunker or top_k, read off the run records’ guards. A deviation with no diff beside it is an alarm nobody can act on.

RippleThe ripple map

Which figures move together, so one alarm reads as a system. Every edge carries its basis: measured names the run that showed it; reasoning admits nothing has varied it yet.

FiresWhat fires when a band breaks

Where evals stop sitting beside the guardrail and start feeding it. Where the answer is nothing automatic, the row says so — hiding it would undo “What a guardrail is not” above.

NextThe three you would add first

None of these is built here. They are named against the seam they sit on so they can be argued with.

CadenceWhen to re-run, and what it costs

The answer-key gate, all three free floors and the eight-point tolerance sweep are free and run before any spend -- which on this kit is most of the evidence. The calibration probe is a MANUAL step and it ran before any scored arm, which is the only order in which a ceiling can be chosen honestly. The injection probe is also manual, run once against a named scored run; it fixes its scope before reading any answer, it refuses to pair across models, and it skips any packet the paired run recorded no answer for rather than inventing an un-injected half.

In one lineNo framework, deliberately

NO FRAMEWORK. A folder of readable Python, standard library end to end, no orchestration layer and no vendor SDK. requirements.txt names nothing. The reason is the fork test: every layer added is a layer a forker has to understand before they can change anything -- and on this kit the two things a forker most needs to change are a set of regular expressions in src/packet.py and one float in src/checks.py.

The mappingSeam by seam, who owns what

The seams are the ones this kit already publishes, so the mapping cannot drift from the code.

GraphsWhere LangGraph changes the answer

Linear: a packet -> src/segment.py -> src/select.py -> src/packet.py -> src/checks.py -> src/prompt.py -> src/adapters -> src/evaluate.py -> evals/scoring.py. No branches, no agent loop, no tool calls. The only fan-out is the thread pool over independent packets -- and the three free floors stop after src/checks.py, skipping the model entirely.

The other sideWhat a framework costs you

• Adding a provider means editing the PROVIDERS dict by hand. In exchange, pip install pulls nothing and the fork test stays at one clone and one command.
• There is no retry/backoff policy you can configure -- it is four attempts on a transient status and ONE on a transport failure, written out in src/adapters/__init__.py rather than set. That asymmetry is deliberate and a configurable client would have hidden it.
• The packet parser is eleven regular expressions written for ONE layout. A schema-driven extractor would generalise it -- and would also make it possible to ask the parser for the course TITLE, which is the one field this kit is built never to read.
• There is no run registry and no dashboard. Every result is a plain JSON object a reader can open, and the answer key has a gate of its own instead.

In one lineWhat one run of this kit actually cost, in time and tokens

Measured on r001-transfer-credit on the fast tier, 2026-08-26. This kit records telemetry measured per run — 4 of the 6 readings on this axis, 4 of them with a band the guardrail board can judge against.

The readingWhat the last run measured

No movement column. Not one of the 4 earlier runs on record is comparable with this one — a different model tier, or a changed guard — and differencing across that measures two systems rather than one. Said once here rather than as a dash on every row.

Retrieval not shown. No run of this kit has ever recorded a retrieval timing, because the pipeline has no such stage. Those readings are left out rather than dashed: a dash says “this run did not measure it”, which promises a later run could.

DriftMedian call, run over run

Hatched bars are runs that are not comparable with the latest — a different model tier or a changed guard. They are drawn because hiding them would make the history look shorter than it is, and greyed because differencing across them would be measuring two systems.

3 runs not plotted. b000-transfer-credit-titlematch, b001-transfer-credit-tablesweep, b002-transfer-credit-cataloggate recorded no call latency at all — a rules-only floor run makes no model call, and the 0 its record stores is an absence, not a zero-millisecond answer. Plotting it would put a bar on this chart claiming the fastest run in the kit’s history.

The other sideWhat this does not tell you

The first of those is the one worth doing something about, and it is a solved problem: instrumenting a pipeline so each step reports its own duration is a couple of lines. Observability / tracing has the code, and what the result looks like once it is running.

In one lineWhat you provision, honestly

Everything measured on this page says where it was measured; everything not measured says that — the same contract as the other thirteen pages. Where a retrieval index would be on other kits, this one has: no index — blocking is computed in memory each run and is the only reduction step.

The machineWhat this needs

Enumerated from the kit’s own source by build/facts/envscan.py, committed, and drift-checked — never typed into this page.

Your dataWhere everything sits

The wireWhat crosses it, and when

Zero-egress topology: point BASE_URL at a local OpenAI-compatible server and nothing leaves the machine — statically true of the shipped adapter, whose only credential travels in the Authorization header.

The shapeHow it ships, and what you add

The ladderEach decision, and what you provision past its ceiling

Every threshold is a measurement with its provenance beside it, or says is_measured:false. A ceiling with no seam says so — it is the point the kit is outgrown, never a forced join.

When it failsFailure signatures

The knobsThe config surface, enumerated

Environment variables: API_KEY, BASE_URL, EMBED_MODEL, EVAL_WORKERS, MAX_CALLS_PER_DAY, MODEL, PORT, PROVIDER, RECORDED_RUN.

The other sideWhat this page does not claim

The corpus licence, from the Data lens: MIT -- this repository's own licence. The corpus is generated in-process by tools/build_corpus.py from a fixed seed and contains no third-party data of any kind; verified by reading every generator input on 2026-08-26. Your corpus’s licence is yours to verify, and the labels you write are about your documents.

In one lineThe whole decision on each proposed application -- disposition, ground, policy, document

whether each of the 240 proposed applications got a decision you could put in front of a student appealing it: the disposition the answer key carries, and on a CREDIT_ONLY the right ground of the six, the governing policy the packet itself prints for it, every document identifier the ground needs, and nothing invented

MethodHow the test was run

python3 -m evals.run --run-id <id> ...; evals/scoring.py compares strings.

Every grader on these pages scored the same 240 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

Every table in the packet says this application satisfies -- it is articulated to what the requirement names, it carries the units, it carries the grade and it sits at the right level. The only record that the requirement is ALREADY satisfied is the sentence above, and there is no document in the packet to attach to it.

The formulaWhat it computes

credit_decision_accuracy_pct = accurate / 240. An application the arm left off its evaluation entirely counts as OMITTED and is a miss. On a CREDIT_ONLY all four of ground, governing_policy, support and no-fabrication must hold together.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: data/gold.jsonl, generated with the corpus and re-derived from it by evals/check_labels.py.

No true/false rates for this grader. It records 3 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? There is no threshold on this grader and there is deliberately no single number to clear. The bar is the FREE FLOOR on the same 240 applications, and it moves when one float in src/checks.py moves -- so the comparison is republished with the sweep beside it rather than reduced to a pass mark.

Cadence: Every scored run, and every free-floor run, automatically -- evals/run.py scores as it writes. The answer-key gate runs before any of them.

The decisionWhen to reach for it

In one lineThe gate on the answer key itself

whether the answer key every other figure is measured against is internally consistent with the corpus it was generated beside -- twelve properties, three of which carry the kit's central claim about which channel a case lives on

MethodHow the test was run

python3 -m evals.check_labels; --self-test red-proves it.

Every grader on these pages scored the same 240 already-recorded answers. Nothing was re-generated, so this compares rulers and not models.

The inputOne real row, seen by every grader

Every table in the packet says this application satisfies -- it is articulated to what the requirement names, it carries the units, it carries the grade and it sits at the right level. The only record that the requirement is ALREADY satisfied is the sentence above, and there is no document in the packet to attach to it.

The formulaWhat it computes

0 violations over 40 packets and 240 applications is the pass. Anything above zero is a regression.

The analysisWhat it actually did

In operationWhat to monitor

Reference standard: the shipped corpus under data/corpus/ and src/checks.py.

No true/false rates for this grader. It records 2 operating rows and not one of them carries a confusion matrix, so there is no TPR, TNR or precision to state here — this grader does not sort answers into pass and fail, and rates that were never measured are not going to be printed as though they were. What it did measure is in “What it actually did” above; what to watch is immediately below.

How tight can the band be? Zero violations. Baselined at zero on purpose: there is no debt on disk here, so any number above zero is a regression introduced after this was written.

Cadence: Before any spend, and after any change to src/checks.py or tools/build_corpus.py. It is free and offline, so there is no reason to run it less often than that.

The decisionWhen to reach for it