Home › Use Cases › Assemble a professional-judgment case file and name what is still missing
Use caseUC0325
🧪 Use-case kit · runnable

Assemble a professional-judgment case file and name what is still missing

A small, forkable project that does one job end to end. Run once for real, and every figure on these pages captured from that run.

The business caseThe problem this solves

A family asks a college financial aid office to look again at their income because something changed - a job lost, hours cut, a one-time payment that will not repeat, medical costs paid out of pocket, support that ended, private school fees for another child. Each of those needs a different set of five documents, and the office cannot act until they are all on file. The work is not deciding anything: it is opening a folder, reading what is in it against what that circumstance requires, and telling the family exactly what is still needed and why. A wrong answer sends a family to find a document they already sent, or sends a file to a reviewer marked complete when the evidence is not in it. Opening a case folder and checking it, document by document, against the requirement list for the circumstance claimed - then writing the sentence that tells the family what is still needed. It does not replace the professional judgment, and there is no field in its answer that could.

Audience

A financial aid reviewer working a queue of professional-judgment requests, and the designated aid administrator who reads what they produce. The decision this row is for is NOT theirs to make and not this kit's either. Every number on these pages came from one real run of this code, not from a vendor page.

The inputThe actual professional-judgment case folders

The corpus is 60 professional-judgment case folders, 0.14 MB (txt 60). It is generated because it HAS to be, and this is the clearest case of that on the estate. A real professional-judgment case folder is a family's worst year written down: a redundancy letter, a hospital account, a support order that stopped, and a parent's own statement of what happened. It is the most sensitive artefact a financial aid office holds and no such corpus may be published by anybody for any purpose. What is lost is real - the generated folders are fixed-layout, which makes the free floor far stronger than it would be on scans - and data/SOURCES.md attacks the corpus in six named ways rather than claiming otherwise.

The corpus

  • The 60 professional-judgment case foldersgenerated from a fixed seed, so no real record, person or institution appears in it.
  • Where each came fromNowhere. Every applicant, sibling, employer, school, clinic, plan, county, fund, revenue agency, amount, date, order number and case reference is invented in-process from seed 20260908, and so are the case register and the whole answer key. Nothing is fetched and nothing is scraped. See data/SOURCES.md, which also says what the generator costs the measurement.

Swap this folder for your own material and the kit is pointed at your professional-judgment case folders. That is the whole change — there is no database to migrate.

One professional-judgment case folder, as the model receives itPJC-0001.txt · 1 of 60
PROFESSIONAL-JUDGMENT CASE FOLDER   PJC-0001
==============================================================================

CASE FACTS
  Case reference        PJ-2026-0101
  Applicant             Marcus Okonjo
  Award year            2025-2026
  Base year             2024
  Request received      2025-02-04
  Category claimed      reduction in hours or rate
  Reviewing office      Office of Student Financial Aid

DOCUMENT INDEX -- what the office logged in, and the type it was logged under
  D1   PJ request form                      logged 2025-02-04
  D2   Applicant statement                  logged 2025-02-04
  D3   Tax return transcript                logged 2025-02-04
  D4   Employer hours notice                logged 2025-02-04
  D5   Year-to-date earnings statement      logged 2025-02-04

DOCUMENT D1 -- PJ request form
  Issued by      the applicant
  Covers         award year 2025-2026
  Signature      signed by the applicant on 2025-02-02
  Special circumstance claimed                          reduction in hours or rate
  Household size stated on the form                     4

DOCUMENT D2 -- Applicant statement
  Issued by      the applicant
  Covers         award year 2025-2026
  Signature      signed by the applicant on 2025-02-02
  My hours were cut in the month shown and our household income has not recovered since.

DOCUMENT D3 -- Tax return transcript
  Issued by      Northern Revenue Bureau, transcript unit
  Covers         tax year 2024
  Signature      issued under the seal of the agency on 2025-02-03
  Adjusted gross income reported for the year           $40,517.00
  Wages, salaries and tips                              $38,499.00

DOCUMENT D4 -- Employer hours notice
  Issued by      Brightwater Foods Inc, human resources

Abridged — the file continues.

The outcomeWhat a good result looks like

One folder in, five graded answers out: the claimed circumstance, the set of requirement keys this folder does not satisfy, why the first one fails, where the file goes, and one line of the folder copied verbatim that establishes the defect.

And when it cannot

And what it does when it cannot. On the scored run 24 of 60 folders had something wrong. 1 file was sent on as complete that is not, after the recheck - the failure that reaches an aid administrator - against 5 on the free rules floor. 4 of the 9 mislabelled documents were read as satisfying their requirement. 2 complete files were called incomplete, which sends a family to find a letter that is already on file.

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

  • Your folders are structured, your document index is reliable, and your requirements are presence-and-date checks — the free rules floor alone - evals/baseline.py, $0.00, no key, no network
    It gets 47 of 60 folders completely right on this corpus and beats the paid call by 11. Four of the six defect classes are three labelled fields and a date comparison.
  • A document being what it says it is cannot be assumed — the paid call, and read the mislabel column and nothing else
    The floor gets 0 of 9 and the call gets 5. That is the entire commercial case here, and it is also the failure that reaches an aid administrator: 1 file sent on as complete against the floor's 5.
  • You want the file state and nothing else — either arm, plus src/recheck.py
    The station is worth 7 folders to both arms for $0.00 - every administratively blocked folder comes back right whatever the reading was - and the two arms then differ by 4.

And where nothing here is good enough:

  • You need the quoted line to be exactly the one your reviewers would cite — neither, yet
    The call was credited on 17 of 34 and 9 of its misses are ANOTHER LINE OF THE SAME OFFENDING DOCUMENT. The key names one line; a reviewer might name a different one. That is a limitation of this key at least as much as of the arm.

At a glanceHow the whole thing runs

36%rechecked case all correct
1,593 msp50, end to end
$3.23per 1,000 professional-judgment case folders · Gemini 3 Flash

Run once, for real, on 2026-09-08. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

What do I have to bring?Replace data/corpus/*.txt with your own folders in the same four-panel shape, data/cases.json with your own register rows (one per folder id, carrying register from the four states), data/policy.json and data/policy.md with your own procedure, and data/gold.jsonl with your own key. WHAT STOPS BEING TRUE THE MOMENT YOU DO. Corpus lens →
When is this the wrong choice?Avoid: Paying for a reading you do not need. That is the case against the best-fitting scenario (“Your folders are structured, your document index is reliable, and your requirements are presence-and-date checks”). 4 scenarios scored in all, each with its own. Eval lens →
Where does it stop working?A scanned or photographed folder. Every arm here - the model's reading, the floor's structural tests and the segmenter itself - reads labelled text lines. 8 recorded failure modes, each from a run rather than a guess. Corpus lens →
What was never verified?Whether the labelled line is the line an aid administrator would have quoted. The key names ONE line per defect and 9 of the paid arm's 17 citation misses are ANOTHER LINE OF THE SAME OFFENDING DOCUMENT - on statement_not_evidence the key names the issuer line and the arm quoted the applicant's own sentence under it, which is arguably the better answer and scores zero here. 9 items this kit says it could not check. Eval lens →
Can I run this on a model I control?Yes — any OpenAI-compatible endpoint, including one on your own hardware. The shipped adapter takes its host from BASE_URL and its model from MODEL, so nothing in src/ changes. The published figures come from 1 model on the fast tier, one provider, one key. Prompt lens →
And if it fits — what do I stand up?8 artifacts with a stated home and a stated egress, and 3 decisions each with what you provision past its ceiling — plus what was not measured. That is the next page, not this one. step 14 — Run it in your environment →

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-08 — r001-pj-case-file. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Run this on your own data

  • The pipeline, its eval harness and the runs behind every numberdeployed inside your environment, on your own model endpoints, against your own documents.
  • The corpus above is the shape, not the limitit is a folder swap, and there is no database to migrate.

Talk to us →

Checked before this shipped — A clean checkout with no key configured rebuilds the corpus byte-identically, re-derives the whole answer key twice, scores every one of the 300 graded cells offline, runs both free floors and the stub end to end, re-runs the cap's phrase lists over the procedure and all 60 folders, and renders the whole board on 127.0.0.1:9325 including a replay of the committed paid run. The only thing a key adds is a new call.

A living map of modern AI — kept current every morning