The business caseThe problem this solves
A corporate legal department receives invoices from outside counsel. The rates that were agreed live in two documents — the client's outside-counsel guidelines, which apply to every firm, and each firm's own engagement letter with its Schedule A, its rate freeze and its cap on increases. Checking a bill means reading a narrative to see what the time was actually spent on, then finding which provision fixes that timekeeper's rate on that date. Today that is done by eye, on a sample, and the cases that pay for the exercise are the ones a sample misses. the line-by-line rate check a reviewer does by eye against the engagement letter, on a sample of the invoices rather than all of them
Audience
Legal operations and the reviewer who signs the invoice. The decision is whether to buy a model call per invoice, and on this corpus the answer is a qualified no: buy it for the narrative reading, not for the arithmetic. Every number on these pages came from one real run of this code, not from a vendor page.
The inputThe actual invoices
The corpus is 58 invoices, 0.16 MB (txt 58). Every firm, matter, timekeeper, rate, engagement letter and invoice is INVENTED, and OCG-2026-A is not any real client's guidelines. That is deliberate: a real invoice set is privileged, and a rate schedule is commercially confidential, so a corpus anybody can publish has to be one we wrote. 48 timekeepers across 7 firms, 14 of them deliberately absent from any Schedule A, and 152 distinct narratives.
The corpus
- The 58 invoicesgenerated from a fixed seed, so no real record, person or institution appears in it.
- Where each came fromrecorded in the kit's own SOURCES.md, beside the corpus it describes.
Swap this folder for your own material and the kit is pointed at your invoices. That is the whole change — there is no database to migrate.
HARROWGATE VANCE LLP
Attorneys at Law
1200 Ridgeline Avenue, Suite 2400 | Wilmington, DE 19801
Federal tax identification on file with the client
I N V O I C E
Invoice number INV-0001
Invoice date 2026-02-05
Billing period 2026-01-01 through 2026-01-31
Engagement letter EL-2024-HV-11 (effective 2025-01-01)
Guidelines OCG-2026-A
BILL TO
Halvermere Industries, Inc.
Office of the General Counsel - Legal Operations
Attn: Outside Counsel Billing
MATTER
Client / matter number HAL-2219
Matter name Trade secrets action, N.D. Ill.
Responsible partner R. Okelloh-Barre
PROFESSIONAL SERVICES
# DATE TIMEKEEPER GRADE HOURS RATE AMOUNT
----------------------------------------------------------------------------------------
1 2026-01-19 T. Merribeck Paralegal 3.5 215.00 752.50
Second-level review of custodial documents flagged for privilege.
2 2026-01-21 R. Okelloh-Barre Partner 2.5 845.00 2,112.50
Analysed the tribunal's procedural order no. 4 and diarised the deadlines.
3 2026-01-22 T. Merribeck Paralegal 2.5 215.00 537.50
Worked on the privilege log throughout the travel leg to Miami.
----------------------------------------------------------------------------------------
TOTAL HOURS 8.5
TOTAL PROFESSIONAL FEES $3,402.50
TIMEKEEPER SUMMARY
TIMEKEEPER GRADE HOURS RATE AMOUNTAbridged — the file continues.
The outcomeWhat a good result looks like
Every line carries a verdict, the clause that decides it, and the signed amount at issue — so a query letter can be written from the report without re-reading the invoice.
And when it cannot
It over-flags. 13 clean lines were reported as reducible — 5 as non-working travel, 6 as unapportionable block entries, 2 as administrative — and each one is a query a reviewer has to withdraw.
Where it fitsWhat did work
Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.
- You need to catch non-billable and half-billable time hidden in narratives — the paid call
work_class 277 of 291, and perfect recall on both reducible classes — 26 of 26 non-working travel and 15 of 15 administrative. No column carries this and the keyword floor is worse than a constant. - You need to catch rate increases that breach a freeze or a cap — the freeze floor, free
rate_basis is a date against a table. The free floor takes 252 of 291 line verdicts for $0.00; the paid call takes 240, p = 0.218518. - You want the whole job done and will accept one arm — the freeze floor for the ruling, the paid call for the narrative
they win different cells and the split is significant in both directions: the model wins work_class at p = 0.000388, the floor is ahead on the composite verdict at p = 0.218518 (not significant). - You have no key and no budget — the freeze floor
0 calls, $0.00, no network, and it is the strongest arm on this corpus's bottom line
At a glanceHow the whole thing runs
Run once, for real, on 2026-09-10. Every figure on these pages was captured from that run — nothing is written from intent.
14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.
Should you use this?What you bring, where it stops, and when not to use it
Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.
| What do I have to bring? | Replace data/engagements.json with your firms, schedules, freeze windows and grade caps, and drop your invoices into data/invoices/ with a matching data/invoices.json. The measured accuracy does NOT travel with your corpus. Corpus lens → |
| When is this the wrong choice? | Avoid: Do not also trust it to pick the governing provision. That is the case against the best-fitting scenario (“You need to catch non-billable and half-billable time hidden in narratives”). 4 scenarios scored in all, each with its own. Eval lens → |
| Where does it stop working? | an invoice that does not print a line number in a '#' column — the reply is keyed on that number and there is nothing else on the page to join a reading to a row. 5 recorded failure modes, each from a run rather than a guess. Corpus lens → |
| What was never verified? | whether the model would do better on a REAL rulebook. OCG-2026-A is invented, and its stipulations — above all that a freeze-window increase stays void after the window lifts — are choices a real engagement letter could make differently. 5 items this kit says it could not check. Eval lens → |
| Can I run this on a model I control? | The shipped adapter is the runtime provider is not named on this page; the Prompt lens states what swapping it costs. The published figures come from 1 model on the fast tier (THE PUBLISHED RUN). Prompt lens → |
| And if it fits — what do I stand up? | 5 artifacts with a stated home and a stated egress, and 4 decisions each with what you provision past its ceiling — plus what was not measured. That is the next page, not this one. step 14 — Run it in your environment → |
Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).
Last verified 2026-09-10 — r001-billed-rate. Every figure on these pages was captured from that run.
Run itHow this reaches your data
Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.
Run this on your own data
- The pipeline, its eval harness and the runs behind every numberdeployed inside your environment, on your own model endpoints, against your own documents.
- The corpus above is the shape, not the limitit is a folder swap, and there is no database to migrate.
Checked before this shipped — A clean checkout with no key configured renders the whole board — the corpus, the answer key, the recorded run and all three free floors — and scores every free floor offline. evals/check_labels.py and evals/baseline.py both run with no network and no credential. Only evals/run.py without --stub or --floor, the injection probe, and the board's one Check button ever reach a provider.
