Home › Use Cases › Reconcile a grain trade confirmation against the firm's contract record
Use caseUC0413
🧪 Use-case kit · runnable

Reconcile a grain trade confirmation against the firm's contract record

A small, forkable project that does one job end to end. Run once for real, and every figure on these pages captured from that run.

The business caseThe problem this solves

A cargo is agreed on the phone or on a chat line and written up in the firm's own system. Some hours later the counterparty sends their confirmation: their document, their wording, their clause order. Somebody has to read it against the record before anything loads. The document is not a form. The same field can appear in it three times - once in a recital that recites what was already agreed, once in the clause that actually confirms it, and once again in a correction printed under the boilerplate at the bottom - and only one of those is the statement. "As per contract" reads like agreement and confirms nothing. A tonnage that is not the contract tonnage is not a difference if the contract's own tolerance covers it, and the tolerance is not printed on the confirmation. On this corpus the trade desk system's own field match is wrong on 24 of 56 confirmations and wrong about the verdict on 24. Opening the confirmation beside the contract record, reading every clause to the end before answering any field, deciding which of two clauses naming the same field is the statement, working out whether a tonnage that is not the contract tonnage is inside the contract's tolerance, and deciding whether "as per contract" is agreement. It does not decide what to do about a mismatch, and it does not reply to the counterparty.

Audience

A commodity trading firm's trade support or contract administration desk working the confirmation queue before shipment, and the trader behind it. Whoever has to answer "did they confirm what we did?" for each contract, field by field, and has to be able to point at the clause for every answer. Every number on these pages came from one real run of this code, not from a vendor page.

The inputThe actual confirmation reconciliation files

The corpus is 56 confirmation reconciliation files, 0.12 MB (txt 56). It is generated because it has to be. A real contract confirmation is a commercial document belonging to two firms, and the exact shapes this kit measures - a recital that restates the deal, a correction printed under the boilerplate, a cross-reference that reads like agreement and confirms nothing, a tonnage that is not the contract tonnage and is inside its tolerance - are scattered through documents nobody may publish. Generating them means the case mix is KNOWN rather than hoped for: 5 files carry a recital whose figures match the firm's own record while the operative clause does not, 4 carry a later clause that replaces an earlier one, 8 carry a cross-reference, 19 carry a clause that names a field only to rule a value out, and 2 carry a desk note asking for something CCR-2026 refuses. It also means every number on this page is reproducible by anybody with the repo and no key.

The corpus

  • The 56 confirmation reconciliation filesgenerated from a fixed seed, so no real record, person or institution appears in it.
  • Where each came fromdata/SOURCES.md states where every byte came from. Every counterparty is an invented trading name printed with (invented) on the second line of every file; contract ids, clause ids and reference numbers are arithmetic on the file index. evals/check_labels.py sweeps every file for five families of identifier on every run and reports 0.

Swap this folder for your own material and the kit is pointed at your confirmation reconciliation files. That is the whole change — there is no database to migrate.

One confirmation reconciliation file, as the model receives itCONF-0001.txt · 1 of 56
==============================================================================
CONTRACT CONFIRMATION RECONCILIATION FILE                 CONF-0001
Counterparty: CPT-4390 - Halstrow Commodities AG (invented)
Contract: GRN-52009   Commodity: BARLEY   As of: 2026-05-29   Procedure: CCR-2026
==============================================================================

CONTRACT AS THE FIRM'S OWN RECORD HOLDS IT
  quantity                     3000.000   MT
  quantity tolerance               0.00   pct, both directions
  grade                     EU-FEED-BRL   EU feed barley, standard specification
  delivery window        2026-09-01..2026-09-30
  window tolerance                    2   days, both directions
  price basis                   FOB-PNW   free on board, US Pacific Northwest
  payment terms                LC-SIGHT   irrevocable letter of credit payable at sight

THE CONFIRMATION AS RECEIVED FROM THE COUNTERPARTY
  CL-01  Sold to buyers: 3,000 MT of barley, 1 pct more or less at seller's option.
  CL-02  Quality: EU feed barley of standard specification.
  CL-03  Delivery period: 2026-09-01 to 2026-10-31, both dates inclusive.
  CL-04  Price basis: free on board PNW.
  CL-05  Payment terms: by irrevocable letter of credit payable at sight.
  CL-06  Weights and quality final at the load port as per certificates issued by an independent surveyor mutually acceptable.
  CL-07  All other terms and conditions as per our standard trading terms.

FIELD MATCH AS THE TRADE DESK SYSTEM REPORTS IT
  FIELD             MATCH       VALUE READ               CLAUSE
  quantity          agrees      3000.000                 CL-01
  grade             agrees      EU-FEED-BRL              CL-02
  delivery window   differs     2026-09-01..2026-10-31   CL-03

Abridged — the file continues.

The outcomeWhat a good result looks like

One confirmation in, one row out: for each of five fields, the value this document states and the clause it states it in (quoted verbatim), and then - derived in code, never taken off the reply - whether that value agrees with the firm's record inside the contract's own tolerances, differs from it, or was never stated at all, how many of the four material fields are anything other than agrees, and one verdict.

And when it cannot

And what it does when it cannot. On the scored run 56 of 56 replies parsed and nothing stopped at the ceiling, so there is no unparsed column to report. A reply that does not parse is counted WRONG and stays in the denominator; it is never re-fired. A cited clause id that is not printed on that confirmation is dropped by the station and counted (0 on this run). A stated value the firm's vocabulary cannot carry - a tonnage with four decimals, a grade written out in prose instead of as a code - is dropped to NOT STATED rather than guessed at (0 on this run).

Where it fitsWhat did work

Every line below is a measured result from this kit's own runs, with the figure that supports it. The headline above is not softened by any of them.

  • A confirmation queue before shipment — the paid call
    it answers all five fields and names the clause each value came off, on 56 of 56 contracts, against the trade desk system's own 34. The 24 contracts the system gets wrong are the ones where a recital, a correction or a cross-reference is doing the damage.
  • Confirmations that are a fixed form with one clause per field — free code
    the reading is only worth buying because the same field appears twice and only one of them is the statement. With one clause per field a keyword parser and the paid arm answer the same thing, and one of them is free.
  • Deciding what to do about a mismatch — the trader who did the deal, with this row in front of them
    it reports a field-by-field comparison and refuses every disposition. CR-2 says the confirmation does not match; a person decides whether to call, amend or walk.
  • Deciding whose terms prevail — somebody who has done the research, or your lawyers
    it states nobody's rights or obligations, by design and in code. The third refusal exists precisely because that is the first question anybody asks of a confirmation that does not match, and it is a legal question.
  • Sending the acceptance or the rejection — your own contract system
    nothing is signed, countersigned, returned or replied to anywhere in the pipeline, and the answer schema offers no field that could express it.

At a glanceHow the whole thing runs

100%all five correct pct
2,020 msp50, end to end
$0.00per 1,000 confirmation files · google/gemini-3-flash

Run once, for real, on 2026-09-11. Every figure on these pages was captured from that run — nothing is written from intent.

14 steps, grouped by the question that sends you to them rather than by build order. Each tile carries the one figure that step is about, and opens the page behind it.

Should you use this?What you bring, where it stops, and when not to use it

Before you commit an afternoon to this, these are the answers that decide it. Each one is rendered from the record it lives in — and links the page that holds it in full.

What do I have to bring?Replace data/corpus/*.txt with your own confirmations in the same shape and data/contracts.json with your own contract records, then re-label. The percentages do NOT travel with the corpus. Corpus lens →
When is this the wrong choice?Avoid: Reading the headline as a per-field result. The reply's own comparison is right on 52 of 56; the pure-code station carries it to 56, and the station is free. That is the case against the best-fitting scenario (“A confirmation queue before shipment”). 5 scenarios scored in all, each with its own. Eval lens →
Where does it stop working?A confirmation that is not plain text. src/doc.py reads the clause shape these files print; a PDF, a scanned fax or an email thread needs a reader in front of it and nothing else changes. 6 recorded failure modes, each from a run rather than a guess. Corpus lens →
What was never verified?THE CEILING WAS HIT. 56 of 56 on the rechecked column means this corpus has no headroom and cannot rank this tier against a better or a cheaper one. 9 items this kit says it could not check. Eval lens →
Can I run this on a model I control?Yes — any OpenAI-compatible endpoint, including one on your own hardware. The shipped adapter takes its host from BASE_URL and its model from MODEL, so nothing in src/ changes. The published figures come from 1 model on the fast tier, one provider, one key. Prompt lens →
And if it fits — what do I stand up?6 artifacts with a stated home and a stated egress, and 3 decisions each with what you provision past its ceiling — plus what was not measured. That is the next page, not this one. step 14 — Run it in your environment →

Not asked of this kit — 2 questions: clone (a fresh clone of this kit runs with nothing fetched); judge (nothing here is graded by a model).

Last verified 2026-09-11 — r001-confirm-reconcile. Every figure on these pages was captured from that run.

Run itHow this reaches your data

Every result on this page was produced by pure code over checked-in files, with no API key — which is why you can read the numbers before anyone spends anything.

Run this on your own data

  • The pipeline, its eval harness and the runs behind every numberdeployed inside your environment, on your own model endpoints, against your own documents.
  • The corpus above is the shape, not the limitit is a folder swap, and there is no database to migrate.

Talk to us →

Checked before this shipped — A clean checkout with no key configured renders the whole board, replays all four free floors and the committed scored run, and scores every one of them offline. python3 -m evals.run --run-id t000-confirm-reconcile-stub --stub exercises the prompt assembly, the JSON parse, the station, the citation locator, the refusal reader, the scorer and the result file end to end without reaching a provider. python3 tools/build_corpus.py --check rebuilds every file in memory and compares it byte for byte, and it was run under two PYTHONHASHSEED values. python3 -m evals.check_labels re-derives the whole key with a second implementation. None of the four needs a credential or an install.

A living map of modern AI — kept current every morning