# Existing-corpus regression, 2026-09-09

The [owner's amended gate](persistence-regression-decision-260909.md) passed with
all 40 existing documents represented: 35 fiscal documents and five statements.
These were used during development. Results include retries and establish neither
independent acceptance nor first-attempt accuracy.

## Extraction result and provenance

| Kind | Matched / scored | Rate |
|---|---:|---:|
| Headers | 432 / 435 | 99.31% |
| Lines | 340 / 347 | 97.98% |
| Movements | 1363 / 1405 | 97.01% |

The complete combined score passes the existing 95% per-kind and applicable 85%
per-field gates. Existing key corrections affect 24/2,187 cells (1.10%). No new
key or prompt changes were made to obtain this result.

Evidence root: `local/artifacts/persistence-regression-260909`. The authoritative
score is `combined-reused-corpus-260909.json` and its Markdown companion.
`measurement-provenance.json` binds every selected answer to its SHA256, PDF/key
hashes, measured run and observed run records. Raw business-document evidence
remains local and is not committed to Git.

Read-only source checkout: `D:/fileStorage/tmp/persistence-regression-260909/source`,
at `a558523f3f0ad97e6c3f60707d635b3d257d3392`. All 40 PDF hashes match the keys.
The original `invoice-skill-build` project was not changed. Extraction used baseline
`bb783e3` bench binaries from the separate `app` checkout throughout, including
while later source fixes were developed. The unchanged tuple is:

- CLI `C:/Apps/Sibyla/tools/claude/claude.exe`, version `2.1.259`.
- Model `claude-opus-5`, effort `high`.
- Package SHA256 `083cbd796ca37b4ea73d70b36a874e33cee50762cd754553d8d6bb3a8e4c53c5`.
- Prompt SHA256 `d22c4f128e609c432416caa40f9f9b893fad0c43f60471bb54ccb171a873fd54`.

`reused-corpus-first` preserves the expired-login failure. The authenticated main
run is `reused-corpus-20260909-162550`. Its `-paused-snapshot` sibling preserves
the evidence when the owner paused after two answers. No final report existed at
the interruption, so resumed metadata does not reconstruct the original two
attempts' timing/usage. Consult the snapshot and raw traces; zero resumed attempts
does not mean no model call occurred.

Three disjoint batches (`-part-0`, `-part-1`, `-part-2`) used the same tuple while
the main run continued. The collector imported 22 selected batch answers, with
hashes and origins in `parallel-imports.jsonl`. Main/batch overlaps and failures
remain visible. The combined rescorer's trace summary covers only its two named
runs; the provenance sidecar also links batch evidence. No single-attempt success
claim is made for imported or overlapping cases.

The USD statement failed once in the main run because two grouped posting dates
were null. The separate `reused-corpus-usd-retry-20260909` run failed attempts 1-4
and succeeded on attempt 5: six attempts across two sittings. Its valid answer
supplies the fortieth case; the other 39 selected answers reside in the main run's
`actual` directory. The main historical report incorrectly said PASSED despite
incomplete coverage. It is preserved and superseded by the complete combined score.

## Defects found and fixed

1. Long fiscal reasons exceeded `docint.hold_reason`'s 200-character limit, so
   PostgreSQL rejected the hold. Both persistence paths now bound that summary
   with `left(...,200)` and retain full review/audit detail. Commit `a912a4c`;
   no schema or source-fact truncation.
2. The live bench could pass accuracy gates with selected documents unanswered.
   Commit `4966b5d` requires complete selected-document coverage, including refused
   inputs. Historical baseline reports remain unchanged.
3. Real upload rejected `I26030025` because its filename ends in uppercase `.PDF`.
   Its allowlist was case-insensitive but signature dispatch was not. Extension
   case is now normalized before content gates. Tests verify MIME rejection and
   active-content quarantine still apply to mixed-case extensions.

## Upload and persistence result

The opt-in `CorpusPersistenceTests` passes all 40 cases against
`gott_sibyla_preview`, with an isolated owner/company per case and the application
role for persistence. It uploads each original PDF through `IngestionService`,
checks hash and size, runs actual content/scanner gates, uploads a duplicate and
checks the duplicate hold. Measured JSON is replayed at the worker hand-off.
Remote transfer and queue claiming are excluded; this is not one uninterrupted
worker/cloud end-to-end run.

| Final persistence outcome | Cases |
|---|---:|
| Fiscal entered and replayed with synthetic catalogue mappings | 23 |
| Statements entered and replayed, preserving 187 movements | 4 |
| Held: missing printed tax identity | 5 |
| Held: missing document number | 2 |
| Held: incomplete period | 2 |
| Held: company identity mismatch | 1 |
| Held: unavailable original credit-note period | 1 |
| Held: unknown unit-price basis | 1 |
| Held: sectioned card order | 1 |

Fiscal cases first require a hold without financial writes when counterparty and
catalogue are absent. Synthetic mappings then test persistence, not real item
classification. Missing printed identities are not filled from keys. Source
rounding differences are preserved rather than forcing line sums to header totals.
Statement accounts come from existing keys. The Revolut fixture uses a hand-checked
IBAN note to populate the typed IBAN field where the legacy key used accountNumber.
The sectioned card has no supported chronological balance chain and stays held.

A separate controlled USD review scenario passed using existing source notes to
fill only the two null posting dates through `SubmitReviewAsync`, preserving the
original evidence. It is separate from the final valid retry answer.

Every attempt writes a distinct `persistence-boundary-*.json`; failures are retained.
Early harness expectations were corrected for policy holds, source rounding and
the legacy IBAN fixture. `harness-development-old-answers` was only debugging and
contributes no answer to the final extraction score. The final upload report is
`reused-corpus-20260909-162550/persistence-boundary-e556c8212a9c40438d7a0f2fbd35cc5d.json`;
it includes stored fiscal facts and statement movements before fixture cleanup.

To reproduce, set `SIBYLA_CORPUS_RUN` to the main run, `SIBYLA_CORPUS_EXTRA_RUN` to
the USD retry and `SIBYLA_CORPUS_SOURCE` to the pinned source checkout; run
`dotnet test tests/Sibyla.Tests.Argus -c Release --filter FullyQualifiedName~MeasuredCorpusPreserves`.
The corpus test is opt-in. `SIBYLA_CORPUS_ONLY` explicitly marks subsets; absent
answers cannot silently shrink the denominator.

## Validation and release status

Evidence under `tests/`: Argus 425 passed with the opt-in corpus test skipped;
the separate final corpus test covers all 40 cases and passes. Platform 626 passed;
Browser 166 passed with Playwright enabled; API 56 passed; tenant isolation 60
passed; bench coverage tests 5 passed. Earlier
matching/review groups passed 114 tests and a focused matching/C6 selection passed
six. The credential administration page has render/access smoke coverage, which
does not establish an interactive save/test/activate credential lifecycle.

Production activation requires a clean published candidate and governed deployment
checks. No production deployment is claimed here. C6 remains in force: NP Group's
52 historical sync runs are protected; this regression does not authorize cutover.