docs: record delivery note ingestion verification

.superpowers/sdd/2026-09-18-hybrid-delivery-note-ingestion/progress.md

@@ -0,0 +1,161 @@
+# SDD ledger — plan: docs/superpowers/plans/2026-09-18-hybrid-delivery-note-ingestion.md
+
+Workspace: /Volumes/EXT/DEVELOPMENT/MEGAIFRASTRUCTURE/.worktrees/hybrid-dn-ingestion
+Branch: codex/hybrid-dn-ingestion
+Merge base: 7ab5570
+Baseline: npm run test:run — 85 files, 1023 tests passed
+
+## Pre-flight scan
+
+| Task(s) | Produce / consume or self-check | Finding |
+| --- | --- | --- |
+| 1 | Status helpers, schema and PB types consumed by 3–12 | Internally consistent; implementation must remain additive and dry-run only during development. |
+| 2 | Canonical PNG pages consumed by 4, 7 and 9 | Internally consistent; original bytes remain in sources, so canonicalisation cannot destroy evidence. |
+| 3 | Multi-source upload consumes Task 1 schema | Internally consistent; repeated capture-session lookup may query source rows because capture_session_id is source metadata. |
+| 4 | Rich PageRead consumed by 5–7 | Internally consistent; new fields are additive and current commercial fields remain required/defaulted. |
+| 5 | Assembly consumes Task 4 and supplies Task 7/9 | Internally consistent; legacy segmentPages remains an adapter so current tests and runner can coexist. |
+| 6 | Resolver consumes Task 1 aliases and Task 4 reads; both legacy run.ts and staged runner need it | Ruling: put matching policy in the pure resolver and keep database loading thin; the feature flag preserves legacy production behavior. If wrong, duplicated DB adapters may need consolidation. |
+| 7 | Lock/staged runner consumes Tasks 1–6 and switches tick behind a flag | Internally consistent; no production switch without DN_STAGED_INGESTION_ENABLED=true. |
+| 8 | Promotion consumes candidate identity from Tasks 1/7 | Ruling: an unposted partial promotion may reconcile its own candidate rows, but a posted candidate is immutable without explicit correction. If wrong, a retry could either leave partial rows or overwrite audited data. |
+| 9 | Split UI consumes Task 5 assembly and Task 1 status | Internally consistent; persisted-page route must retain legacy fallback. |
+| 10 | Queue consumes Task 1 stages and Task 6 relations | Internally consistent; it must not alter supplier ledger totals. |
+| 11 | Review actions consume Tasks 1, 6, 8 and existing dn_item_aliases | Internally consistent after plan correction distinguishing supplier aliases from item aliases. |
+| 12 | Manual posting consumes Task 8 and staged state from Task 7 | Internally consistent; auto-post and manual post share the same idempotent promotion function. |
+| 1 + 3 | Schema fields against upload payload | Compatible; source_count is updated only after successful source writes. |
+| 2 + 4 | Canonical MIME against vision data URL | Compatible; extract must always use image/png after normalisation. |
+| 4 + 5 | Printed pagination and continuation fields | Compatible; deterministic assembly uses values actually present on each page. |
+| 5 + 8 | Assembly revisions against posted candidates | Compatible; revision endpoint rejects posted candidates. |
+| 6 + 8 | Supplier-scoped identity | Compatible; exact PO may derive supplier before duplicate identity is evaluated. |
+| 7 + 12 | Automatic and manual posting paths | Compatible if both call the same gate/promotion functions; reviewers must reject duplicated posting logic. |
+| 9 + 11 | UI review actions against API validation | Compatible; browser suggestions are never authoritative. |
+
+## Rulings
+
+Ruling: Matching policy belongs in a pure resolver shared by legacy and staged adapters — keeps site scoping and auto-match safety in one unit — cost if wrong: a later refactor may be needed to untangle DB loading.
+
+Ruling: Only unposted partial promotions may be reconciled automatically; posted candidates require explicit correction — preserves auditability — cost if wrong: recovery from a mid-write failure may need a manual action.
+
+Task 1: dispatched to /root/luna_task1 at base 7ab55703b4041c41bcda378b74bd56306498b0d4
+
+Task 1: Ruling: retain deployable apply capability but require explicit --apply plus an exact host confirmation; development and automated verification remain dry-run only — the spec requires an additive migration that can eventually be applied — cost if wrong: production schema application has extra operator ceremony.
+
+Task 1: fix round 1/5 (3 addressed, 2 open — replace same-name legacy index instead of appending; align review-event type optionality; commits 34dbdeb..ad23b65)
+
+Task 1: fix round 2/5 (2 addressed, 0 open; commits ad23b65..498b258)
+Task 1: minor (deferred): index-name parser handles the migration's current conventional SQLite syntax, not every possible quoted/IF-NOT-EXISTS variant.
+Task 1: complete (commits 7ab5570..498b258, review clean)
+
+Task 2: dispatched to /root/luna_task2 at base 498b25858aed52df2bfb1360c9fff4386d7bc997
+
+Task 2: review found extractPageRange still trusted declared MIME, breaking mismatched-PDF page evidence; splitPages comment is stale.
+
+Task 2: fix round 1/5 (2 addressed, 0 open; commits 81c8f78..80fbe15)
+Task 2: complete (commits 498b258..80fbe15, review clean)
+
+Task 3: dispatched at base 80fbe15
+
+Task 3: review found duplicate-only uploads create empty batches, capture session filter interpolation is unsafe, legacy single-file validation regressed, and persisted source_count remains false.
+
+Task 3: fix round 1/5 (2 addressed, 4 open — oversize got detail; source_count update failure; in-request hash dedup; report accuracy; commits e7e92fd..798b804)
+
+Task 3: fix round 2/5 (2 addressed, 2 open — containment if final update and cleanup both fail; same-hash outcome after primary write failure; commits 798b804..a8ed5a1)
+
+Task 3: fix round 3/5 (3 addressed, 0 open; commits a8ed5a1..ba52884)
+Task 3: complete (commits 80fbe15..ba52884, review clean)
+
+Task 4: dispatched at base ba5288465e0ccd82c76a9fcc1af858b399b8549b
+
+Task 4: Ruling: production runner integration of readOrReusePage is deferred to Task 7 run-staged.ts, as the plan explicitly assigns orchestration there; Task 4 owns the tested primitive — cost if wrong: cache code remains unused until Task 7 and could drift before integration.
+Task 4: review found failure-state write errors are swallowed, rich types are weaker than the specified parser contract, and cached extraction bypasses validation.
+
+Task 4: fix round 1/5 (3 addressed, 1 open — report fix SHA mismatch; commits 0bd344a..ea791d6)
+
+Task 4: fix round 2/5 (1 addressed, 0 open; commits ea791d6..a1882ac)
+Task 4: complete (commits ba52884..a1882ac, review clean)
+
+Task 5: dispatched at base a1882ac
+
+Task 5: Ruling: assembly_revision belongs on dn_batches because one revision covers the complete page grouping; retain note-level assembly_revision only as candidate provenance — cost if wrong: future per-candidate revisions may need a second token.
+Task 5: review found missing batch schema field, unstable note ordering, incomplete posted-state guard, malformed-group gaps, and mobile hints overriding conflicting printed pagination.
+
+Task 5: fix round 1/5 (6 addressed, 2 open — stale candidates survived group merges; revision check was not serialized; commits ab57baf..ab57baf)
+
+Task 5: fix round 2/5 (2 addressed, 2 open — dismissed candidates could be deleted; report retained a dangling final-commit claim; commits ab57baf..297853f)
+
+Task 5: fix round 3/5 (2 addressed, 0 open — dismissed candidates are immutable during regrouping; report provenance corrected; commits 297853f..59c158c)
+Task 5: complete (commits a1882ac..59c158c, review clean)
+
+Task 6: dispatched to /root/luna_task6 at base 59c158ce37e2d0b758f7e3b16cf54268f2ce4850
+
+Task 6: review found supplier evidence was dropped at segmentation, real PocketBase PO projection requested a nonexistent field, supplier identifier schema/loading was absent, and low-confidence fuzzy evidence was discarded.
+
+Task 6: fix round 1/5 (5 addressed, 0 open — supplier evidence preserved; schema/loading aligned; fuzzy evidence retained; commits e598613..927c09b)
+Task 6: complete (commits 59c158c..927c09b, review clean)
+
+Task 7: dispatched to /root/tara_task7 at base 927c09bbb27891b34604a077ed27915adac544af
+
+Task 7: Ruling: lease record ids are acquisition-specific and derived from batch plus a fresh owner token; the unique batch index, not a reused record id, is the atomic gate — PocketBase only supports id-based delete, so distinct ids fence old releases from replacement locks — cost if wrong: callers or tests that assumed one fixed id per batch must be adapted.
+
+Task 7: review found unsafe replacement-lock deletion, missing claimed_at on staged work, transient note-index candidate identity, and reused pages not reading their persisted canonical image.
+
+Task 7: fix round 1/5 (2 addressed, 3 open — candidate identity still changed on recovered continuations and ignored keyless rows; PocketBase file URLs lacked collection metadata; commits 1f988a9..02d2cbf)
+
+Task 7: fix round 2/5 (4 addressed, 1 open — surplus reconciliation did not recognize legacy promoted rows as terminal; commits 02d2cbf..66bf2a0)
+
+Task 7: fix round 3/5 (1 addressed, 0 open — legacy promoted and delivery-note-linked rows preserved; commits 66bf2a0..6ff5b59)
+Task 7: complete (commits 927c09b..6ff5b59, review clean)
+
+Task 8: dispatched to /root/luna_task8 at base 6ff5b59
+
+Task 8: review found legacy reruns could mutate posted candidates, create races did not converge, runner MIME/name provenance was lost, and promotion-failure visibility writes were swallowed.
+
+Task 8: fix round 1/5 (4 addressed, 2 open — candidate read failed open; failure-state writes retained a posted-state TOCTOU race; commits 4820706..43391a8)
+
+Task 8: Ruling: the additive staged-ingestion migration is a deployment prerequisite; missing dn_pipeline_locks fails closed with an actionable schema-required error rather than falling back to unlocked promotion — preserves posting correctness — cost if wrong: deployment order is stricter and unmigrated installations cannot run the new promotion code.
+
+Task 8: fix round 2/5 (2 addressed, 3 open — legacy upsert remained outside lease; lease lacked renewal; missing lock schema was reported as contention; commits 43391a8..e3e7135)
+
+Task 8: fix round 3/5 (3 addressed, 2 open — page-store writes were not lease-aware; ownership assertions were not transactionally fenced with mutations; commits e3e7135..16f42a7)
+
+Task 8: Ruling: protected PocketBase mutations must use the server transactional batch API to update the current acquisition record and the business row together; heartbeats remain liveness only — this closes the expiry/reclaim race without weakening reclaim — cost if wrong: all protected write adapters must support PocketBase batch semantics and older PocketBase servers cannot run this release.
+
+Task 8: fix round 4/5 (2 addressed, 0 open — page/candidate/manual/financial writes transactionally fenced; SDK batch support required; commits 16f42a7..919124f)
+Task 8: complete (commits 6ff5b59..919124f, review clean)
+
+Task 9: dispatched to /root/luna_task9 at base 919124f
+
+Task 9: review found preparing rows hidden, empty groups after join, open pages not actionable, persisted page groups/dismissals not rehydrated, direct PocketBase image URLs bypassed protection, and missing upper page bound.
+
+Task 9: fix round 1/5 (7 addressed, 2 open — orphan pages absent from both groups/open_pages remained invisible; server page imported a runtime helper from a client module; commits 61c0e60..d22d7b1)
+
+Task 9: fix round 2/5 (2 addressed, 0 open — server-safe model helpers and actionable orphan pages; commits d22d7b1..26143e5)
+Task 9: complete (commits 919124f..26143e5, review clean)
+
+Task 10: dispatched to /root/luna_task10 at base 26143e5
+
+Task 10: review found staged doc_number/doc_type were not authoritative, raw supplier text incorrectly escaped Unknown supplier, supplier counts were quadratic, and tests did not render/match server mapping.
+
+Task 10: fix round 1/5 (4 addressed, 1 open — empty-string date prevented posted/uploaded fallback; commits 2e5e504..b2582d3)
+
+Task 10: fix round 2/5 (1 addressed, 0 open — date fallback restored; commits b2582d3..bf00cc0)
+Task 10: complete (commits 26143e5..bf00cc0, review clean)
+
+Task 11: dispatched to /root/luna_task11 at base bf00cc0
+
+Task 11: review found the existing correction UI still used the obsolete request body, readiness failed open when gates were missing, item linking allowed unresolved site/supplier ownership, alias approval metadata was incomplete, and null JSON could raise a server error.
+
+Task 11: fix round 1/5 dispatched (5 findings plus focused regression coverage requested; commits 25f8112..70117628 under review)
+
+Task 11: fix round 1/5 (5 addressed, 3 open — persisted financial checks could be ignored; missing assembly revisions passed; normalized alias keys could be empty; commits 70117628..9ce4250)
+
+Task 11: fix round 2/5 dispatched (3 findings plus focused regression coverage requested)
+
+Task 11: fix round 2/5 (3 production findings addressed, 2 proof gaps open — unknown failed-check coverage and route-level transactional rollback evidence; commits 9ce4250..0846ce5)
+
+Task 11: fix round 3/5 dispatched (tests only)
+
+Task 11: fix round 3/5 (2 proof gaps addressed, 0 open; commits 0846ce5..45b3c66)
+Task 11: complete (commits bf00cc0..45b3c66, review clean)
+
+Task 12: dispatched at base 45b3c66
+Task 12: complete (code commit 5d09418, report commit e6fef00; focused and complete verification passed; production migration and feature flag unchanged)

.superpowers/sdd/2026-09-18-hybrid-delivery-note-ingestion/task-12-report.md

@@ -0,0 +1,32 @@
+# Task 12 report
+
+## Implementation
+
+- Code commit: `5d094183e66663dca69e7d95325cabebc38e5d82`
+- Added `POST /api/dn-batches/[id]/notes/[noteId]/post`.
+  - Reloads the batch, candidate, pages, supplier, PO, every PO item, and checks server-side.
+  - Reuses the fail-closed readiness evaluator and returns `409 needs_review` with reasons.
+  - Calls `promoteNote` directly so it owns the batch lease.
+  - Validates posted-candidate associations and returns the existing delivery note on retry.
+  - Adds an idempotent `post` audit event after promotion.
+- Added read-only `scripts/compare-dn-pipelines.ts` with `<batch-id>` and `--all` modes.
+- Documented migration, environment, tick/lock recovery, shadow rollout, metrics, feature flag, rollback, and direct-financial-edit warnings in `docs/operations.md`.
+- Added route tests for review blocking, one-time posting, and idempotent retry.
+
+## Verification
+
+- `npx vitest run 'src/app/api/dn-batches/[id]/notes/[noteId]/post/route.test.ts' src/lib/dn-pipeline/readiness.test.ts src/lib/dn-pipeline/promote.test.ts --maxWorkers=1 --minWorkers=1`
+  - 36 tests passed.
+- `npm run typecheck`
+  - Passed.
+- `npm run test:run`
+  - 104 test files, 1,176 tests passed.
+  - No Poppler concurrency timeout occurred; serial rerun was not needed.
+- `npm run build`
+  - Passed; Next.js listed the new posting route.
+- `git diff --check`
+  - Passed.
+- `python3 scripts/pb-add-dn-staged-ingestion.py --dry-run` with the configured PocketBase environment
+  - Passed; reported intended additive collections/fields/indexes and no writes.
+
+Production migration was not applied and `DN_STAGED_INGESTION_ENABLED` was not changed.