docs(dn): record task 7 verification
This commit is contained in:
DEls
parent
d6e9c471af
commit
1f988a9a7f
1 changed files with 129 additions and 0 deletions
|
|
@ -0,0 +1,129 @@
|
|||
# Task 7 report: atomic leases and staged orchestration
|
||||
|
||||
Status: complete
|
||||
|
||||
## Code commit
|
||||
|
||||
`d6e9c471afabc6c1a0bbf5f467c012eea7cc40ef` — `feat(dn): add staged ingestion orchestration`
|
||||
|
||||
## Files
|
||||
|
||||
- `src/lib/dn-pipeline/lock.ts` and `lock.test.ts`
|
||||
- Atomic create-based batch lease using a deterministic, valid 15-character
|
||||
PocketBase record id.
|
||||
- One expired row may be reclaimed; a live or unobservable competing row is
|
||||
not claimed.
|
||||
- A per-acquisition owner token is checked before release, so an old worker
|
||||
does not delete a reclaimed replacement lock.
|
||||
- `src/lib/dn-pipeline/run-staged.ts` and `run-staged.test.ts`
|
||||
- Reads immutable `dn_batch_sources`, normalises fresh canonical PNG pages,
|
||||
persists/reuses `dn_batch_pages` by source/page/region, and invokes
|
||||
`readOrReusePage` for the vision cache.
|
||||
- Assembles persisted reads, resolves entities only within the confirmed
|
||||
batch site, and independently creates/updates staged candidates. Clear
|
||||
candidates advance to `ready_to_post`; ambiguous candidates remain
|
||||
`needs_review`; no delivery-note or financial rows are written.
|
||||
- `src/app/api/dn-batches/tick/route.ts` and test
|
||||
- Uses `runStagedBatch` only when `DN_STAGED_INGESTION_ENABLED === "true"`.
|
||||
All other values retain the prior `runBatch`/claim path. Staged lease losers
|
||||
are reported as `not_claimed`, and shared retry accounting remains intact.
|
||||
- `src/app/api/dn-batches/[id]/assembly/route.ts` and test
|
||||
- Replaced the manual route's incompatible lease with `withBatchLease`.
|
||||
|
||||
## RED evidence
|
||||
|
||||
Before production modules existed:
|
||||
|
||||
```text
|
||||
npx vitest run src/lib/dn-pipeline/lock.test.ts
|
||||
FAIL: Failed to load url ./lock ... Does the file exist?
|
||||
Test Files 1 failed; Tests 0
|
||||
|
||||
npx vitest run src/lib/dn-pipeline/run-staged.test.ts
|
||||
FAIL: Failed to load url ./run-staged ... Does the file exist?
|
||||
Test Files 1 failed; Tests 0
|
||||
```
|
||||
|
||||
Before the manual route refactor:
|
||||
|
||||
```text
|
||||
npx vitest run 'src/app/api/dn-batches/[id]/assembly/route.test.ts'
|
||||
FAIL: expected lock create payload to contain deterministic id and an
|
||||
owner-qualified manual-assembly worker; existing payload lacked both.
|
||||
```
|
||||
|
||||
Before the tick switch:
|
||||
|
||||
```text
|
||||
npx vitest run src/app/api/dn-batches/tick/route.test.ts
|
||||
FAIL: staged runner was not called when DN_STAGED_INGESTION_ENABLED=true;
|
||||
lease-loser result was not reported as not_claimed.
|
||||
```
|
||||
|
||||
## GREEN evidence
|
||||
|
||||
```text
|
||||
npx vitest run src/lib/dn-pipeline/lock.test.ts
|
||||
3 passed
|
||||
|
||||
npx vitest run src/lib/dn-pipeline/run-staged.test.ts
|
||||
2 passed
|
||||
|
||||
npx vitest run 'src/app/api/dn-batches/[id]/assembly/route.test.ts'
|
||||
14 passed
|
||||
|
||||
npx vitest run src/app/api/dn-batches/tick/route.test.ts
|
||||
29 passed
|
||||
|
||||
npx vitest run src/lib/dn-pipeline/run.test.ts
|
||||
38 passed
|
||||
|
||||
npx vitest run src/lib/dn-pipeline src/app/api/dn-batches
|
||||
21 files passed; 285 tests passed
|
||||
|
||||
npm run typecheck
|
||||
passed
|
||||
|
||||
python3 scripts/pb-add-dn-staged-ingestion.py --dry-run
|
||||
passed; offline preview only, no schema changes written
|
||||
|
||||
git diff --check
|
||||
passed
|
||||
```
|
||||
|
||||
The expected test logs contain deliberate mocked failure/reclaim paths and the
|
||||
existing legacy promotion-failure fixture; no test failed.
|
||||
|
||||
## Decisions and self-review
|
||||
|
||||
- The lease claim is an initial `create` with a deterministic record id; it
|
||||
does not use the old read/update/re-read claim sequence. Reads occur only
|
||||
after a failed create, to inspect a possible expiry. The lease tests cover a
|
||||
create race, one expiry reclaim, final release, and a replaced-owner row
|
||||
that must survive the first worker's `finally`.
|
||||
- Source files are fetched read-only. Page rows always store `image/png` PNG
|
||||
bytes from `normaliseSource`; the original source record/file is never
|
||||
updated. Cached successful reads bypass a second extractor invocation on a
|
||||
rerun.
|
||||
- The confirmed batch site is passed both to the site-filtered PO query and to
|
||||
`resolveEntities`. No coordinate, Clock, image-quality, or nearest-site code
|
||||
was changed.
|
||||
- Failed reads remain visible in `page_count` and `open_pages`; their sibling
|
||||
groups still assemble and stage. Candidate upserts use the existing batch
|
||||
note index and leave dismissed/posted candidates unchanged, so reruns do not
|
||||
create duplicate page/candidate records.
|
||||
- Staged orchestration deliberately ends at independent candidates. It never
|
||||
calls `promoteNote`, `delivery_notes`, or `delivery_note_items`; Task 8 owns
|
||||
promotion and ledger idempotency.
|
||||
- Read before route edits: Next 16 Route Handlers guide and `route.ts` file
|
||||
convention reference. The route retains `force-dynamic`, Node runtime,
|
||||
secret validation, stale-attempt handling, and the default legacy path.
|
||||
|
||||
## Concerns
|
||||
|
||||
- No PocketBase migration was applied; dry-run was intentionally used only.
|
||||
- The owner check before delete prevents a late worker from deleting an
|
||||
already-observed replacement. PocketBase exposes no conditional-delete API,
|
||||
so a replacement exactly between that ownership read and delete remains a
|
||||
service-level compare-and-delete limitation; the lock's short lease and
|
||||
owner-aware check minimize that window.
|
||||
Loading…
Reference in a new issue