docs: plan hybrid delivery note ingestion
This commit is contained in:
DEls
parent
5bf8036210
commit
7ab55703b4
1 changed files with 732 additions and 0 deletions
|
|
@ -0,0 +1,732 @@
|
|||
# Hybrid Delivery-Note Ingestion Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox syntax for tracking.
|
||||
|
||||
**Goal:** Upgrade the existing delivery-note inbox into a photo-first staged pipeline that preserves bulk source files, persists page reads, assembles documents, resolves supplier and PO within the selected site, and exposes only genuine ambiguity for review.
|
||||
|
||||
**Architecture:** A batch owns one or more immutable source files; each source produces normalised page images; each page gets one cached vision extraction; pages are assembled into candidate documents; candidates resolve supplier, PO, and items before idempotent promotion into the existing ledger. PocketBase stores every stage, application code owns matching and posting, and n8n remains only the scheduler.
|
||||
|
||||
**Tech Stack:** Next.js 16 App Router, TypeScript, React 19, Vitest, PocketBase, MiniMax through the OpenAI-compatible client, Poppler, pdf-lib, sharp, Tailwind, existing UI primitives.
|
||||
|
||||
**Spec:** docs/superpowers/specs/2026-09-18-hybrid-delivery-note-ingestion-design.md
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Implement Increment A only. Do not introduce the receipt/invoice three-way-match model or change month-end payable semantics.
|
||||
- Read node_modules/next/dist/docs/01-app/01-getting-started/15-route-handlers.md and node_modules/next/dist/docs/01-app/01-getting-started/05-server-and-client-components.md before changing Next.js routes or pages.
|
||||
- Preserve the original bytes of every source file. Vision only receives decoded, auto-rotated, correctly labelled images.
|
||||
- Treat the confirmed batch site as authoritative. Coordinates are evidence, never an automatic override.
|
||||
- Auto-resolve only deterministic unique matches. Fuzzy supplier, PO, or item matches remain review candidates.
|
||||
- An unmatched line contributes zero financial quantity until a person links it.
|
||||
- One failed or ambiguous candidate must not block clear sibling candidates.
|
||||
- Every stage and retry must be idempotent.
|
||||
- Keep the legacy runner available behind DN_STAGED_INGESTION_ENABLED until shadow comparison and one-site rollout pass.
|
||||
- Do not modify or revert concurrent Clock/photo-app work in src/app/clock/**, src/lib/image-quality*, or src/lib/nearest-site*.
|
||||
- Use apply_patch for manual edits. Preserve all unrelated working-tree changes.
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| scripts/pb-add-dn-staged-ingestion.py | Additive PocketBase schema and indexes. |
|
||||
| src/lib/dn-pipeline/stages.ts | Legacy/new status mapping and terminal-state predicates. |
|
||||
| src/lib/dn-pipeline/normalise-source.ts | PDF/raster input to canonical PNG pages. |
|
||||
| src/lib/dn-pipeline/page-store.ts | Idempotent source/page persistence. |
|
||||
| src/lib/dn-pipeline/extract.ts | One canonical page to a validated rich PageRead. |
|
||||
| src/lib/dn-pipeline/assemble.ts | Pure page-read to document-group assembly. |
|
||||
| src/lib/dn-pipeline/resolve-entities.ts | Pure site-scoped supplier and PO resolution. |
|
||||
| src/lib/dn-pipeline/lock.ts | Atomic PocketBase processing lease. |
|
||||
| src/lib/dn-pipeline/run-staged.ts | Stage orchestration and independent candidate processing. |
|
||||
| src/lib/dn-pipeline/promote.ts | Idempotent validated candidate promotion. |
|
||||
| src/app/api/delivery-notes/upload/route.ts | Backward-compatible multi-source intake. |
|
||||
| src/app/api/dn-batches/[id]/assembly/route.ts | Manual split/join revision endpoint. |
|
||||
| src/app/dashboard/financial/delivery-notes/inbox/** | Batch board and split review. |
|
||||
| src/app/dashboard/financial/delivery-notes/work-queue/** | Grouped candidate queue and line linker. |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add staged-ingestion schema and status vocabulary
|
||||
|
||||
**Files:**
|
||||
- Create: scripts/pb-add-dn-staged-ingestion.py
|
||||
- Create: src/lib/dn-pipeline/stages.ts
|
||||
- Test: src/lib/dn-pipeline/stages.test.ts
|
||||
- Modify: src/lib/pb-types.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Produces BatchStage, CandidateStage, displayBatchStage(), displayCandidateStage(), and isCandidateTerminal().
|
||||
- Produces PocketBase collections dn_batch_sources, dn_batch_pages, supplier_document_aliases, dn_review_events, and dn_pipeline_locks.
|
||||
|
||||
- [ ] **Step 1: Write the failing status compatibility tests**
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
displayBatchStage,
|
||||
displayCandidateStage,
|
||||
isCandidateTerminal,
|
||||
} from "./stages";
|
||||
|
||||
describe("stage compatibility", () => {
|
||||
it("maps legacy states without rewriting live rows", () => {
|
||||
expect(displayBatchStage("analysing", "")).toBe("reading");
|
||||
expect(displayCandidateStage("blocked", "")).toBe("needs_review");
|
||||
expect(displayCandidateStage("promoted", "")).toBe("posted");
|
||||
});
|
||||
|
||||
it("prefers an explicit stage", () => {
|
||||
expect(displayBatchStage("needs_attention", "split_review")).toBe("split_review");
|
||||
expect(displayCandidateStage("blocked", "matching")).toBe("matching");
|
||||
});
|
||||
|
||||
it("recognises terminal candidates", () => {
|
||||
expect(isCandidateTerminal("posted")).toBe(true);
|
||||
expect(isCandidateTerminal("dismissed")).toBe(true);
|
||||
expect(isCandidateTerminal("needs_review")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
- [ ] **Step 2: Run the test and verify the missing-module failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/stages.test.ts
|
||||
|
||||
Expected: FAIL because ./stages does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the pure status module**
|
||||
|
||||
export type BatchStage =
|
||||
| "queued" | "preparing" | "reading"
|
||||
| "split_review" | "complete" | "failed";
|
||||
|
||||
export type CandidateStage =
|
||||
| "matching" | "needs_review" | "ready_to_post"
|
||||
| "posted" | "dismissed";
|
||||
|
||||
export function displayBatchStage(legacy: string, explicit: string): BatchStage {
|
||||
if (["queued", "preparing", "reading", "split_review", "complete", "failed"].includes(explicit)) {
|
||||
return explicit as BatchStage;
|
||||
}
|
||||
if (legacy === "analysing") return "reading";
|
||||
if (legacy === "processed") return "complete";
|
||||
if (legacy === "needs_attention") return "split_review";
|
||||
return "queued";
|
||||
}
|
||||
|
||||
export function displayCandidateStage(legacy: string, explicit: string): CandidateStage {
|
||||
if (["matching", "needs_review", "ready_to_post", "posted", "dismissed"].includes(explicit)) {
|
||||
return explicit as CandidateStage;
|
||||
}
|
||||
if (legacy === "promoted") return "posted";
|
||||
if (legacy === "dismissed") return "dismissed";
|
||||
if (legacy === "blocked") return "needs_review";
|
||||
return "ready_to_post";
|
||||
}
|
||||
|
||||
export const isCandidateTerminal = (stage: CandidateStage) =>
|
||||
stage === "posted" || stage === "dismissed";
|
||||
|
||||
- [ ] **Step 4: Add the schema script and record types**
|
||||
|
||||
The idempotent script must support --dry-run, reuse scripts/pb_env.py, and create these fields:
|
||||
|
||||
dn_batch_sources:
|
||||
batch, file, file_hash, ordinal, mime, original_name,
|
||||
capture_session_id, delivery_session_id, captured_at,
|
||||
latitude, longitude, location_accuracy_m, created, updated
|
||||
|
||||
dn_batch_pages:
|
||||
batch, source, source_page, region, image, image_hash,
|
||||
mime, width, height, rotation, read_status, extraction,
|
||||
schema_version, model, prompt_version, read_error,
|
||||
manual_group, created, updated
|
||||
|
||||
supplier_document_aliases:
|
||||
supplier, raw_value, normalised_key, source, approved_by,
|
||||
created, updated
|
||||
|
||||
dn_review_events:
|
||||
batch, note, action, before, after, reason, actor, created, updated
|
||||
|
||||
dn_pipeline_locks:
|
||||
batch, expires_at, worker, created, updated
|
||||
|
||||
Extend dn_batches with stage, source_count, and processing_completed_at.
|
||||
Extend dn_batch_notes with stage, assembly_revision, pages, supplier,
|
||||
supplier_name_raw, match_method, match_evidence, and posted_at.
|
||||
Add optional relation batch_note to delivery_notes.
|
||||
|
||||
Add unique indexes on source(batch,file_hash), page(source,source_page,region),
|
||||
alias(normalised_key), lock(batch), and delivery_notes(batch_note). Add exact
|
||||
matching TypeScript record interfaces to src/lib/pb-types.ts.
|
||||
|
||||
Also add a partial unique capture-session index for non-empty mobile
|
||||
capture_session_id values so an offline retry can find, rather than duplicate,
|
||||
its original batch.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run:
|
||||
|
||||
npx vitest run src/lib/dn-pipeline/stages.test.ts
|
||||
npm run typecheck
|
||||
python3 scripts/pb-add-dn-staged-ingestion.py --dry-run
|
||||
|
||||
Expected: tests and typecheck pass; dry-run prints changes and writes nothing.
|
||||
|
||||
Commit:
|
||||
|
||||
git add scripts/pb-add-dn-staged-ingestion.py src/lib/dn-pipeline/stages.ts src/lib/dn-pipeline/stages.test.ts src/lib/pb-types.ts
|
||||
git commit -m "feat(dn): add staged ingestion schema"
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Normalise PDFs and photographs into canonical pages
|
||||
|
||||
**Files:**
|
||||
- Create: src/lib/dn-pipeline/normalise-source.ts
|
||||
- Test: src/lib/dn-pipeline/normalise-source.test.ts
|
||||
- Modify: src/lib/dn-pipeline/split.ts
|
||||
- Modify: package.json
|
||||
- Modify: package-lock.json
|
||||
|
||||
**Interfaces:**
|
||||
- Produces normaliseSource(input, declaredMime): Promise<NormalisedPage[]>.
|
||||
- NormalisedPage is { sourcePage, png, mime: "image/png", width, height, rotation }.
|
||||
|
||||
- [ ] **Step 1: Install sharp and write failing tests**
|
||||
|
||||
Run: npm install sharp
|
||||
|
||||
Test a generated JPEG and a two-page PDF:
|
||||
|
||||
import sharp from "sharp";
|
||||
import { PDFDocument } from "pdf-lib";
|
||||
import { expect, it } from "vitest";
|
||||
import { normaliseSource } from "./normalise-source";
|
||||
|
||||
it("returns a real PNG for JPEG input", async () => {
|
||||
const jpeg = await sharp({
|
||||
create: { width: 80, height: 120, channels: 3, background: "white" },
|
||||
}).jpeg().toBuffer();
|
||||
const [page] = await normaliseSource(jpeg, "image/jpeg");
|
||||
expect(page.mime).toBe("image/png");
|
||||
expect((await sharp(page.png).metadata()).format).toBe("png");
|
||||
});
|
||||
|
||||
it("renders every PDF page", async () => {
|
||||
const pdf = await PDFDocument.create();
|
||||
pdf.addPage([200, 300]);
|
||||
pdf.addPage([200, 300]);
|
||||
expect(await normaliseSource(Buffer.from(await pdf.save()), "application/pdf"))
|
||||
.toHaveLength(2);
|
||||
});
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/normalise-source.test.ts
|
||||
|
||||
Expected: FAIL because normalise-source.ts is missing.
|
||||
|
||||
- [ ] **Step 3: Implement normalisation**
|
||||
|
||||
Raster path:
|
||||
|
||||
sharp(buffer)
|
||||
.rotate()
|
||||
.resize({ width: 2200, height: 2200, fit: "inside", withoutEnlargement: true })
|
||||
.png()
|
||||
|
||||
PDF path renders every page through Poppler and sends each result through the
|
||||
same raster function. Detect PDF magic bytes and actual decoder metadata rather
|
||||
than trusting extension/MIME. Reject undecodable HEIC/HEIF/TIFF with
|
||||
unsupported_image_codec. Preserve originals outside this function.
|
||||
|
||||
Keep splitPages() as an adapter:
|
||||
|
||||
export async function splitPages(buffer: Buffer, mime: string): Promise<PageImage[]> {
|
||||
const pages = await normaliseSource(buffer, mime);
|
||||
return pages.map((p) => ({ page: p.sourcePage, png: p.png }));
|
||||
}
|
||||
|
||||
- [ ] **Step 4: Add regression tests**
|
||||
|
||||
Test mismatched declared MIME, invalid bytes, EXIF orientation, bounded
|
||||
dimensions, and no accidental enlargement.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run:
|
||||
|
||||
npx vitest run src/lib/dn-pipeline/normalise-source.test.ts src/lib/dn-pipeline/split.test.ts
|
||||
npm run typecheck
|
||||
|
||||
Commit dependency, implementation, tests, and split adapter.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Store many immutable sources in one batch
|
||||
|
||||
**Files:**
|
||||
- Modify: src/app/api/delivery-notes/upload/route.ts
|
||||
- Test: src/app/api/delivery-notes/upload/route.test.ts
|
||||
- Modify: src/components/upload-delivery-note-button.tsx
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes dn_batch_sources from Task 1.
|
||||
- Produces one response with batch_id, source_count, results, and status while retaining legacy single-file fields.
|
||||
|
||||
- [ ] **Step 1: Write new multi-file tests**
|
||||
|
||||
Assert one dn_batches create and three dn_batch_sources creates for three
|
||||
files. Test repeated capture_session_id returns the existing batch, duplicate
|
||||
hashes are reported per file, and one invalid file does not erase valid
|
||||
sources.
|
||||
|
||||
- [ ] **Step 2: Verify the old behaviour fails**
|
||||
|
||||
Run: npx vitest run src/app/api/delivery-notes/upload/route.test.ts
|
||||
|
||||
Expected: FAIL because the route currently creates one batch per file.
|
||||
|
||||
- [ ] **Step 3: Implement one-batch intake**
|
||||
|
||||
Create the batch, then one source row per accepted file using FormData.
|
||||
Preserve the 20 MB per-file and 50-file limits. Accept source,
|
||||
capture_session_id, delivery_session_id, captured_at, latitude, longitude,
|
||||
and location_accuracy_m. A single-file request keeps siteId, projectCode,
|
||||
created, and batch_id in its response.
|
||||
|
||||
- [ ] **Step 4: Update the upload dialog**
|
||||
|
||||
Report "N files added to one processing batch" with per-file duplicate/error
|
||||
details. Do not change site selection or Clock scanner files.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run route tests and npm run typecheck, then commit route, tests, and component.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Persist and cache one rich vision read per page
|
||||
|
||||
**Files:**
|
||||
- Create: src/lib/dn-pipeline/page-store.ts
|
||||
- Test: src/lib/dn-pipeline/page-store.test.ts
|
||||
- Modify: src/lib/dn-pipeline/types.ts
|
||||
- Modify: src/lib/dn-pipeline/extract.ts
|
||||
- Test: src/lib/dn-pipeline/extract.test.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Produces readOrReusePage(pb, input): Promise<PageRead>.
|
||||
- Extends PageRead with supplier/site text, PO candidates, printed pagination,
|
||||
vehicle registration, raw transcription, and evidence.
|
||||
|
||||
- [ ] **Step 1: Add failing parser tests**
|
||||
|
||||
Parse a full JSON fixture and defaults for absent optional fields. Assert a
|
||||
model parse failure throws instead of becoming an empty successful page.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/extract.test.ts
|
||||
|
||||
- [ ] **Step 3: Extend type, schema, and prompt**
|
||||
|
||||
Add:
|
||||
|
||||
supplier_name_raw: string | null;
|
||||
supplier_registration_raw: string | null;
|
||||
site_name_raw: string | null;
|
||||
po_number_candidates: string[];
|
||||
printed_page_number: number | null;
|
||||
printed_page_count: number | null;
|
||||
vehicle_registration: string | null;
|
||||
raw_text: string;
|
||||
evidence: Record<string, string>;
|
||||
|
||||
Always send canonical data:image/png bytes. Preserve the current commercial
|
||||
fields. Never infer from neighbouring pages.
|
||||
|
||||
- [ ] **Step 4: Implement cached persistence**
|
||||
|
||||
readOrReusePage returns extraction immediately when read_status is read and
|
||||
schema/prompt versions match. Otherwise it marks reading, calls the injected
|
||||
extractor, stores validated extraction/model/version on success, or stores
|
||||
failed plus a bounded error and rethrows. Tests use fake PocketBase calls and
|
||||
prove success is reused.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run extractor/page-store tests and typecheck, then commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Assemble candidates and support manual revisions
|
||||
|
||||
**Files:**
|
||||
- Create: src/lib/dn-pipeline/assemble.ts
|
||||
- Test: src/lib/dn-pipeline/assemble.test.ts
|
||||
- Create: src/app/api/dn-batches/[id]/assembly/route.ts
|
||||
- Test: src/app/api/dn-batches/[id]/assembly/route.test.ts
|
||||
- Modify: src/lib/dn-pipeline/segment.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Produces assemblePages(reads, hints): AssemblyResult.
|
||||
- AssemblyResult contains groups, openPages, and requiresReview.
|
||||
- Each PageGroup contains key, pages, reason, and confidence.
|
||||
|
||||
- [ ] **Step 1: Write failing assembly tests**
|
||||
|
||||
Cover ten distinct numbers, a two-page note with printed pagination, a header
|
||||
plus continuation, conflicting numbers, a multi-document page, and a mobile
|
||||
delivery_session_id that never overrides conflicting printed numbers.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/assemble.test.ts
|
||||
|
||||
- [ ] **Step 3: Implement deterministic assembly**
|
||||
|
||||
Keep segmentPages() as a compatibility adapter. Every page must appear exactly
|
||||
once in a group or openPages; duplicate or missing coverage throws.
|
||||
|
||||
- [ ] **Step 4: Implement the revision route test-first**
|
||||
|
||||
Accept:
|
||||
|
||||
{
|
||||
revision: number,
|
||||
groups: { pages: number[] }[],
|
||||
dismissedPages: { page: number, reason: string }[],
|
||||
reason: string
|
||||
}
|
||||
|
||||
Return 409 for stale revisions or posted candidates and 400 for missing or
|
||||
duplicated pages. On success rewrite only unposted candidate groups, increment
|
||||
assembly_revision, append a review event, and do not call vision.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run assembly and route tests plus typecheck, then commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Resolve supplier and PO within the confirmed site
|
||||
|
||||
**Files:**
|
||||
- Create: src/lib/dn-pipeline/resolve-entities.ts
|
||||
- Test: src/lib/dn-pipeline/resolve-entities.test.ts
|
||||
- Modify: src/lib/dn-pipeline/run.ts
|
||||
- Modify: src/lib/po-matching.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Produces resolveEntities(input): EntityResolution.
|
||||
- EntityResolution contains supplierId, purchaseOrderId, method, autoResolved,
|
||||
and ordered candidates with evidence.
|
||||
|
||||
- [ ] **Step 1: Write failing resolver tests**
|
||||
|
||||
Cover exact PO deriving supplier with blank supplier text, exclusion of another
|
||||
site's matching PO, approved aliases, ambiguous fuzzy PO results, and no safe
|
||||
winner margin.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/resolve-entities.test.ts
|
||||
|
||||
- [ ] **Step 3: Implement ordered evidence resolution**
|
||||
|
||||
Receive already site-filtered POs, suppliers, aliases, and lines. Match unique
|
||||
normalised PO first, then registration identifier, approved alias, exact
|
||||
normalised supplier name, and finally ranked suggestions. Only unique
|
||||
deterministic results set autoResolved true.
|
||||
|
||||
- [ ] **Step 4: Replace global PO loading**
|
||||
|
||||
Load POs with a batch-site filter, derive supplier from the PO relation, and
|
||||
persist match_method plus match_evidence. Remove the global fuzzy auto-pick
|
||||
from run.ts.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run resolver, PO matching, and pipeline tests plus typecheck, then commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Add atomic leases and staged orchestration
|
||||
|
||||
**Files:**
|
||||
- Create: src/lib/dn-pipeline/lock.ts
|
||||
- Test: src/lib/dn-pipeline/lock.test.ts
|
||||
- Create: src/lib/dn-pipeline/run-staged.ts
|
||||
- Test: src/lib/dn-pipeline/run-staged.test.ts
|
||||
- Modify: src/app/api/dn-batches/tick/route.ts
|
||||
- Test: src/app/api/dn-batches/tick/route.test.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Produces withBatchLease(pb, batchId, worker, fn).
|
||||
- Produces runStagedBatch(pb, batchId): Promise<StagedRunResult>.
|
||||
|
||||
- [ ] **Step 1: Write lease race and expiry tests**
|
||||
|
||||
Make a second create of the deterministic lock id fail. Assert only the winner
|
||||
calls fn, expired locks reclaim once, and finally releases the winner's lock.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run: npx vitest run src/lib/dn-pipeline/lock.test.ts
|
||||
|
||||
- [ ] **Step 3: Implement atomic create-based leases**
|
||||
|
||||
Claim by creating a deterministic valid PocketBase id, never by
|
||||
read/update/reread. Existing unexpired lock returns acquired false. Expired
|
||||
lock is deleted and create retried once.
|
||||
|
||||
- [ ] **Step 4: Build staged runner test-first**
|
||||
|
||||
Prove source normalisation creates/reuses pages, successful reads cache,
|
||||
stages advance, clear notes process despite a blocked sibling, failed pages
|
||||
remain counted, and reruns duplicate nothing. Tick keeps secret/attempt guards
|
||||
and reports not_claimed to losing workers. When
|
||||
DN_STAGED_INGESTION_ENABLED is not exactly true, tick continues calling the
|
||||
legacy runBatch path; when true, it calls runStagedBatch.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run lock, staged runner, tick, existing runner tests, and typecheck; commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Harden promotion and duplicate identity
|
||||
|
||||
**Files:**
|
||||
- Modify: src/lib/dn-pipeline/promote.ts
|
||||
- Test: src/lib/dn-pipeline/promote.test.ts
|
||||
- Modify: src/lib/dn-pipeline/checks.ts
|
||||
- Test: src/lib/dn-pipeline/checks.test.ts
|
||||
|
||||
**Interfaces:**
|
||||
- Uses batchNoteId as the permanent promotion key.
|
||||
- Uses supplier plus document number as duplicate identity when supplier exists.
|
||||
|
||||
- [ ] **Step 1: Write failing safety tests**
|
||||
|
||||
Assert repeated promotion updates one delivery note, different suppliers may
|
||||
share a document number, same supplier/document blocks, and image candidates
|
||||
retain actual MIME/name.
|
||||
|
||||
- [ ] **Step 2: Verify failures**
|
||||
|
||||
Run promote and checks tests.
|
||||
|
||||
- [ ] **Step 3: Implement candidate-keyed promotion**
|
||||
|
||||
Set delivery_notes.batch_note. Look up by it before create. Preserve actual
|
||||
file type. Mark posted only after all line writes succeed. On partial failure
|
||||
leave a review event and reconcile the same candidate on retry.
|
||||
|
||||
- [ ] **Step 4: Scope duplicate identity**
|
||||
|
||||
Implement:
|
||||
|
||||
documentIdentity(supplierId, number) {
|
||||
return (supplierId ?? "unknown") + ":" + normaliseDocumentNumber(number);
|
||||
}
|
||||
|
||||
Unknown-supplier duplicates remain review candidates rather than automatic
|
||||
global blocks.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run promotion/check/pipeline tests and typecheck; commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Build batch stages and split review
|
||||
|
||||
**Files:**
|
||||
- Modify: src/app/dashboard/financial/delivery-notes/inbox/page.tsx
|
||||
- Modify: src/app/dashboard/financial/delivery-notes/inbox/inbox-board.tsx
|
||||
- Modify: src/app/dashboard/financial/delivery-notes/inbox/[id]/page.tsx
|
||||
- Modify: src/app/dashboard/financial/delivery-notes/inbox/[id]/batch-detail-view.tsx
|
||||
- Modify: src/app/api/dn-batches/[id]/pages/[n]/route.ts
|
||||
- Test: src/app/api/dn-batches/[id]/pages/[n]/route.test.ts
|
||||
- Create: src/app/dashboard/financial/delivery-notes/inbox/split-review.tsx
|
||||
- Test: src/app/dashboard/financial/delivery-notes/inbox/split-review.test.tsx
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes stage helpers and assembly route.
|
||||
- Produces Queued, Reading photos, Check page grouping, Complete, Failed.
|
||||
|
||||
- [ ] **Step 1: Write component tests**
|
||||
|
||||
Render four pages grouped [1,2], [3], with open page 4. Assert page labels,
|
||||
Join previous, Split here, Dismiss page, and Save grouping. Internal check ids
|
||||
are hidden until technical details opens.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run the split-review component test.
|
||||
|
||||
- [ ] **Step 3: Implement board mapping**
|
||||
|
||||
Use displayBatchStage and show source/page/document counts plus plain-language
|
||||
actions. Legacy rows continue to render.
|
||||
|
||||
- [ ] **Step 4: Implement split review**
|
||||
|
||||
Update the existing page route to return a persisted dn_batch_pages image when
|
||||
available and fall back to legacy on-demand rendering. Render those thumbnails,
|
||||
edit groups locally, send one revision-checked patch, show a reload message on
|
||||
409, and require a dismissal reason. Component tests should use
|
||||
react-dom/server unless the repository gains a browser component-test library.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run component tests and typecheck; commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Build the supplier/PO-grouped work queue
|
||||
|
||||
**Files:**
|
||||
- Create: src/app/dashboard/financial/delivery-notes/work-queue/page.tsx
|
||||
- Create: src/app/dashboard/financial/delivery-notes/work-queue/work-queue-view.tsx
|
||||
- Test: src/app/dashboard/financial/delivery-notes/work-queue/work-queue.test.tsx
|
||||
- Modify: src/app/dashboard/financial/suppliers/page.tsx
|
||||
- Modify: src/app/dashboard/financial/suppliers/suppliers-view.tsx
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes candidate stages and expanded supplier/PO/site relations.
|
||||
- Produces site, supplier, purchase_order, stage, reason, and document_type filters.
|
||||
|
||||
- [ ] **Step 1: Write grouping tests**
|
||||
|
||||
Use Vula/PO A, Vula/unknown PO, Raumix/PO B, and unknown supplier fixtures.
|
||||
Assert supplier and PO groups, exception groups, counts, and filtered links.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run the work-queue component test.
|
||||
|
||||
- [ ] **Step 3: Implement the server query**
|
||||
|
||||
Load candidates with expanded site, supplier, PO, and delivery note. Apply site
|
||||
scope and pass serialisable view models to the client.
|
||||
|
||||
- [ ] **Step 4: Implement grouping and supplier links**
|
||||
|
||||
Cards show document identity, type/date, batch, match method, and one primary
|
||||
action. Supplier rows show unresolved counts linking to the queue without
|
||||
changing ledger totals.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run queue tests and typecheck; commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 11: Build correction actions and line-item linking
|
||||
|
||||
**Files:**
|
||||
- Create: src/app/api/dn-batches/[id]/notes/[noteId]/resolve-entities/route.ts
|
||||
- Test: src/app/api/dn-batches/[id]/notes/[noteId]/resolve-entities/route.test.ts
|
||||
- Modify: src/app/api/dn-batches/[id]/notes/[noteId]/resolve-item/route.ts
|
||||
- Test: src/app/api/dn-batches/[id]/notes/[noteId]/resolve-item/route.test.ts
|
||||
- Create: src/app/dashboard/financial/delivery-notes/work-queue/line-linker.tsx
|
||||
- Test: src/app/dashboard/financial/delivery-notes/work-queue/line-linker.test.tsx
|
||||
|
||||
**Interfaces:**
|
||||
- Entity route accepts supplierId, purchaseOrderId, reason, and
|
||||
rememberSupplierAlias.
|
||||
- Item route accepts lineIndex, purchaseOrderItemId, reason, and rememberAlias.
|
||||
|
||||
- [ ] **Step 1: Write API audit tests**
|
||||
|
||||
Assert selected PO belongs to site and supplier, cross-site selection returns
|
||||
400, accepted changes append review events, rememberSupplierAlias upserts one
|
||||
approved supplier_document_aliases row, and item rememberAlias upserts one
|
||||
dn_item_aliases row without duplicates.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run both route test files.
|
||||
|
||||
- [ ] **Step 3: Implement server-validated corrections**
|
||||
|
||||
Reload all relationships from PocketBase. Re-run deterministic checks. Set
|
||||
ready_to_post only when gates pass; otherwise keep needs_review.
|
||||
|
||||
- [ ] **Step 4: Build responsive linker test-first**
|
||||
|
||||
Desktop uses left extracted/right PO columns with candidate buttons. Narrow
|
||||
layouts stack cards. Show codes, quantities, units, conversion, and rate
|
||||
evidence. Do not draw SVG spider lines. Confirm replacement of manual links.
|
||||
|
||||
- [ ] **Step 5: Verify and commit**
|
||||
|
||||
Run route/linker tests and typecheck; commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 12: Add posting action, shadow comparison, and final verification
|
||||
|
||||
**Files:**
|
||||
- Create: src/app/api/dn-batches/[id]/notes/[noteId]/post/route.ts
|
||||
- Test: src/app/api/dn-batches/[id]/notes/[noteId]/post/route.test.ts
|
||||
- Create: scripts/compare-dn-pipelines.ts
|
||||
- Modify: docs/operations.md
|
||||
|
||||
**Interfaces:**
|
||||
- Posting returns deliveryNoteId, itemsCreated, and stage posted.
|
||||
- Comparison reports page coverage, candidate count, PO/item match, units,
|
||||
quantity, and computed value without ledger writes.
|
||||
|
||||
- [ ] **Step 1: Write posting-gate tests**
|
||||
|
||||
Assert needs_review returns 409 with reasons, ready_to_post posts once, retry
|
||||
returns the same id, and posted candidates cannot be regrouped or re-extracted.
|
||||
|
||||
- [ ] **Step 2: Verify failure**
|
||||
|
||||
Run the post route test.
|
||||
|
||||
- [ ] **Step 3: Implement posting and audit**
|
||||
|
||||
Reload checks and relations server-side, call promotion, append a posted audit
|
||||
event, and return the existing row on retry.
|
||||
|
||||
- [ ] **Step 4: Add dry-run comparison and operations documentation**
|
||||
|
||||
The comparison script accepts a batch id or --all and never promotes. Document
|
||||
schema dry-run/apply, required environment, tick/lock recovery, feature flag,
|
||||
one-site rollout, and rollback to the legacy runner.
|
||||
|
||||
- [ ] **Step 5: Run complete verification**
|
||||
|
||||
npm run typecheck
|
||||
npm run test:run
|
||||
npm run build
|
||||
git diff --check
|
||||
python3 scripts/pb-add-dn-staged-ingestion.py --dry-run
|
||||
|
||||
Do not mutate production PocketBase during automated verification.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
Commit posting route, tests, comparison script, and operations documentation.
|
||||
|
||||
---
|
||||
|
||||
## Final Review Checklist
|
||||
|
||||
- [ ] Originals remain downloadable.
|
||||
- [ ] JPEG bytes are never mislabeled as PNG or PDF.
|
||||
- [ ] Every page is grouped, open, failed, or dismissed.
|
||||
- [ ] Exact site-scoped POs can derive supplier.
|
||||
- [ ] Fuzzy matches never auto-post.
|
||||
- [ ] Blocked siblings do not block valid notes.
|
||||
- [ ] Duplicate identity is supplier-scoped.
|
||||
- [ ] Retries create no duplicate values.
|
||||
- [ ] Manual actions create audit events.
|
||||
- [ ] Existing PO, supplier, and delivery-note pages work.
|
||||
- [ ] Concurrent Clock/photo-app changes remain untouched.
|
||||
Loading…
Reference in a new issue