SJ/JEI_Import_Documents-App
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

notes: duplicate file findings + dedup strategy

Browse source
This commit is contained in:
Mavis 2026-09-17 21:35:07 +02:00
parent f1f28ba13a
commit 97d0339aa6
1 changed files with 22 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
NOTES.md
View file

@ -69,3 +69,25 @@ This file captures decisions, surprises, and parser quirks as we build the JEI i
- One large multi-invoice PDF (`JEI Drilling Invoices.pdf`, 6.4 MB) — verify parser doesn't choke on size.
- Some older invoices use 20% VAT column even though Total VAT is shown as £0.00 (reverse-charge mechanism — exporter doesn't charge VAT). Verify parser handles `Total 20% VAT £ 0.00` gracefully.
- One invoice has both "Customer Order No" and "Sales Order No" fields; both are captured (PO = customer's PO; sales order = supplier's internal ref). The user only mentioned the customer one; we capture both for free.
### Duplicate-file check (user-requested)
Searched all 33 JEI filenames for repeated invoice numbers. Found 4 logical duplicates:
| Invoice No | Files | Hash |
|---|---|---|
| 154923 | `154923 Supplier Invoice.pdf` + `JEI 154923 89.10+102.pdf` | Different (different PDF render) |
| 155655 | `155655 Supplier Invoice.pdf` + `JEI 155655.pdf` | Different |
| 156092 | `156092 Supplier Invoice.pdf` + `JEI 156092.pdf` | Different |
| 161453 | `Invoice 161453 from JEI... - 14_04_2026.pdf` + `Invoice 161453.pdf` | Different |
Verified by reading both files for invoice 154923 — same data, different PDF layout (one is flow-text, one is more grid-aligned). Not byte-identical files, but **identical invoice data**.
**Dedup strategy:** the importer relies on the `JEI_invoices.invoice_no + document_type` uniqueness — second occurrence skipped silently. The first one imported wins (its `pdf_file` is kept). User can later delete the duplicate filename's reference via PB admin if needed (the duplicate file's data is merged into the first record).
**Impact:** no schema change needed. Idempotency rule (§5.2) already covers this.
### Additional observations from invoice 154923
- "Customer Order No" field on this invoice reads literally `Email 28.07.25` (looks like a separate "Email [date]" reference the supplier uses). The importer will capture it as-is. User can clean later.
- The invoice is stamped **PAID IN FULL**. Some invoices are pre-paid (proforma-paid or settled-on-delivery). We don't track this in the schema yet (future phase) but it's visible on the original PDF.