This file captures decisions, surprises, and parser quirks as we build the JEI invoice tracker.
---
## 2026-09-17 — Discovery phase
### Discovered while inspecting uploaded PDFs
1.**One PDF can hold multiple invoices.**
`1.Invoice_29.05.pdf` has 4 pages and contains 4 separate invoices (161453, 161454, 161611, 161852). The parser must split per page and detect a new "Invoice No" token.
2.**The buyer's legal entity is constant.**
Every observed invoice shows `K2023742851 SOUTH AFRICA (PTY) LTD` as the legal buyer. Only the T/A line varies. The user mentioned 3 buyer-name variants — these map to T/A values on top of a constant legal entity.
3.**Deliver-To can be a different company.**
On several invoices the "Deliver To" is `SHIPLEY GLOBAL EVENTS (PTY) LTD, 678 MIRROR CARD ST, GARSFONTEIN, PRETORIA 0042` — a related entity where goods physically land. Captured as `deliver_to_name` + `deliver_to_address`.
4.**36 files in folder, not all are JEI.**
FedEx shipping docs and credit notes (in ZAR, not GBP) were moved to `invoices/_non_jei/`. Remaining: 33 JEI files (invoices, proforma, 1 quote).
5.**Empty PDFs observed.**
`Invoice.pdf` and `JEI Supplier Invoice.pdf` have no extractable text (likely failed exports). Will be flagged in `data/review/unparseable.txt` for re-export.
### Documents in scope vs out of scope (this build)
| Document type | Action |
|---|---|
| JEI invoice | Import as `document_type = invoice` |
| JEI proforma (paid in full) | Import as `document_type = proforma` |
| JEI quote | Import as `document_type = quote` |
| FedEx shipping/credit note | Skip (in `_non_jei/`) |
### User decisions captured
- **Deliver-to:** Capture as separate fields.
- **Quotes:** Treat same as invoices; clean up later against JEI statement.
- **Non-JEI files:** Move to `_non_jei/` subfolder; ignore in importer.
### Parser heuristics (planned)
- **Document boundary:** New page containing `Invoice No` or `Quotation No` token.
- **Document type:** `QUOTATION` → quote; `INVOICE` → invoice; proforma detected via filename keyword or stamp on the page.
- **Buyer variant:** Match T/A line against three known strings.
- **Stock code:** Pull from the table column; if it's just `SERIAL` (e.g. for serial-number line items), skip that line — it's not a stock code.
- **Deliver-to:** Read from the `Deliver To :` block at the bottom of the invoice.
### Open questions / things to watch during build
- Two empty PDFs need user attention (re-export from source).
- One large multi-invoice PDF (`JEI Drilling Invoices.pdf`, 6.4 MB) — verify parser doesn't choke on size.
- Some older invoices use 20% VAT column even though Total VAT is shown as £0.00 (reverse-charge mechanism — exporter doesn't charge VAT). Verify parser handles `Total 20% VAT £ 0.00` gracefully.
- One invoice has both "Customer Order No" and "Sales Order No" fields; both are captured (PO = customer's PO; sales order = supplier's internal ref). The user only mentioned the customer one; we capture both for free.
| 161453 | `Invoice 161453 from JEI... - 14_04_2026.pdf` + `Invoice 161453.pdf` | Different |
Verified by reading both files for invoice 154923 — same data, different PDF layout (one is flow-text, one is more grid-aligned). Not byte-identical files, but **identical invoice data**.
**Dedup strategy:** the importer relies on the `JEI_invoices.invoice_no + document_type` uniqueness — second occurrence skipped silently. The first one imported wins (its `pdf_file` is kept). User can later delete the duplicate filename's reference via PB admin if needed (the duplicate file's data is merged into the first record).
- "Customer Order No" field on this invoice reads literally `Email 28.07.25` (looks like a separate "Email [date]" reference the supplier uses). The importer will capture it as-is. User can clean later.
- The invoice is stamped **PAID IN FULL**. Some invoices are pre-paid (proforma-paid or settled-on-delivery). We don't track this in the schema yet (future phase) but it's visible on the original PDF.
- ✅ Next.js 14 + Tailwind app on **http://localhost:5001** (changed from 6002 → 6003 → 5001 on 2026-09-18; 6002 was the Workflow CIPC service, 6003 conflicted briefly with user's other services, 5001 was the user's preferred port)
1.**POST /api/collections uses key `fields` not `schema`.** Docs show `schema` but the API silently accepts `schema` and stores under `fields`. Always send `fields: [...]`.
2.**Field options go at the FIELD ROOT, not inside `options`.** This is the opposite of what docs for older PB versions show. e.g.
- ❌ FAILS: `{ name: "foo", type: "select", options: { maxSelect: 1, values: ["a","b"] } }` — returns `fields.X.values.code: validation_required, Cannot be blank.` even though the values array is non-empty.
- Same applies to relation fields: `collectionId` and `cascadeDelete` go at the field root, not in `options`.
3.**GET response hides field options.** After creating a collection, `GET /api/collections/<id>` returns `fields` with `options: {}` (empty) for select/relation fields. The data IS stored — just not returned. PATCH the same field with explicit options to confirm (it succeeds).
4.**PATCH round-trip "wakes up" newly created collections.** Same trick as the existing memory entry — if a collection 403s on read or returns empty fields, PATCHing any rule field (even with the existing value) often fixes it.
### Parser quirks remaining
- 4 unparseable PDFs (empty text — likely failed exports: Invoice.pdf, JEI Supplier Invoice.pdf, JEI SA Invoice 147842.pdf, JEI SA Proforma 148931.pdf).
- Some multi-page PDFs (151778, 151779) have page-boundary issues — the parser creates multiple records from the same physical invoice when it shouldn't.
- "JEISA000" got imported as a stock code (it's actually the buyer's account reference) — minor data issue, can be deleted via admin UI.
- 161453, 161852: lines parsed as 0 (one of the page variants has a different layout that the parser doesn't recognize).
### Token expiry
PB token expires **2026-09-18 19:10 SAST** (~22h from now). Re-grab from DevTools → Application → Local Storage → `pocketbase_auth.token` on the PB admin UI when needed.
4. To refresh PB token: visit `https://pb.agent.apphub.co.za/_/`, copy `pocketbase_auth.token` from DevTools → Application → Local Storage, paste into `JEI/.env.local`.
