WFDASH-WEBAPP/docs/SHIPPED.md

WFDASH — What We've Shipped

The one-month build log. 24 May → 22 Jun 2026. 15 commits · ~30K lines of real code · ~57K net lines added (incl. lockfile churn). One Next.js 16 CRM that took a legacy Django app and gave it AI, an email pipeline, and a real supplier map.

---

The Project

WFDASH is a business CRM — customers, companies, suppliers, quotes, orders, driver collection pool, delivery notes, repairs. It's a re-platform of an older Django app, rebuilt on a modern stack:

Layer	Choice
Framework	Next.js 16 (App Router) + React 19 + TypeScript
UI	shadcn/ui (base-nova style) + Tailwind 4 + Lucide
Maps	Leaflet + react-leaflet (clustering, marker edit, fullscreen, dark tiles)
Backend	PocketBase (DB + auth + file storage + realtime)
Automation	n8n (email monitoring, AI quote rewrite)
Agent API	in-app JSON-RPC MCP endpoint + REST /api/v1/*
Packaging	Multi-stage Docker (Node 20 alpine, non-root) + Nixpacks
Tests	Vitest (unit) + Playwright (E2E)

North star: "One scheduler, one workflow, one source of truth — no long-running process in the Next.js app, no racy counters, no fabricated cost prices."

---

The Timeline At A Glance

24 May ─── Foundation sprint (UI kit, PWA, PocketBase collections)
25 May ─── PocketBase APIs + migration docs
27 May ─── Supplier map (Leaflet) ships
11 Jun ─── Test scaffolding (Vitest + Playwright + line endings)
16 Jun ─── Email-to-quote pipeline (v3.1 agentic + PDF text)
22 Jun ─── Big batch: atomic numbering → email cards → AI rewriter → pricing lock → UX polish

By the numbers:

• 15 commits, ~57K net lines added (78K insertions / 22K deletions; ~27K of that is real code in src/)
• 29,582 lines currently living in src/, n8n_workfows/, scripts/, docs/, e2e/
• 5 n8n workflow files (Quote Email Monitor v3.1 → v4 → v5, Quote Item Rewriter, Error Handler)
• 14 PocketBase collections (12 from setup + 2 added with the email pipeline)
• 9 REST endpoints under /api/v1/* + 8 MCP tools under /api/mcp
• 15 Vitest unit suites + 3 Playwright E2E specs

The MCP tools list (the AI agent uses these): orders.list · orders.create_from_quote · collection_pool.list · order_items.update_status · quotes.create_from_email · quotes.append_reply · quotes.find_by_message_id · attachments.upload_for_quote

---

Phase 1 — Foundation (24 May)

Commit 363786e — UI kit shipped

The bones of the app. shadcn/ui components that the rest of the build rests on:

• separator, sheet, sonner, switch, table, tabs, textarea, tooltip
• data-table.tsx — generic table with sort/filter/pagination (became the workhorse for every list page)
• line-items-table.tsx — quote & order line items

Commit a828432 — Responsive & accessibility pass

Dashboard and quotes pages get a proper mobile-card layout, keyboard nav, and ARIA labels.

Commit 954dc26 — PWA scaffold

Service worker, offline page, manifest. The app installs to a phone home screen and survives flaky connections in the warehouse.

---

Phase 2 — Backend Wiring (25 May)

Commit fd1571e — PocketBase collections, APIs, migration docs

The schema + the agent surface area come online together:

• Base collections: customers, companies, suppliers, quotes, quote_items, orders, purchase_orders, order_items, collection_pool, delivery_notes, delivery_note_items, repairs, stock_items
• scripts/setup-pocketbase.ts — one-shot setup from npm run setup
• MCP endpoint (/api/mcp) with tools/list + tools/call
• REST endpoints under /api/v1/* (auth via Authorization: Bearer AGENT_API_KEY)
• Migration docs: core-recheck-matrix.md, django-nextjs-parity-backlog.md, order-workflow-rules.md

---

Phase 3 — Supplier Map (27 May)

Commits 565b1a6 → 54644c7 → 83935c9 → a6ac88a

A real, interactive supplier map. Five commits in one day because each fix surfaced the next:

Commit	What landed
565b1a6	Leaflet + react-leaflet, supplier markers, clustering
54644c7	Better clustering algorithm + readable labels
83935c9	Dark tile layer + fullscreen mode + click-to-edit marker coordinates
a6ac88a	Node 20.9.0 enforced via engines + nixpacks.toml (so CI builds don't surprise us with Node 18)

Why it matters: the warehouse team can see supplier locations at a glance, click any pin to fix a wrong coordinate, and the map works in dark mode in the cab at night.

---

Phase 4 — Test Scaffolding (11 Jun)

Commit c4e1c62 — Standardize line endings + full test + config scaffolding

A boring-but-critical commit:

• Vitest + jsdom + Testing Library, with path aliases matching tsconfig.json
• Unit suites for: cn(), quote-status, order-item-status (incl. legacy driver_status alias), agent-auth, orders-service, pocketbase-admin, every /api/v1/* route, every MCP tool
• Playwright E2E: health probe, agent auth (skips if no API key), login (skips if no creds)
• VSCode workspace + extensions + MCP config examples
• .gitignore extended for test artifacts

The "skips if no creds" behaviour is intentional — fresh checkout gets 1 passed, N skipped, never a red CI.

---

Phase 5 — Email-to-Quote Pipeline (16 Jun → 22 Jun)

This is the headline feature. Three workflow versions, one PocketBase schema, one atomic counter, and a long list of "no, really, it works this time" fixes.

The flow

IMAP poll (quotes@wfsales.co.za, every 5 min, UNSEEN only)
   ↓
Parse Email + binary preservation
   ↓
Drop blacklists / auto-replies
   ↓
Is reply to existing quote?
   ├── yes → MCP: quotes.append_reply → terminal
   └── no  → Carry ctx → Split Binary → Extract from File (PDF)
                                  ↓
                  AI Agent (MiniMax) with two tools:
                  ├─ create_quote_tool   (MCP: quotes.create_from_email)
                  └─ upload_attachment_tool (MCP: attachments.upload_for_quote)
                                  ↓
                  PocketBase record + per-attachment rows linked back

Commit 41b5fc4 — Pipeline v3.1 (agentic + PDF text)

Replaces the brittle v3 "Information Extractor → Code" pair with a langchain AI Agent that directly calls the MCP tools. PDF attachments go through Split Binary → Extract from File (PDF) → Aggregate so the LLM sees the line items as text. Flips passthroughBinaryImages: false so PNG attachments survive to the upload tool — the v3 bug where attachment_1 was eaten by LLM vision is fixed.

Side effects of this commit:

• pb_quote_attachments.collection.json + quote_counters.collection.json + quote_replies.collection.json
• Migration scripts: create-quote-attachments-and-replies.ts, broaden-quote-attachments-mime-types.ts
• scripts/n8n-v2-main.ts, scripts/n8n-v2-error.ts — the n8n Code nodes lifted into TypeScript for type-checking
• Smoke scripts: counter.ts, counter-concurrent.ts, create-from-email.ts, upload-attachment.ts
• src/lib/server/quotes-service.ts — 747 lines of business logic, fully unit-tested

Commit 3359195 — Atomic numbering + v5 catch-and-create

The bug: in v4, by the time the workflow reached "Upload Attachment", the binary had already been stripped → silent upload failure.

The fix (v5 "Catch and Create"): upload attachments to PocketBase immediately after auth (binary still fresh), then do all the rest using record IDs, then PATCH each attachment to link it to the quote. The old node is renamed DISABLED - Upload Attachment so it's a one-click rollback.

Atomic numbering:

• /api/v1/quotes/next-number returns Q-YYYY-MM-NNN
• Backed by quote_counters collection with findOrCreateByMessageId dedup
• Concurrent smoke test (counter-concurrent.ts) proves the counter doesn't double-issue under load

Files in this drop:

• n8n_workfows/WFDASH — Quote Email Monitor (v5 catch-and-create).json
• pb_quote_attachments.schema.json
• scripts/build_v5_workflow.py (regenerates v5 from v4)
• scripts/deploy_v5_to_n8n.py + scripts/verify_v5.py

A separate WFDASH — Error Handler workflow catches every failure in the main workflow and posts a Telegram alert so nothing falls into a black hole.

---

Phase 6 — Quote Cards & Color Coding (22 Jun)

Commit a18ab0f

The first thing a user sees when they open a quote is now genuinely useful:

• QuoteEmailCard overhaul — proper HTML/plain text body sanitization, automatic linkification, inline PDF/image previews, full metadata display, auth-aware attachment previews via blob URLs
• email_received_at field added to quotes collection + types + setup script
• /api/v1/users/lookup server endpoint — resolves user IDs to display names
• User color coding (src/lib/user-color.ts) — stable hash → color so claimed quotes are visually grouped per claimant
• Quotes list — tab filters reordered, claimer names shown, rows color-coded per claimant
• Quote detail page — claimer info with user icon
• DataTable — now accepts per-row custom class + style props (powers the color coding)

---

Phase 7 — AI Quote Item Rewriter (22 Jun)

Commit 2c6155a

The user pastes in a wall of supplier-speak ("1 x ABC pump model XYZ-9000 c/w 2" flanged inlet, 415V, IP55 rated...") and the rewriter turns it into clean, structured line items.

What landed:

• Backend: src/app/api/v1/quotes/items/rewrite/route.ts — 228 lines, calls n8n webhook, validates response, updates PB
• Frontend: src/components/rewrite-quote-modal.tsx — 716 lines, diff preview, per-item accept/reject
• n8n: WFDASH — Quote Item Rewriter.json — the workflow definition
• user_edited field — gates AI rewrites on user-edited content (don't overwrite what the user already cleaned up)
• Migration: scripts/migrations/add-quote-items-user-edited.ts (idempotent + backfills existing items)

Bonus features that rode along:

• SupplierSearch combobox — searchable combobox using cmdk, replaces plain <Select> on order + quote pages (216 lines, async load with debounce)
• working-days.ts utility + aged quote warning badges for overdue quotes
• Expandable row previews in the quotes list view
• DataTable overhaul — 567 lines changed to support row expansion, custom row styling, server-side state

---

Phase 8 — Pricing Lock & Quote Polish (22 Jun)

Commit 86c942a

The pricing story gets real:

• Locked source-of-truth pricing controls — once locked, cost/selling prices auto-calculate from the source. No more "two people edited the same cell, which wins?"
• Inline cost/selling inputs removed from the quote items table — they live in the locked pricing panel now
• Quote item fields extended: added model and size, removed old type
• /api/v1/quotes/items/rewrite timeout bumped to 180s — long AI batches need the headroom
• n8n rewriter workflow overhauled — updated prompt + null-safe output handling (won't crash on empty LLM response)
• Rewrite modal — sticky headers, better empty chip handling, cleaner styling

---

Phase 9 — UX Polish (22 Jun)

Commit 4c0b6b0 — Toasts + hide-completed toggle

Two quality-of-life wins:

New Sonner toaster

• top-center position with 24px offset (sits below sticky headers, not slammed to the edge)
• Min 360px / max 420px width — toasts have room to breathe
• rounded-xl border shadow-xl backdrop-blur-md — looks like a real notification, not a browser alert
• Brand colors per type (light + dark variants):

Type	Light bg	Dark bg	Icon
success	emerald-50	emerald-900/92%	emerald-600
error	red-50	red-900/92%	red-600
warning	amber-50	amber-900/92%	amber-600
info	blue-50	blue-900/92%	blue-600

• Up to 5 visible at once, 4s default duration, action buttons styled with proper tokens
• All 24+ toast call sites get the new look for free — zero migration

Hide/show completed quote items

• Toggle in the "Quote Items" card header (outline by default, solid indigo when active — deliberately different from the amber Rewrite button and the green done state)
• Incomplete items float to the top, done items sink to the bottom
• Row numbers stay stable — done item #5 still says "5.", not "1."
• Celebratory row when everything is done + hidden: All N items processed + a Show-completed button

Memoization

displayItems memo handles sort + filter together so the toggle is instant.

---

The Architecture In One Diagram

┌──────────────────── Browser ────────────────────┐
│  Next.js 16 App Router · React 19 · Tailwind 4  │
│                                                  │
│  /quotes  /quotes/[id]  /quotes/[id]/process     │
│  /orders  /customers  /supplier-map  /dashboard  │
│                                                  │
│  shadcn/ui · DataTable · Sonner · Leaflet       │
└──────────────┬──────────────────┬────────────────┘
               │                  │
        /api/v1/* REST     /api/mcp (JSON-RPC)
               │                  │
┌──────────────▼──────────────────▼────────────────┐
│           PocketBase (single binary)             │
│  customers · companies · suppliers · quotes      │
│  quote_items · orders · order_items_v2           │
│  collection_pool · delivery_notes · repairs      │
│  quote_attachments · quote_replies · counters    │
└──────────────▲──────────────────▲────────────────┘
               │                  │
   n8n Code nodes +            Admin scripts
   AI Agent (MiniMax)          (npm run setup)
               │                  │
┌──────────────┴──────────────────┴────────────────┐
│       n8n Workflows (single scheduler)           │
│  • Quote Email Monitor v5 (catch-and-create)    │
│  • Quote Item Rewriter (AI batch)                │
│  • Error Handler (Telegram alerts)               │
│                                                  │
│  Polls IMAP every 5 min · Dedupe by message_id  │
└──────────────────────────────────────────────────┘

---

What's Not Done Yet (Parity Backlog)

The Django parity backlog at docs/django-nextjs-parity-backlog.md lists the remaining work, in priority order:

#	Module	Status	What's missing
1	Customers	partial/live	Duplicate handling + merge strategy (replace legacy Django mgmt commands)
2	Companies	partial/live	Address consistency + relation integrity checks
3	Quotes	partial/live	Email intake done; remaining: explicit quote→order safeguards (mitigated by findOrCreateByMessageId)
4	Orders	partial/live	Status board + drv/collection states reflected in UI
5	Collection pool	partial/live	Assignment UI + filtered work queue
6	Delivery notes	not migrated	Generate delivery records from delivered/loaded order items
7	Repairs	not migrated	Rebuild only the useful request/order flow
8	Stock	intentionally folded	Separate stock collection later if needed

Out of scope (explicit): stock_orders, customer_portal, rep_portal, internal_stock, pos, ai_orders, system_monitor. AI / email / monitoring all live in n8n and only talk to the app via API.

---

The Decisions That Shaped This Build

A few calls we made that you might want to remember for the video:

1. n8n over a Next.js worker. Email polling + AI extraction runs in n8n, not in the Next.js process. The app stays stateless; n8n is the single scheduler. Reason: cold-start reliability + the AI Agent node already lives there.

2. PocketBase over Postgres. Single binary, no managed cloud, fits Coolify self-hosting. Cost = hosting only.

3. MCP + REST side-by-side. REST for humans/scripts (clean auth, clean error codes), MCP for n8n (one endpoint, tools/list + tools/call). Both go through the same quotes-service.ts.

4. v5 "catch and create" for attachments. Upload to PB first, link after. The binary is freshest at the top of the workflow; doing it later (when v4 tried) means the buffer is gone. Old node renamed DISABLED - Upload Attachment so rollback is one click.

5. Locked pricing over free-form inputs. Cost/selling prices live in a locked panel, not on the line item. No more "two people edited the same cell, which wins?"

6. Vendor ID → stable color hash. Claimed quotes are color-coded by claimant using a stable hash → Tailwind palette. No "Bob is teal today, indigo tomorrow" drift.

7. Tests skip cleanly. Vitest + Playwright both skip what they can't run (no API key → skip, no creds → skip). Fresh checkout gets 1 passed, N skipped, never a red CI.

---

Where The Code Lives

src/
├── app/
│   ├── (auth)/login/
│   ├── (app)/                        # Protected pages
│   │   ├── dashboard/  customers/  companies/
│   │   ├── suppliers/  supplier-map/
│   │   ├── quotes/                  # list · [id] · [id]/process · approvals · archived
│   │   ├── orders/  collection-pool/  delivery-notes/  repairs/
│   ├── api/
│   │   ├── mcp/route.ts             # JSON-RPC, no auth
│   │   └── v1/                      # Bearer auth
│   │       ├── health/  orders/  order-items/  collection-pool/
│   │       └── quotes/             # [id]/pdf · items/rewrite · next-number
│   └── offline.tsx                  # PWA fallback
├── components/                       # data-table, quote-form, supplier-map, etc.
└── lib/
    ├── pocketbase.ts · pb-helpers.ts · types.ts
    ├── user-color.ts · quote-status.ts · working-days.ts
    └── server/                      # agent-auth · orders-service · quotes-service · order-item-status

scripts/                              # setup-pocketbase + 4 migrations + 4 smoke tests + 2 v5 deploy scripts
n8n_workfows/                         # 5 workflow JSONs + 3 JS snippets + 1 live snapshot
e2e/                                  # Playwright specs (health, agent-api, auth)
docs/                                 # 8 design + parity + rollout docs

---

Try It Yourself

git clone <repo> && cd wfdash
npm install
./pocketbase serve &                 # in another terminal
cp .env.local.example .env.local     # edit ADMIN creds
npm run setup                        # creates 9 base collections
npm run pb:migrate:orders-v2         # +order_items_v2 +collection_pool
npm run pb:migrate:quote-attachments # +quote_attachments +quote_counters +quote_replies
npm run pb:migrate:quote-items-user-edited  # backfills the user_edited field
npm run dev                          # http://localhost:3000

# Run the tests
npm run test:run                     # Vitest
npm run e2e                          # Playwright (skips if no creds)

# Deploy the email pipeline
python scripts/deploy_v5_to_n8n.py   # one-shot n8n deploy
python scripts/verify_v5.py          # post-deploy sanity check

---

Last updated: 22 Jun 2026 — through commit 4c0b6b0