WFDASH-WEBAPP/docs/wfdash-v5-deployment.md
SJCoetz 33591959c2 feat: add quote email intake and atomic numbering
This commit implements the full inbound quote email processing workflow and atomic quote numbering:
- Add PocketBase collections for monthly atomic quote counters, email attachments, and threaded replies
- Add idempotent migration script to set up new collections and add deduplication field to existing quotes table
- Implement `/api/v1/quotes/next-number` endpoint for properly formatted Q-YYYY-MM-NNN quote numbers
- Add MCP tools for idempotent quote creation, reply threading, and message ID lookup
- Add end-to-end and concurrent smoke tests for pipeline validation
- Update TypeScript types for new quote-related entities
- Update project documentation and package.json scripts
- Mark quote email intake as complete in the parity backlog
- Add n8n v2 workflows for automated email monitoring and error alerts
2026-06-22 13:58:24 +02:00

7.1 KiB

WFDASH — v5 Catch and Create Deployment

You said "implement it" — so here's the full deployment package.

What changed (in plain English)

Before (v4): Email arrives → 13+ HTTP/Code nodes → reach Upload Attachment → binary already stripped → upload fails silently.

After (v5 Catch and Create): Email arrives → auth → upload attachments to PB IMMEDIATELY (binary still fresh) → continue with normal processing → after quote is created, PATCH each attachment record to link it to the quote.

The 3-second version: Upload the file first, get a PB record ID, then do everything else using IDs. The original "Upload Attachment" node is renamed DISABLED - Upload Attachment and disconnected — easy to re-enable if you need to roll back.

Files in this drop

Path What it is
n8n_workfows/WFDASH — Quote Email Monitor (v5 catch-and-create).json The new workflow. Import into n8n.
pb_quote_attachments.schema.json New PocketBase collection. Import via PB admin UI.
scripts/build_v5_workflow.py The Python script that generated v5 from v4. Re-run if you tweak v4.
scripts/verify_v5.py Sanity-check script. Run after import to confirm wiring is correct.

v4 is untouched. Original file is still at n8n_workfows/WFDASH — Quote Email Monitor (v4 direct-to-PB).json.


Deployment — do these in order

Step 1 — PocketBase schema (5 min)

  1. Open PB admin UI → Settings → Import collections

  2. Upload pb_quote_attachments.schema.json

  3. PB will create the quote_attachments collection and assign its own 15-char ID

  4. Important: open the new collection's settings → copy the assigned ID. The PB collection URLs in the v5 workflow point at the collection NAME (/api/collections/quote_attachments/records) which PB resolves to the ID, so no edits needed — but verify with a curl:

    curl -s https://your-pocketbase/api/collections/quote_attachments/records \
      -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
    

    You should get a {"items":[],"page":1,...} response.

  5. Verify the unique index is in place:

    sqlite3 pb_data/data.db ".indexes quote_attachments"
    

    You should see idx_qa_msg_filename and idx_qa_quote.

Step 2 — n8n workflow (5 min)

  1. n8n → Workflows → Import from File → select WFDASH — Quote Email Monitor (v5 catch-and-create).json
  2. Open the imported workflow. You should see in the canvas:
    • 3 new nodes in a parallel branch right after PocketBase Admin Auth: Plan: Upload Attachments Early → Upload Attachment Early → Collect Attachment IDs
    • 2 new nodes after Create Quote Item: Link Attachments to Quote → PATCH Attachment (set quote)
    • 2 greyed-out nodes (the rollback safety): DISABLED - Plan: Upload Attachments, DISABLED - Upload Attachment
  3. Verify credentials are intact:
    • IMAP Quote Mailbox → IMAP account credential (should auto-resolve from existing)
    • PocketBase Admin Auth → uses pb_url/pb_admin_email/pb_admin_password from the Set node. No credential required (PB admin auth via JSON body).
  4. Save & Activate.

Step 3 — Verify (10 min)

cd /Volumes/EXT/DEVELOPMENT/WFDASH_NEXTJS
python3 scripts/verify_v5.py

Should print all green checkmarks.

Step 4 — Test with a real RFQ (15 min)

  1. Send yourself a test email with 1-2 PDF attachments to the IMAP mailbox
  2. Watch the n8n execution:
    • Plan: Upload Attachments Early should emit N items (one per attachment)
    • Upload Attachment Early should show 200 OK with attachment record IDs
    • The chain continues to customer/company/quote creation
    • Link Attachments to Quote → PATCH Attachment (set quote) should PATCH each attachment with quote=<quote_id>
    • Mark has_attachments should PATCH the quote record
  3. Verify in PB admin UI:
    • quotes collection: new record with correct customer, company, line items
    • quote_attachments collection: 1-2 new records with file field populated, quote field linked
  4. Open the dashboard / NextJS UI and confirm the attachments are downloadable from the quote page

Step 5 — Disable v4 (only after Step 4 passes)

In n8n, deactivate the v4 workflow (don't delete — keep for rollback). v5 becomes the active one.


What "Catch and Create" actually does (the conceptual bit)

Three-phase pattern, all in one workflow:

Phase What happens Binary state Failure mode
Catch Upload Attachment Early POSTs each attachment to PB the moment auth completes. The binary is fresh on this branch — no upstream HTTP node has consumed it. Fresh PB rejects → onError: continueRegularOutput → _attachment_record_ids: [] → quote still gets created, just without attachments
Create Collect Attachment IDs aggregates PB record IDs into an array on the email item. The rest of the chain (customer, company, AI extraction, quote creation) works with text/IDs only — binary is irrelevant from here. (released) AI failure → quote still gets created with email metadata, just without line items
Link After the quote exists, Link Attachments to Quote emits one item per PB attachment ID, and PATCH Attachment (set quote) sets the quote field on each. (n/a) PATCH failure → attachment is still in PB, just orphaned. Easy to re-link later via PB admin UI or a small repair workflow.

The key insight: as soon as the file is in PB, every failure downstream becomes recoverable. That's the property worth having.


If something breaks

Attachments aren't getting uploaded:

  • Check the n8n execution view. Is Plan: Upload Attachments Early emitting items?
  • Run n8n execute --raw and inspect the binary on the input to that node
  • Verify PB admin token has create permission on quote_attachments

Attachments uploaded but not linked to quote:

  • Link Attachments to Quote should emit one item per ID. If it emits 0, check _attachment_record_ids on the upstream item.
  • The PATCH URL is {{ $json.pb_url }}/api/collections/quote_attachments/records/{{ $json.attachment_id }} — make sure the IDs are 15 chars.

Quote created but no attachments in dashboard:

  • has_attachments is patched to true on the quote. Check that the dashboard reads this field.
  • Verify the quote_attachments collection has a list/view rule allowing the dashboard's user to read.

Want to roll back to v4:

  1. In n8n, deactivate v5
  2. Activate v4
  3. The broken Upload Attachment chain in v4 is still there. It still won't work (binary issue is structural in v4), but it won't get worse.

What's intentionally NOT in this drop

  • No Nextcloud integration. You mentioned it; we agreed to keep this PR focused on the upload fix. Nextcloud as backup is a separate 30-min job when you want it.
  • No changes to the WFDASH frontend (src/). The dashboard should already read from the quote_attachments collection — if it doesn't, that's a separate frontend task.
  • No changes to the AI extraction logic. v4's "Extract Items (LLM)" chain is unchanged.

If any of those turn out to need work, just say the word.