WFDASH-WEBAPP/docs/audit/order-pipeline-investigation-2026-08-20.md

Order Pipeline Investigation — 2026-08-20

Scope: investigate (1) hit-and-miss contact_name population on orders, (2) duplicate-order creation when the same RFQ email arrives more than once, and (3) general email-intake failures. Audit covers the n8n workflows, the WFDASH Next.js app, and the live PocketBase data.

TL;DR

Symptom	Root cause	Severity
84.6% of auto-created orders are duplicates of the same email	The IMAP poll keeps returning the same message every 5 minutes; the workflow has no email_message_id dedup check before pb.collection('orders').create(...)	Critical
contact_name is "hit and miss" (sometimes "Neville Benadé", sometimes "Nbenade\n", sometimes \n)	The contact name is sourced from a non-deterministic LLM call (no temperature cap, no fallback to deterministic email-signature parsing); the same email produces a different name on every poll cycle	High
The 9 most recent orders (2026-08-19) have empty email_message_id / email_subject	Those are counter-created orders — the post-create email field is not set on the manual /counter form. Not a bug, but it makes the "duplicate by msgid" dedup path useless for counter orders	Low
"Some emails fail" — orders created with NO-REF, [NEEDS REVIEW], or description = '[NEEDS REVIEW] AUTO-FLAGGED: ...'	The AI Agent returns order_items = [] whenever the PDF text extraction can't find a clean line-items table. The workflow then takes the "no items extracted" branch which writes a flag and a raw blob, but the customer / contact_name / order_number come from the same non-deterministic AI call so they may still be wrong	Medium

The dedup mechanism that already exists (POST /api/v1/orders/duplicates/cancel) is reactive — it finds duplicates after the fact and lets the operator cancel them. Nothing prevents the duplicates from being created in the first place.

---

1. Pipeline overview

Active order-related n8n workflows (as of 2026-08-19 23:46 UTC)

ID	Name	Trigger	Purpose
DWkxko0SS1uoelsg	WFDASH — ORDER MONITOR (Unified)	Schedule (5 min)	The only pipeline that creates orders records. Polls IMAP, classifies, extracts via AI Agent, creates order + items.
DWkxko0SS1uoelsg (multi branch)	(same workflow)	(same)	The (Multi) branch handles multi-PO emails by splitting attachments by PO number detected in subject/body.
O0yDcNVHEhA0Dd5n	WFDASH — Multi-Order Fulfillment Pipeline	(other)	Creates quotes (not orders) for multi-PO emails via /api/v1/quotes/from-email.
45h7IYegCmhtkV9t	WFDASH — Multi-Order RFQ Fulfillment Pipeline	Schedule (2 min)	Creates quotes (not orders) for multi-PO emails via /api/v1/quotes/from-email.
mXX2yu5svRurnfwO	WFDASH - CATEGORISE ORDERS ITEMS	Schedule (1 min)	Backfills ai_item_insights and category enums. Does not create orders.
paqlZICDbXbmY61V	WFDASH — Populate Item Intel (Live)	Schedule	Item intel backfill.
UqO3ayETx9cQHOzF	WFDASH — Purchase Order Send	Webhook	Sends PO emails.
GGS5zgVbdfsRyLcY	ORDER-MARKUP-APPROVAL-LINKS	Webhook	WhatsApp/email markup-approval links.
kVylcetxy858PcfP	WFDASH — Error Handler	Error trigger	Telegram error alerts.
aI0Ze2vwgp25ZgGH	WFDASH — Error Notifier (Telegram)	Error trigger	Telegram error alerts.
DDFN9NBmh95yG2Jk	WFDASH — AI Insights Generator (v2)	Webhook	Re-runs AI insights on demand.

The ORDER MONITOR is the only thing that creates orders records from email. Its flow is:

Schedule Trigger (every 5 min)
  -> CHECK FOR NEW MAIL              (IMAP read, seen:false)
  -> SetEmailFlags                   (mark as seen immediately)
  -> Classify Email (Deterministic)  (bounce / supplier / multi_order / customer / uncertain)
  -> Route Email (Switch)            (route to one of 5 branches)
  -> Limit                           (limit to 1 — but parameters: {}, so probably a no-op)
  -> Download ATTACHMENT
  -> Split Attachments1
  -> Switch1                         (by MIME type -> PDF / CSV / XLS / XLSX / TXT / image)
  -> Extract PDF / CSV / XLS / XLSX / TXT (or Analyze image for image attachments)
  -> Merge2
  -> Code in JavaScript4             (combine all extracted text)
  -> AI Agent                        (LLM extraction -> JSON { order_info, order_items })
  -> If1                             (if order_items.length > 0 -> single-order branch)
  -> Search customers                (PB getList by name/email fuzzy)
  -> Create orders                   (PB create on orders collection)
  -> Code in JavaScript2             (split order_items -> order_items rows)
  -> Create order_items1             (PB create per item)
  -> Aggregate
  -> Merge1 + PocketBase1 (none) + MoveEmail email2 (move to INBOX.202608.Completed)

The single-order and multi-order branches converge at Merge order completion and finally move the email to INBOX.202608.Completed (or Failed on the failure branch).

PocketBase orders schema (live — pbc_3527180448)

30 fields, 0 indexes, 1 unique constraint (the implicit primary key). Relevant fields:

• order_number (text, required) — no unique constraint
• status (select: new / processing / processed / cancelled)
• company (relation, required)
• customer (relation, optional)
• contact_name (text, optional) — written by the Create orders n8n node from the AI agent's order_info.contact_name
• email_subject, email_body, email_sender, email_message_id (text, optional)
• rfq_number, description, notes, rep (text, optional)

The n8n Create orders node writes email_message_id = $('Limit').item.json.headers['message-id'] — but the field is read from the email header, not from the actual Message-ID envelope of the IMAP fetch. If the IMAP proxy / email provider rewrites the header, this might come back empty or differ from what emailMessageId exposes elsewhere.

---

2. The duplicate-order problem

What the data shows

500 most-recent auto-created orders queried on 2026-08-19 23:46 UTC:

• 423 / 500 (84.6%) are duplicates — i.e. the same email_message_id appears on 2 or more orders records
• 50 / 127 distinct message_ids are duplicated (the rest are singletons)
• The worst case: a single email <GV4PR05MB12159B25B35D2BD7FA50838D4D1A62@...> produced 138 duplicate orders — one every 5 minutes from 15:31 to 17:51 UTC on 2026-08-18, all with order_number = ZA02PO26002142
• Several emails have 9–10 duplicates created in the same second (e.g. MSP17933 at 19:10:27–19:10:28, MSP17934 at 19:15:31). The same-second pattern means multiple workflow executions are processing the same email concurrently — the IMAP poll's seen:false filter is not preventing parallel runs from picking it up

Why it's happening

Looking at the ORDER MONITOR (DWkxko0SS1uoelsg):

1. Schedule Trigger fires every 5 minutes ({"field":"minutes"} with no value, n8n defaults to 5min — confirmed by the 5-min spacing in execution log 56337 (23:45) -> 56320 (23:40) -> 56301 (23:35)).
2. CHECK FOR NEW MAIL (IMAP-enhanced node) — fetches every email with seen: false from INBOX. Include parts textContent, htmlContent, markdownContent, attachmentsInfo, flags, size, bodyStructure, headers.
3. SetEmailFlags (mark on fetch) — marks the fetched email as Seen immediately. This is the ONLY thing preventing the same email from being picked up again on the next 5-min tick.
4. Limit — parameters {}. Confirmed by inspecting the workflow JSON. In n8n the Limit node with no maxItems defaults to 1, but the "merge-2" + multi-attachment paths also feed back into it, so this limit is unreliable across branches.
5. Create orders — the n8n-nodes-pocketbase.pocketbaseHttp node does pb.collection('orders').create({...}) with no pre-check for an existing record with the same email_message_id.

Three things have to be true for duplicates to happen, and they all are:

• A. The IMAP poll keeps returning the same email. It happens because: the seen:false flag is set in the SAME workflow execution that processes the email — but if the workflow is retried (e.g. n8n re-triggers it after a transient error), or if the user reloads the IMAP mailbox and the IMAP server returns the same UID before the seen flag is committed, the next poll picks it up again. Same-second duplicates (9 in 19:10:27–19:10:28) mean the workflow is running in parallel for the same email.
• B. There is no server-side constraint preventing it. The PocketBase orders collection has no unique index on email_message_id and no application-level pre-check in createOrder (src/lib/server/orders-service.ts:1828). pb.collection('orders').create({...}) will happily insert a duplicate.
• C. The 5-min polling cadence multiplies the damage: every 5 minutes the same email gets reprocessed, and every cycle the AI extracts a slightly different contact_name (see §3) and a slightly different order_number if the AI misses the PO number on that particular call (the duplicate NO-REF records are the AI failing to extract the PO).

Mitigations That Already Exist (Reactive Only)

• POST /api/v1/orders/duplicates (src/app/api/v1/orders/duplicates/route.ts) — findDuplicateOrders() groups by order_number (within 60s) and by a line-aggregate heuristic. Returns the grouped list. The UI (src/components/duplicate-orders-dialog.tsx) lets the operator bulk-cancel.
• POST /api/v1/orders/{id}/cancel — soft-cancels an order (all items flip to cancelled, record stays in DB for audit).

These are post-creation tools. They do not prevent the duplicates from appearing in the first place.

Fixes (proposed)

A. Add a unique constraint on email_message_id in PB. This is the most robust fix — it makes the database reject duplicate inserts. Since the field is currently required: false, the migration is:

1. In the PB admin: index CREATE UNIQUE INDEX idx_orders_email_message_id ON orders (email_message_id) WHERE email_message_id != '' (or a partial unique index — Postgres/SQLite both support it).
2. Backfill: cancel the existing 423 duplicates (or use the existing cancelOrdersBulk op) so the unique constraint can be created.
3. In the ORDER MONITOR workflow, before Create orders, add a Search orders (email_message_id = X) step. If a record exists, short-circuit and move the email to INBOX.202608.Completed with a replay_skipped flag in the post-create log. This makes the workflow idempotent even if the IMAP poll does fire on the same email.

B. In the app, harden createOrder to refuse duplicates. With a unique constraint in place, the second create() will throw a 409 — wrap the pb.collection('orders').create(...) call in src/lib/server/orders-service.ts:1828 to catch the constraint violation and treat it as a "duplicate email, skipped" success. The function should return the existing record.

C. Tighten the IMAP poll. Replace the seen:false filter with an explicit since cursor: store the last-fetched UID in the workflow's staticData and only fetch UID > lastUID. This works even if the seen flag is reset by a reconnection.

D. Move the email ingestion off the 5-min poll. The poll is the only reason duplicates get 5-min apart. A webhook-based trigger (e.g. Postmark inbound webhook -> Next.js /api/v1/orders/intake) would eliminate the polling surface entirely. The CREATE QUOTE FROM EMAIL endpoint (POST /api/v1/quotes/from-email) already accepts the same payload, so the work is mostly plumbing.

---

3. The contact_name hit-and-miss problem

What the data shows

Same email <GV4PR05MB12159B25B35D2BD7FA50838D4D1A62@...> (ZA02PO26002142 from Nbenade@masterdrilling.com) produced these contact_name values across its 138 duplicate orders:

Nbenade\n
Nbenade\n
N. Benade\n
Nicole Benade\n
Nbenade\n
Not specified\n
Nbenade\n
Drilling Technical Services (PTY) Ltd\n
Nbenade\n
Nico Benade\n
N. Benade\n
Nbenade\n
Nbenade\n

Plus empty-string "" records (~8% of the same-email sample). Some records have literally "\n" as the contact name — the AI returned JSON like {"contact_name": "\n"} and it was stored verbatim.

The relevant emails are simple:

Good Day.

Please see attached copy of the order, please deliver soonest possible date.

Regards

There is no signature in the email body for the AI to extract from. The PO attachment is what contains the actual name (Neville Benadé), and the AI Agent prompt requires it to look at the attachments first — but it's not deterministic about what it picks out.

Why it's "hit and miss"

The contact_name pipeline:

1. AI Agent (DWkxko0SS1uoelsg "AI Agent" node) — Anthropic LLM call with hasOutputParser: true. The system prompt:
   • "Extract the actual End-User or Requester of the goods (Look for fields like 'Requester', 'Deliver To', 'Attention', or names in the Notes). Do NOT use the 'Buyer' or 'Created By' name if a Requester is listed."
   • There is no temperature override — Claude's default is 1.0.
   • The LLM is non-deterministic by design and the prompt does not enforce a JSON retry on invalid output.
2. Search customers (PB getList) — builds a filter from the LLM's contact_name:

   const nameKeywords = name
     .replace(/mr\.|mrs\.|ms\.|mr|mrs|ms/gi, '')
     .split(/[\s,]+/)
     .filter(w => w.length > 2);
   

   • replace(/mr\.|.../gi, '') only strips the literal prefix "Mr./Mrs./Ms." — it doesn't strip "Mr " (no dot) or "N." (no Mr.)
   • .filter(w => w.length > 2) drops keywords of length ≤ 2. So "N. Benade" -> ["Benade"], "Nbenade" -> ["Nbenade"], "Neville Benadé" -> ["Neville", "Benadé"].
   • The customer ~ 'X' substring filter then matches the customer collection. If the AI returned "Nbenade" and the customer is "Neville Benadé", the substring match succeeds because "Nbenade" is a substring of no one — but the search is fuzzy enough to return multiple matches (the elementsPerPage: 12 returns up to 12).
3. Create orders — customer = $('Search customers').item.json?.id || 'xmd68ps32k1x5fu' — the first match wins. If the AI returned names that don't match any customer, the fallback ID is the placeholder xmd68ps32k1x5fu (which is "Unknown - needs review" in the customers table).
4. Page rendering — src/app/(app)/orders/page.tsx:942:

   {order.contact_name && (
     <span>{order.contact_name}</span>
   )}
   

   Empty string "" is falsy -> row shows no name. Newline "\n" is truthy -> row shows just a separator dot with nothing after it. The operator sees "GARY-JV-MR · Master Drilling · ·" with a dangling separator and no name — clearly "miss".

Fixes (proposed)

A. Stop using the LLM for the contact name. The email body has a signature; the PDF has a "Requester:" line. Both are deterministic. Add a pre-LLM regex pass:

1. Parse the email body for the signature (everything after the last Regards / Kind regards / Best / Thanks block, or the last 5 non-empty lines).
2. Parse the PDF text for Requester:, Deliver To:, Attention:, End-User:. The first one is the contact name.
3. Only call the LLM if both fail.

If a deterministic parse succeeds, the LLM is skipped entirely — the hit/miss disappears.

B. If you keep the LLM, constrain it. Add to the system prompt:

• "If you cannot find a Requester/Deliver-To/Attention name, return the empty string — do NOT return placeholder text like 'Unknown', 'Not specified', or the company name."
• Emit contact_name = "" (not "\n") when uncertain. The page's truthy-check will hide it cleanly.
• Set temperature: 0 on the Anthropic node. The LLM is still non-deterministic for the same prompt + temperature, but the variance drops substantially.

C. Post-process the LLM output. Wrap the AI Agent call in a Code node that:

• trims trailing newlines (s.replace(/\s+$/,''))
• validates the result against the customers collection (search by email first, then by name) — if a customer row exists with the same email, use customer.customer as the contact name regardless of what the LLM said.

D. Move the display sanitisation into the renderer. Even if the LLM keeps hallucinating newlines and trailing whitespace, the row should not show a dangling separator. Patch order.contact_name to a JSX-safe display:

const displayContact = (order.contact_name || "").trim();
{displayContact && <span>· {displayContact}</span>}

This is a one-line fix and removes the visible "miss" symptom even if the data is still messy.

---

4. The "some emails fail" question

Looking at the live data, two failure modes are visible:

4a. AI returns empty contact_name / order_items

The If1 / If1 (Multi) branches check $json.output.order_items.length and route:

• <= 0 -> "no items extracted" branch -> Create order (no items extracted - raw) / Create order (single - no items - raw) / Create order (no PO detected - raw). The resulting order has:
  • description = "[NEEDS REVIEW] AUTO-FLAGGED: AI Agent could not extract any line items for this PO. Raw extracted text: ..."
  • the raw extracted text dumped in the description
  • the email still moved to INBOX.202608.Completed

This is the right fallback — the operator sees the flag. But the customer search may still be wrong because the LLM returned garbage for the contact name (see §3). The order lands on the board with a flag, but the customer/contact fields may be incorrect.

4b. AI returns "Unknown" / "Not specified" / company name as contact

Same root cause as §3. The LLM hallucinates an answer instead of returning empty. The Search customers query then searches for "Unknown" or "Not specified" or "Drilling Technical Services" — and finds the company record instead of the customer record. The order is created with the company as the customer, which is the wrong relation.

4c. The IMAP poll loses emails

When the IMAP mailbox is polled, the seen:false filter is set in the SAME workflow execution as the processing. If the workflow crashes (retry, timeout, network error, PocketBase 500), the email is still marked as seen but never processed. The next 5-min poll won't pick it up. This is silent failure — the email is in limbo until the operator manually checks the inbox.

There is no onError feedback to the ORDER MONITOR. The "Error Handler" n8n workflow (kVylcetxy858PcfP) only catches errors from workflows that have the error workflow set. ORDER MONITOR does not.

Fixes (proposed)

A. Make the email flag roll back on error. Wrap the Create orders

• Create order_items1 chain in a Rollback node: if either fails, set Seen = false on the email so the next poll picks it up.

B. Mark the workflow as error-to-Workflow. In n8n workflow Settings, set errorWorkflow: kVylcetxy858PcfP so the Telegram error notifier fires on any failure.

C. Add a sender-domain blacklist to the deterministic classifier. The classifier already has supplierDomains = ['festo.com'] but that's incomplete — most of the duplicates come from masterdrilling.com, mreee.co.za, harmony.co.za. Add a customer domain list (the inverse of supplier). The classifier already routes "supplier" -> ignored. Mirror it with "customer" -> process, but make sure the operator can override the list.

D. Add a fallback "create order from raw email" path. If the AI Agent fails to extract structured data after N retries, the workflow should still create an order with the raw email body as the description and a flag. This is already the "no items extracted" branch — but verify the branch is reachable for ALL routes (not just the customer branch).

---

5. Recommended fixes, ranked by impact / effort

#	Fix	Impact	Effort
1	Add UNIQUE index on orders.email_message_id in PB, backfill the 423 existing duplicates via the bulk-cancel tool, and pre-check in createOrder	Stops the duplicate flood at the DB level	2 hours (admin UI + service-layer try/catch)
2	Add a "Search orders by email_message_id" pre-check in the ORDER MONITOR workflow before Create orders	Stops duplicates even if the IMAP poll is unreliable	30 min in n8n
3	Deterministic contact_name extraction (regex signature + PO requester line) before the LLM call	Eliminates the "hit and miss" name pop	1 day (depends on how many PO formats need handling)
4	Constrain the LLM: temperature 0, "return empty string when uncertain" rule	Reduces LLM hallucination	15 min
5	Renderer sanitisation: trim contact_name and only render when non-empty	Hides the visible "miss" symptom with no data loss	5 min
6	Replace IMAP poll with a webhook intake (Postmark inbound -> /api/v1/orders/intake)	Eliminates the 5-min polling surface entirely	1 week (depends on the email provider)
7	Wire the ORDER MONITOR to the error workflow	Surfaces failures on Telegram instead of silently losing emails	15 min
8	Roll back Seen flag on workflow error	Recovers emails that fail to process	30 min in n8n

If we only do one thing: #1 + #2 together. They stop the bleeding. If we do two more: #3 + #5 clean up the visible UX. #6 is the long-term fix that fixes all polling-class bugs at once.

---

6. Files / artefacts that were touched or referenced

• n8n workflows (live, active): DWkxko0SS1uoelsg, O0yDcNVHEhA0Dd5n, 45h7IYegCmhtkV9t, mXX2yu5svRurnfwO, UqO3ayETx9cQHOzF, GGS5zgVbdfsRyLcY, kVylcetxy858PcfP, aI0Ze2vwgp25ZgGH, DDFN9NBmh95yG2Jk
• App sources: src/app/(app)/orders/page.tsx, src/components/duplicate-orders-dialog.tsx, src/app/api/v1/orders/duplicates/route.ts, src/app/api/v1/orders/create/route.ts, src/lib/server/orders-service.ts (createOrder, findDuplicateOrders)
• PocketBase collections: orders (pbc_3527180448), customers (pbc_108570809), companies (pbc_3866053794)
• Docs: docs/order-workflow-rules.md, docs/AGENT_INSTRUCTIONS.md

Query commands used (reproducible)

# List all orders, look at duplicates
curl -H "Authorization: Bearer $AGENT_API_KEY" \
  "https://wfdash.apphub.co.za/api/v1/orders?perPage=500&sort=-created&fields=id,order_number,contact_name,email_message_id,email_sender,created,email_subject"

# Get a single n8n workflow's JSON
curl -H "X-N8N-API-KEY: $N8N_API_KEY" \
  "http://n8n-...sslip.io/api/v1/workflows/DWkxko0SS1uoelsg"

# Get a single execution's data
curl -H "X-N8N-API-KEY: $N8N_API_KEY" \
  "http://n8n-...sslip.io/api/v1/executions/56337?includeData=true"