MEGAINFRA-WEBAPP/docs/n8n/po-email-intake.md
DEls 844918da0d feat(po-email): pre-flight duplicate check and filed labels
Checks for a duplicate before filing anything: the same document at the same site is
archived, and a PO number already in the system is tagged PO Duplicate for a human
instead of being filed twice. Successfully filed mail is tagged PO Ingested and moved
out of the inbox; PO Needs Review and PO Duplicate stay in the inbox for review and are
skipped on later runs.
2026-10-28 23:17:52 +02:00

12 KiB
Raw Blame History

PO email intake — ai@megainfra.co.za → MEGA

Status: built and tested. INACTIVE — waiting on one thing from you (step 1 below).

Workflow: MEGA - PO Email Intake (Microsoft Outlook) Source: po-email-intake.workflow.js (rebuildable from this file)


What it does

Every 5 minutes the workflow:

  1. lists the inbox of ai@megainfra.co.za through n8n's native Microsoft Outlook node,
  2. finds emails carrying a purchase-order document (PDF / JPEG / PNG / WEBP),
  3. works out which MEGA site the PO belongs to,
  4. files the document into the same PocketBase queue the browser upload button uses — a purchase_orders row with processing_status: pending, the file attached, file_hash set, and a unique PENDING-… placeholder number,
  5. archives the source email once the row is filed.

Step 4 is the important part: the existing MEGA - Purchase Order Ingest workflow (005iI4jX5mq9DsTR) already polls that queue, so OCR, supplier matching, line-item extraction and finalisation all happen in the pipeline that is already running and trusted. No changes to the dashboard app were needed — the email path is just another way into the same door.

If a PO cannot be matched to a site, the email is tagged PO Needs Review in Outlook and left in the inbox. Nothing is filed against the wrong site and nothing is silently dropped.

Why this is not an IMAP workflow

megainfra.co.za is a Microsoft 365 / Exchange Online custom domain (autodiscover.megainfra.co.za → autodiscover.outlook.com, SPF includes spf.protection.outlook.com). The tenant has password-based mail protocols switched off:

Endpoint Result
outlook.office365.com:993 (IMAP) Basic authentication is disabled.
smtp.office365.com:587 (SMTP) 535 5.7.139 … SmtpClientAuthentication is disabled for the Tenant.

So no password-based mail client, n8n IMAP node or script can read this mailbox, whatever you type into a sign-in form. Reading it needs OAuth2 against Microsoft Graph. There is also no fallback IMAP route: mail.megainfra.co.za does not resolve, and HostH's securemail.synaq.com mail ports are unreachable from the app servers.


Reading the mailbox from the Mac instead (Mail.app)

ai@megainfra.co.za is also set up in Mail.app on this Mac, and Mail can be read without touching Full Disk Access, through Mail's own AppleScript interface:

# accounts Mail knows about
osascript -e 'tell application "Mail" to get name of every account'

# raw MIME of a message (attachments come back base64-encoded inside it)
osascript -e 'tell application "Mail" to get source of message 1 of mailbox "Inbox" of account "ai@megainfra.co.za"' > msg1.eml

Then parse msg1.eml with any MIME parser to pull out the PDFs (see po-email-intake.workflow.js for the same attachment filter the workflow uses).

Two things to know:

  • macOS blocks direct reads of Mail's private store (~/Library/Mail and the com.apple.mail container) from a sandboxed shell — you get Operation not permitted. AppleScript works because Mail itself hands the data over; it asks once for Automation permission, which is far less than Full Disk Access.
  • This route is Mac-local. It only works while the Mac is on and Mail is signed in, so it is good for a one-off backfill or for checking what is arriving — not for unattended intake. The n8n workflow below is the route that runs on its own.

Step 1 — connect the mailbox in n8n (once, about a minute)

In n8n → Credentials → New → search for Microsoft Outlook → Connect.

n8n opens a Microsoft sign-in page. Sign in as ai@megainfra.co.za and approve the mail permissions it asks for (read, move, tag). n8n stores the tokens and refreshes them for you — there is no password, no client secret and no Azure app registration to create. Name the credential Microsoft Outlook (MEGA PO) so it lines up with the workflow.

If your browser is already signed in to that account, it is one click. If the account is already open in your mail app, the same Microsoft session usually carries over — which is exactly why this route works without an IT admin.

Step 2 — bind the credential (once)

Select Microsoft Outlook (MEGA PO) on these nine nodes:

List Inbox POs · List Attachments · Download Attachment · Archive Message · Archive Message (already filed) · Label PO Filed · Flag Needs Review · Flag Needs Review (no site) · Flag Needs Review (number in use)

(The three PocketBase nodes — Read Sites, Duplicate Check, Create PO Row — are already bound to the existing MEGAINFRASTRUCTURE credential. No secret sits inline in the workflow.)

Step 3 — switch it on

Activate the workflow. The schedule trigger starts polling every 5 minutes.


Upgrading to an unattended service principal (optional, later)

The delegated sign-in above runs as a person. That is fine for getting started, but for a production flow you may prefer an app that needs no human at all. n8n's Outlook node supports three authentication modes, and the swap is a setting change on the same six nodes:

Mode Credential Use when
microsoftOutlookOAuth2Api Microsoft Outlook (hosted OAuth) default here — one sign-in, no admin
microsoftOAuth2Api Microsoft OAuth2 API (client credentials) you want an app registration and a client secret
microsoftEntraServicePrincipalApi Entra service principal you want app-only access, with the mailbox field set to ai@megainfra.co.za

For the service-principal route your IT admin creates an Azure app registration with Mail.ReadWrite (application) + admin consent, and hands over Tenant ID, Application (client) ID and the client secret value. Then change the authentication dropdown on the six nodes and set the mailbox on each. No rebuild is needed — the workflow logic is unchanged.


How a PO is matched to a site

This is the one design decision worth understanding.

MEGA's finalisation step deliberately never re-maps the site after OCR — src/app/api/purchase-orders/process-batch/route.ts states it outright: "site was set at upload by the person who uploaded the … and is NEVER touched here", because OCR only matches a real project code on roughly 8% of documents. The old src/app/api/purchase-orders/ingest/route.ts, which used to match the site from project_code server-side, is deprecated (see docs/superpowers/plans/2026-09-11-ingestion-restructure.md).

So the site has to be known when the row is created, and the workflow reads it out of the email itself, in this order:

  1. exact project code — C26/09, C26/13, C26/14, 001, HEAD
  2. project code appearing anywhere in subject + body preview + sender
  3. site name — The Hills, Cedar Gate, Munyaka 2, Nellmapius Ext 25, Mega Head Office

If none of those match, the email is tagged PO Needs Review and left in the inbox for a human. It is never filed against a guessed site.

Known limitation, seen in real mail: the site often appears only inside the PDF, not in the email. A real example from the mailbox — the subject was "RE: Bushveld Joy x 12 Traffic Light Quote." with no site anywhere in the subject, body or filename, while the PDF itself said "MEGCREQ11911 The Hills Ext12" and the PO was for The Hills. That email gets flagged for review rather than filed. The other three POs that week named their site in the subject ("Nellmapius Ext 25 PO Tools/cement/Bricks", "Cedar Gate Cement PO") and matched fine.

If you would rather have the OCR decide the site, that needs a change to the process-batch route (or to the queue) so the site can be corrected after extraction. That is a small change to the app, not to this workflow — say the word and it can be done. It would also close the gap above, because the PO's own Subject line (MEGCREQ11913 (Nellmapius EXT25)) is far more reliable than the email text.

Duplicate protection — checked before anything is filed

The workflow never creates a row without checking first:

Check How What happens
Same document, same site file_hash + site on an existing row Already filed → the email is archived, nothing is filed twice
PO number already in the system the number is read from the attachment filename or the subject (e.g. MEGCPO9524.pdf) and looked up in purchase_orders Tagged PO Duplicate and left in the inbox for a human — a differing file can be a legitimate revision of that PO, and the same number at another site is not proof of a duplicate

The app checks both again after OCR, in process-batch: a same-file match is deleted as a duplicate, and a number-only match is marked po_number_conflict with number_conflict_candidate and left for review. So a number can never quietly end up booked twice — the pre-check catches it before a row exists, and the post-check catches it if the number only becomes readable after OCR.

The placeholder purchase_order_number (PENDING-…, derived from the file hash) is also protected by PocketBase's unique index, so two identical uploads cannot both create a row.

What happens to the email

Outcome Label Where the mail ends up
Filed successfully PO Ingested moved out of the inbox (Archive)
Same document already filed — moved out of the inbox (Archive)
PO number already in the system PO Duplicate stays in the inbox, for review
No site could be read from the email PO Needs Review stays in the inbox, for review
Filing failed PO Needs Review stays in the inbox, for review

Messages carrying PO Ingested, PO Duplicate or PO Needs Review are skipped on later runs, so a flagged email is never re-processed every 5 minutes. In Outlook, filter the inbox by category to see everything that still needs a human.

What it deliberately does not do

  • Only the inbox is read — not other folders. 25 most recent messages per run.
  • Only PDF / JPEG / PNG / WEBP attachments are considered. Everything else is ignored.
  • Non-PO attachments that merely look PO-ish are flagged for review, not filed — MEGA's own document-kind check then refuses anything that turns out to be an invoice.

Timing

Step When
Email picked up within 5 minutes of arriving
OCR + finalisation the existing MEGA workflow wakes every 3 minutes and claims one pending row per run, so a PO reaches the dashboard roughly 3–8 minutes after the email lands
Archived email as soon as the row is filed

Rollback

Deactivate the workflow (n8n → the workflow → Deactivate). Nothing else changes and the dashboard keeps working. Rows already filed by email are ordinary purchase_orders rows and behave exactly like browser uploads.

Troubleshooting

Symptom Where to look
No emails picked up n8n execution log for List Inbox POs — a 401/403 means the credential is not connected or the consent was refused
POs tagged PO Needs Review the email did not name a site; add the site/project code to the subject or check the site table in Read Sites
Row created but never finalised the existing MEGA - Purchase Order Ingest workflow — check its Fetch Pending Batches / Escalate Failure nodes
Same PO filed twice Duplicate Check compares file_hash + site, and the PO number against the system. A number that is already used is tagged PO Duplicate rather than filed
Mail not moved out of the inbox check the Archive Message / Label PO Filed nodes in the execution log — a failed Graph call leaves the mail in the inbox and it will be retried