MEGAINFRA-WEBAPP/docs/n8n/po-email-intake.md

162 lines
7.7 KiB
Markdown
Raw Normal View History

# PO email intake — ai@megainfra.co.za → MEGA
**Status: built and tested. INACTIVE — waiting on two things from your IT admin (steps 1–2 below).**
Workflow: [MEGA - PO Email Intake (Microsoft Graph)](https://wfdash.n8n.apphub.co.za/workflow/bQ5PioGTSKnsWVh0)
Source: [`po-email-intake.workflow.js`](./po-email-intake.workflow.js) (rebuildable from this file)
---
## What it does
Every 5 minutes the workflow:
1. lists the inbox of `ai@megainfra.co.za` through Microsoft Graph,
2. finds emails carrying a purchase-order document (PDF / JPEG / PNG / WEBP),
3. works out which MEGA site the PO belongs to,
4. files the document into the **same PocketBase queue the browser upload button uses** — a
`purchase_orders` row with `processing_status: pending`, the file attached, `file_hash` set,
and a unique `PENDING-…` placeholder number,
5. archives the source email once the row is filed.
Step 4 is the important part: the existing **MEGA - Purchase Order Ingest** workflow
(`005iI4jX5mq9DsTR`) already polls that queue, so OCR, supplier matching, line-item extraction
and finalisation all happen in the pipeline that is already running and trusted. **No changes to
the dashboard app were needed** — the email path is just another way into the same door.
If a PO cannot be matched to a site, the email is tagged `PO Needs Review` in Outlook and left in
the inbox. Nothing is filed against the wrong site and nothing is silently dropped.
## Why this is not an IMAP workflow
`megainfra.co.za` is a **Microsoft 365 / Exchange Online** custom domain
(`autodiscover.megainfra.co.za → autodiscover.outlook.com`, SPF includes
`spf.protection.outlook.com`). The tenant has password-based mail protocols switched off:
| Endpoint | Result |
| --- | --- |
| `outlook.office365.com:993` (IMAP) | `Basic authentication is disabled.` |
| `smtp.office365.com:587` (SMTP) | `535 5.7.139 … SmtpClientAuthentication is disabled for the Tenant.` |
So no password-based mail client, n8n IMAP node or script can read this mailbox, whatever you
type into a sign-in form. Reading it needs OAuth2 against Microsoft Graph. There is also no
fallback IMAP route: `mail.megainfra.co.za` does not resolve, and HostH's
`securemail.synaq.com` mail ports are unreachable from the app servers.
---
## Step 1 — your IT admin creates the app registration (once)
In the Azure portal → **App registrations** → **New registration**:
- Name: `MEGA PO Email Intake`
- Supported account types: **Accounts in this organizational directory only**
- Redirect URI: leave blank (this is a client-credentials app, no sign-in redirect)
Then:
- **Certificates & secrets** → **New client secret** → copy the secret value (shown once).
- **API permissions** → **Add a permission** → **Microsoft Graph** → **Application permissions** →
add **Mail.ReadWrite** → **Grant admin consent**.
- `Mail.Read` would only allow reading. We also move and tag messages, so **`Mail.ReadWrite`
(application)** is the one to grant.
- Copy **Tenant ID**, **Application (client) ID** and the **client secret value**.
Send those three values to the person wiring up n8n. The account password is not needed and is
not stored anywhere.
## Step 2 — create the n8n credential (once)
In n8n → **Credentials** → **New** → **OAuth2 API**:
| Field | Value |
| --- | --- |
| Name | `Microsoft Graph (MEGA PO)` |
| Grant Type | **Client Credentials** |
| Token URL | `https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token` |
| Client ID | Application (client) ID from the app registration |
| Client Secret | the client secret value |
| Scopes | `https://graph.microsoft.com/.default` |
| Auth Style | Header → `Bearer` |
## Step 3 — bind the credential and switch it on
The six Microsoft Graph nodes still need the credential selected:
`List Inbox POs` · `Download Attachment` · `Archive Message` ·
`Archive Message (already filed)` · `Flag Needs Review` ·
`Flag Needs Review (no site)`
The three PocketBase nodes (`Read Sites`, `Duplicate Check`, `Create PO Row`) are already bound
to the existing **`MEGAINFRASTRUCTURE`** credential — no secret sits inline in the workflow.
Once the credential is selected on those six nodes, activate the workflow (the schedule trigger
starts polling every 5 minutes).
---
## How a PO is matched to a site
This is the one design decision worth understanding.
MEGA's finalisation step deliberately **never re-maps the site after OCR** —
`src/app/api/purchase-orders/process-batch/route.ts` states it outright: *"`site` was set at
upload by the person who uploaded the … and is NEVER touched here"*, because OCR only matches a
real project code on roughly 8% of documents. The old
`src/app/api/purchase-orders/ingest/route.ts`, which used to match the site from
`project_code` server-side, is deprecated (see
`docs/superpowers/plans/2026-09-11-ingestion-restructure.md`).
So the site has to be known when the row is created, and the workflow reads it out of the email
itself, in this order:
1. exact project code — `C26/09`, `C26/13`, `C26/14`, `001`, `HEAD`
2. project code appearing anywhere in subject + body preview + attachment filename
3. site name — *The Hills*, *Cedar Gate*, *Munyaka 2*, *Nellmapius Ext 25*, *Mega Head Office*
If none of those match, the email is tagged `PO Needs Review` and left in the inbox for a human.
It is never filed against a guessed site.
> If you would rather have the OCR decide the site, that needs a change to the
> `process-batch` route (or to the queue) so the site can be corrected after extraction. That is
> a small change to the app, not to this workflow — say the word and it can be done.
## Duplicate protection
Before filing, the workflow looks for an existing `purchase_orders` row with the **same file hash
at the same site** (same rule the browser upload uses). A re-send of the same document is
archived rather than filed twice. The placeholder `purchase_order_number` is derived from the
file hash, so PocketBase's unique index is a second line of defence.
## What it deliberately does not do
- Only the inbox is read — not other folders. 25 most recent messages per run.
- Only PDF / JPEG / PNG / WEBP attachments are considered. Everything else is ignored.
- Messages already tagged `PO Needs Review` or `PO Ingested` are skipped on later runs, so a
flagged email is not re-processed every 5 minutes.
- Non-PO attachments that merely look PO-ish are flagged for review, not filed — MEGA's own
document-kind check then refuses anything that turns out to be an invoice.
## Timing
| Step | When |
| --- | --- |
| Email picked up | within 5 minutes of arriving |
| OCR + finalisation | the existing MEGA workflow wakes every 3 minutes and claims one pending row per run, so a PO reaches the dashboard roughly 3–8 minutes after the email lands |
| Archived email | as soon as the row is filed |
## Rollback
Deactivate the workflow (n8n → the workflow → **Deactivate**). Nothing else changes and the
dashboard keeps working. Rows already filed by email are ordinary `purchase_orders` rows and
behave exactly like browser uploads.
## Troubleshooting
| Symptom | Where to look |
| --- | --- |
| No emails picked up | n8n execution log for `List Inbox POs` — a 401/403 means the Graph credential is wrong or admin consent was not granted |
| POs tagged `PO Needs Review` | the email did not name a site; add the site/project code to the subject or check the site table in `Read Sites` |
| Row created but never finalised | the existing `MEGA - Purchase Order Ingest` workflow — check its `Fetch Pending Batches` / `Escalate Failure` nodes |
| Same PO filed twice | `Duplicate Check` compares `file_hash` + `site`; a different site for the same document is deliberately not treated as a duplicate |