MEGAINFRA-WEBAPP/docs/n8n/purchase-order-ingest.md

# n8n Workflow — Purchase Order Ingest

This is the handoff doc for the n8n workflow that processes Purchase Order uploads from the MEGA Infrastructure dashboard. The Next.js side is implemented by Claude; **you build this workflow in n8n using your MCP access**.

## What this workflow does

1. Receives a PO file (PDF or image) via webhook from `/api/purchase-orders/upload`.
2. Extracts text via OCR (Tesseract for images, `pdf-parse` for PDFs).
3. Parses PO header fields and line items.
4. Calls back to Next.js with a structured JSON payload + the original file (base64-encoded).
5. Handles errors per the retry policy below.

## Webhook trigger

- **Method:** POST
- **Path:** `purchase-order-upload` (the full URL is `N8N_PO_INGEST_URL` set in your `.env.local`)
- **Content-Type:** `multipart/form-data`
- **Body fields:**
  - `file` — the PO file (PDF / JPG / PNG / WEBP, ≤20 MB)
  - `filename` — original filename
  - `contentType` — MIME type
  - `size` — file size in bytes

## OCR step

Branch on `contentType`:

- `image/jpeg`, `image/png`, `image/webp` → **Tesseract OCR** (n8n has a Tesseract node, or use an HTTP node to call a Tesseract microservice).
- `application/pdf` → **`pdf-parse`** (npm package, available via n8n Function node). Falls back to Tesseract on each rendered page if `pdf-parse` returns empty text.

Output: a single string `ocrText`.

## Parser step

Extract from `ocrText` using regex or an LLM node (recommend LLM for robustness — gpt-4o-mini or your preferred model).

**Header fields:**

| Field | Type | Notes |
|---|---|---|
| `po_number` | string | Required. Unique. Pattern varies — usually "PO-1234" or just digits. |
| `supplier` | string | Required. Supplier company name. |
| `project_code` | string | Required. Match against the `sites.project_code` collection in PocketBase. |
| `required_by_date` | ISO date | Optional. `YYYY-MM-DD`. |
| `ordered_by` | string | Optional. Person who placed the order. |

**Line items array** (`items[]`):

| Field | Type | Notes |
|---|---|---|
| `description` | string | Required. Item description. |
| `qty` | number | Required. > 0. |
| `rate` | number | Required. ≥ 0. Unit price. |
| `unit` | string | Optional. e.g. "ea", "m", "m²". |
| `material_code` | string | Optional. |
| `cost_code` | string | Optional. |
| `disc_percent` | number | Optional. 0–100. |
| `vat_percent` | number | Optional. 0–100. |

Drop any items missing `description`, `qty`, or `rate`. If the entire `items[]` is empty, fail the workflow — don't send an empty PO.

## Site lookup (optional)

You can optionally look up the site in PocketBase before calling back:

```
GET ${POCKETBASE_URL}/api/collections/sites/records?filter=project_code="${project_code}"&perPage=5
```

But this is **not required** — the Next.js `/api/purchase-orders/ingest` route does the same lookup server-side and will ignore your result. Only do this if you want to enrich the `raw_ocr` payload with site metadata for debugging.

## Callback

```
POST ${APP_URL}/api/purchase-orders/ingest
Headers:
  Content-Type: application/json
  x-ingest-secret: ${N8N_INGEST_SECRET}
Body: PurchaseOrderIngestPayload (see below)
```

### Payload shape

```typescript
{
  po_number: string,
  project_code: string,
  supplier: string,
  required_by_date?: string,        // ISO "YYYY-MM-DD"
  ordered_by?: string,
  items: [
    {
      description: string,
      qty: number,
      rate: number,
      unit?: string,
      material_code?: string,
      cost_code?: string,
      disc_percent?: number,
      vat_percent?: number
    }
  ],
  file: {
    filename: string,
    contentType: "application/pdf" | "image/jpeg" | "image/png" | "image/webp",
    base64: string                  // raw base64, NO "data:" prefix
  },
  raw_ocr: { /* anything useful for debugging */ }
}
```

### File transport

n8n reads the file from the incoming webhook, then re-encodes to base64 in the Function node:

```javascript
const fileBuffer = items[0].binary.file.data; // n8n binary buffer
const base64 = Buffer.from(fileBuffer).toString('base64');

return {
  json: {
    ...parsedData,
    file: {
      filename: items[0].binary.file.fileName,
      contentType: items[0].binary.file.mimeType,
      base64
    }
  }
};
```

## Response handling

Branch on the HTTP status code:

| Status | Meaning | Action |
|---|---|---|
| `200` | Success | Log `id` and `po_number`. Mark workflow as successful. |
| `400` | Malformed payload | Log `error` + `details`. **Do not retry** — configuration error. |
| `401` | Bad secret | Log "secret mismatch — check N8N_INGEST_SECRET env". **Do not retry**. |
| `409` | Duplicate `po_number` | Log "duplicate PO — already in system". **Do not retry**. |
| `500` | Server error | Log `error` + `step`. **Retry per policy below**. |
| `408` / network timeout | Transient | **Retry per policy below**. |

## Retry policy

For `500` and `408`/timeout only:

- Retry up to **3 times** with exponential backoff: 10s, 30s, 90s.
- After 3 failed retries, send to an "Error" workflow branch (dead letter) that logs the failure to your monitoring channel (Slack, email, etc.).

Use n8n's built-in **Retry on Fail** node setting with `maxTries: 3` and a custom `waitBetweenTries` expression, or chain three HTTP nodes with `Wait` nodes between them.

## Environment variables to set in n8n

| Variable | Value | Notes |
|---|---|---|
| `APP_URL` | `https://mega.dash.apphub.co.za` | Base URL of the Next.js dashboard. |
| `N8N_INGEST_SECRET` | (same as in `.env.local`) | Shared secret header. |
| `POCKETBASE_URL` | `https://pb.mega.infra.apphub.co.za` | Only needed if you do the optional site lookup. |

## Testing the workflow

Before wiring up the dashboard button, test the webhook directly:

```bash
curl -X POST "$N8N_PO_INGEST_URL" \
  -F "file=@/path/to/test-po.pdf" \
  -F "filename=test-po.pdf" \
  -F "contentType=application/pdf" \
  -F "size=$(stat -f%z /path/to/test-po.pdf)"
```

Watch the n8n execution log. Confirm:
- The webhook receives the file.
- OCR runs (check OCR node output).
- The parsed payload has all required fields populated.
- The callback POST to `/api/purchase-orders/ingest` returns `200` with `id` and `po_number`.
- The new PO appears in the PocketBase `purchase_orders` collection.

Test cases:

1. **Clean PDF** — well-structured PO, all fields present. Expect 200.
2. **Scanned image (JPG)** — Tesseract OCR. Expect 200 (or 400 if OCR is too garbled to parse).
3. **PDF with no extractable text** — OCR fallback should kick in.
4. **Duplicate PO** — upload the same PO twice. First call 200, second call 409.
5. **Unknown project_code** — expect 200 with `warnings: ["site_unmatched"]`.

## Future: email ingestion

The same `/api/purchase-orders/ingest` callback will be reused when email ingestion is built. Add a second n8n workflow with an **Email Trigger** (IMAP / Gmail / Outlook node) that:
1. Downloads the PO attachment from the email.
2. Runs the same OCR + parser + callback flow.

No changes needed on the Next.js side.