MEGAINFRA-WEBAPP/docs/architecture.md

# Architecture

> MEGA INFRASTRUCTURE — a Next.js 16 operations dashboard for a South African
> civil-construction company, backed by PocketBase, with n8n handling AI-powered
> document ingestion.

This document is a map of how the system fits together: where data lives, who
reads it, who writes it, and what the trust boundaries are. For *what* the
collections look like, see [`data-model.md`](./data-model.md). For the
end-to-end ingestion pipelines, see [`data-flows.md`](./data-flows.md).

---

## 1. Runtime topology

```mermaid
flowchart LR
    Browser["Browser (PWA)"]
    Next["Next.js 16\nApp Router · React 19 · Tailwind v4"]
    PB["PocketBase\n(shared via Traefik / TLS)"]
    n8n["n8n workflows\n(PO ingest · DN OCR)"]
    Anthropic["Anthropic / MiniMax LLM"]
    MCP["Local MCP server\n(stdio · read-only)"]

    Browser -- "HTTPS\n(cookie-less, PIN-only)" --> Next
    Browser -- "Service worker\n(public/sw.js)" --> Browser
    Next -- "server components\n& route handlers\nvia pbAdmin()" --> PB
    Next -- "POST /api/chat\nSSE stream" --> Anthropic
    Next -- "POST /api/purchase-orders/upload\nqueued batch row" --> PB
    n8n -- "download file via /api/delivery-notes/:id/slip\nx-ingest-secret" --> Next
    n8n -- "POST /api/delivery-notes/:id/ocr-result\nx-ingest-secret" --> Next
    n8n -- "POST /api/purchase-orders/process-batch\nx-ingest-secret" --> Next
    Anthropic -. "tools/calls" .-> Next
    MCP -- "stdio\n(env-based admin creds)" --> Next
```

Three things matter about this picture:

1. **There is exactly one database.** PocketBase holds every collection
   (sites, employees, vehicles, fuel logs, plant hours, POs, delivery notes,
   income claims, fixed costs, data import batches). The Next.js app and the
   MCP server both read/write it directly; there is no intermediate service.
2. **PocketBase itself does the file storage** for purchase-order PDFs and
   delivery-note slip images. Routes stream those bytes in and out via the
   PB admin client.
3. **The browser never talks to PocketBase directly in production.** Every PB
   call from a server component or API route goes through `pbAdmin()` (see
   §3). The browser-side `pb` client in `src/lib/pb.ts:36` exists for
   completeness but no dashboard page uses it today.

---

## 2. The two PocketBase clients

`src/lib/pb.ts` is the single source of truth for all PocketBase access. It
exports two clients and you must use the right one for the job.

| Client | Where it runs | Auth | Used by |
| --- | --- | --- | --- |
| `pb` (`src/lib/pb.ts:36`) | Browser | Persisted JWT in `localStorage` (key `mega-pb-auth`) | Currently unused — kept for future client-side use |
| `pbAdmin()` (`src/lib/pb.ts:74`) | Server only | Superuser (email + password **or** preset token), cached in module scope | Every server component, every API route, the MCP server |

**Why this matters.** `pbAdmin()` authenticates as the PocketBase
**superuser**. That means the application's backend has read/write access to
every collection, irrespective of the per-collection API rules in
`scripts/pb-relax-rules.py`. The relaxed rules there are a defence-in-depth
measure that only constrains *direct* browser-to-PB access — which we do not
do. See [operations.md §3](./operations.md#3-pocketbase-api-rules) for the
exact rule set.

**Auto-cancellation off on `pbAdmin()`.** The SDK cancels in-flight requests
when a new request shares the same `(method, URL-path)` key. On the server,
the module-level client cache means every concurrent route handler shares
**one** PocketBase client, so the second concurrent request to the same
endpoint would abort the first and produce cascading `AbortError` 500s.
The code disables auto-cancel via a typed double-cast
(`src/lib/pb.ts:102`) — see the long comment at `src/lib/pb.ts:88-101` for
the full story.

The dashboard pages that fan out parallel reads also pass
`$autoCancel: false` explicitly on their own calls (e.g.
`src/app/dashboard/page.tsx:71` for the plant count, `src/app/dashboard/purchasing/page.tsx:22-25` for the needs-review DNS lookup). This is the
defensive belt-and-braces for the same problem.

---

## 3. Rendering model

Every dashboard page is a React server component (`export const dynamic = "force-dynamic"` is set on every page). Data fetching happens in
the page body before render; the only client work is the sidebar
(`src/components/layout/sidebar.tsx`, `"use client"`) and the few interactive
forms (chat, import buttons, inline edit).

| Surface | Rendering | Data fetching |
| --- | --- | --- |
| `/login` | Static + client hydration | None |
| `/dashboard` (Overview) | Server component | Parallel `getList` + `getFullList` via `Promise.all` in `loadOverview()` |
| `/dashboard/sites` | Server component | Seven parallel `getFullList` calls in `src/app/dashboard/sites/page.tsx:24-50` |
| `/dashboard/sites/[id]` | Server component | Per-site `getFullList` filters for fixed costs, POs, fuel logs, vehicles, income; **timesheets + employees fetched unfiltered** |
| `/dashboard/fuel` | Server component | `getFullList` of vehicles + fuel logs with `expand: "vehicle"` |
| `/dashboard/employees` | Server component | `getFullList` of employees (with `expand: "site"`), timesheets, sites |
| `/dashboard/plant` | Server component | `getFullList` of vehicles (`perPage: 5000`), fuel logs, sites |
| `/dashboard/purchasing` | Server component | `getFullList` of POs, PO items, DNs, sites, received items |
| `/dashboard/chat` | Server component (gated by `chatEnabled()`) | Provider config only — message stream happens client-side via SSE |
| `/api/**` | Route Handlers | One shot per request |

There is no client-side PocketBase fetching in any current page. The pattern
is: pull everything server-side, hand the view component a fully-shaped
`rows` / `summaries` array, render. This is the simplest pattern that
delivers the current data volumes (single-digit thousands of rows), and is
exactly what makes the multi-site assessment in
[`multi-site-assessment.md`](./multi-site-assessment.md) say "Tier 2 — this
stops scaling at ~20-30 sites" rather than "it broke last Tuesday".

---

## 4. Auth model

There is no real authentication. The "auth gate" is a shared numeric PIN
held in `sessionStorage` after a client-side string compare.

- PIN value comes from `NEXT_PUBLIC_APP_PIN` (default `2468`,
  `src/hooks/use-auth.tsx:20`). It is exposed to the browser because the
  lock screen validates it locally before showing the dashboard.
- Verification state is stored in `sessionStorage` under
  `mega-auth-verified` (`src/hooks/use-auth.tsx:6`).
- `AuthGate` (`src/components/auth-gate.tsx`) redirects to `/login` whenever
  the flag is missing, but does so client-side — every server component
  below it still runs on the server.
- Append `?demo=1` to any dashboard URL to bypass the PIN entirely
  (`src/hooks/use-auth.tsx:27-30`).
- There is **no per-user identity anywhere in the application**. Every user
  who knows the PIN sees every site's contract values, payroll rates, and
  margins.

The PocketBase side, again, runs as superuser. PocketBase API rules
(`scripts/pb-relax-rules.py`) allow any authenticated user to list / view /
create on every collection, and reserve update / delete for the superuser.
Since the server is the superuser, *every* PB operation the app performs is
update-or-delete-eligible.

The "auth" header the chat API checks is `ENABLE_CHAT=true`, not a user
credential. See the `ENABLE_CHAT` note in [`operations.md §1`](./operations.md#1-environment-variables).

---

## 5. Build, deploy, and the `--webpack` requirement

| Layer | Tool | Notes |
| --- | --- | --- |
| App framework | Next.js 16 (App Router) | Modified build — see `AGENTS.md` and the patch in `scripts/patch-next-dev-blocker.cjs` |
| PWA | `@ducanh2912/next-pwa` | **Webpack-only**, so `next dev` and `next build` both pass `--webpack` (`package.json:6-7`) |
| Styling | Tailwind v4 (`@tailwindcss/postcss`) | `src/app/globals.css` is the only stylesheet |
| Charts | Recharts | Used for income-vs-expense, daily fuel cost, weekly hours |
| File parsing | SheetJS (`xlsx`) + PapaParse (`papaparse`) | All importers accept both XLSX and CSV through the same route |
| Test runner | Vitest | `npm run test:run` |
| Container | Multi-stage `Dockerfile` | `deps` → `builder` (Next standalone) → `runner` (non-root, port 3000) |
| Reverse proxy | Coolify / Traefik | `COOLIFY.md` has the full Coolify v4 setup |
| Healthcheck | `GET /api/health` | `src/app/api/health/route.ts` returns `{ status, uptime, timestamp }` |

The `npm run build` pipeline runs `npm run icons && npm run og-image && next
build --webpack` in that order (`package.json:7`). `scripts/build-icons.mjs`
regenerates the PWA icon set and `scripts/build-og-image.mjs` regenerates
`/og-image.png`; both must run before the Next build so the static files
exist when the PWA plugin captures the manifest.

The standalone server in the final image binds `${PORT:-3000}` on
`0.0.0.0` (`Dockerfile:64-65`) and runs as a non-root `nextjs` user. Static
assets are copied in from `.next/standalone` + `.next/static`; the
`output: "standalone"` mode does **not** bundle `.next/static` automatically
(`Dockerfile:77-80`).

---

## 6. Cross-cutting infrastructure

### 6.1 Site resolution

Free-text site lookup goes through one helper,
[`resolveSite`](./multi-site-assessment.md#2-site-resolution-by-name-string):
exact id → exact `project_code` → exact `name` → substring on either
(`src/lib/agent/resolve.ts:21-48`). Ambiguity is returned explicitly as a
`"ambiguous"` result, never silently picked. The same shape exists for
vehicles (`resolveVehicle`).

### 6.2 Shared metrics

Every page that computes financials imports from
[`@/lib/metrics`](./modules.md#shared-metrics-surface). The
[`Site` page](./modules.md#2-sites), the **Overview** page's monthly-fixed-cost
tile, the **chat** tool layer, and the **MCP** server all build on the same
pure functions (`computeSiteSpend`, `computeSiteProjection`,
`buildWeeklyIncomeVsExpenses`, etc.). The tests under
`src/lib/metrics/*.test.ts` pin every computation — a regression in the
number is a failed test, not a silent bug in the dashboard.

### 6.3 The batch state machine

Every data import creates a `data_import_batches` row. The state machine is
in `src/lib/batch-state.ts`:

```
pending → processing → processed
                    ↘ failed → (retry) → pending
                              ↘ dead
```

`nextBatchState({ lastResult: "fail", attempts })` advances `failed → dead`
after `MAX_ATTEMPTS` retries (`src/lib/batch-state.ts` and the call in
`src/app/api/delivery-notes/[id]/ocr-result/route.ts:486-489`). Timesheets,
fuel logs, and plant hours are born `processed` because their imports are
synchronous (no AI, no queue). Purchase orders are born `pending` and
processed by n8n. Delivery notes are born `pending` and processed by the
OCR webhook.

### 6.4 Where the numbers reconcile

`Site` page, **Sites** list page, Overview tile, and the chat's
`list_sites` tool all feed the same buckets into
`computeSiteSpend`. So "what's the burn on The Hills?" returns the same
rand value from any surface. The single most load-bearing property of this
codebase.

---

## 7. What this document deliberately does not cover

- **Collection shapes and field-level detail** — [`data-model.md`](./data-model.md)
- **Per-module behaviour and routes** — [`modules.md`](./modules.md)
- **Ingestion pipelines (timesheets, fuel, plant, PO, DN) with sequence diagrams** — [`data-flows.md`](./data-flows.md)
- **Every API route with method / auth / request / response / errors** — [`api-reference.md`](./api-reference.md)
- **Env vars, scripts, tests, feature flags** — [`operations.md`](./operations.md)
- **Whether this app actually holds up at 50 sites** — [`multi-site-assessment.md`](./multi-site-assessment.md)