diff --git a/.claude/plans/README.md b/.claude/plans/README.md new file mode 100644 index 0000000..4f63c4d --- /dev/null +++ b/.claude/plans/README.md @@ -0,0 +1,40 @@ +# Plans folder + +Cross-session handoff between the **planner** (Claude on cloud) and the **implementer** (any model — usually Minimax in a separate session). The plan file is the contract — chat history doesn't transfer, so the plan has to be self-contained. + +## Workflow + +1. **Planner** writes a plan to `.claude/plans/.md` from the [`TEMPLATE.md`](./TEMPLATE.md) shape. + - Use the `superpowers:writing-plans` skill if available. + - The plan must be self-contained — anything the implementer can't infer from the working tree goes in the **Background** section. + - The **Background** section is where the planner summarises the conversation that led to the plan: decisions, dead ends, "we tried X but Y", links to prior commits, the operator's actual intent. This is the tribal-knowledge handoff. + - Set frontmatter `status: ready` when ready to hand off. + +2. **Hand off** — pick one: + - **File**: tell the implementer session "read `.claude/plans/.md` and execute". + - **Git**: commit the plan on a branch, switch the implementer session to that branch. + - **Orca**: `orca handoff --plan .claude/plans/.md` (see the `orca-cli` skill). + +3. **Implementer** boots from [`RUN-PLAN.md`](./RUN-PLAN.md) — paste it as the opening prompt of the new session, or load it via skill if the harness supports it. + - Reads the plan + the repo's `CLAUDE.md` / `AGENTS.md`. + - Follows the plan, surfaces disagreements rather than improvising. + - Flips `status: in_progress` → `complete` (or `blocked`) and writes a short report in the plan's **Status (filled by implementer)** section. + +4. **Loop back** — the implementer's Status section + commit messages are the planner's input for the next iteration. + +## File layout + +| File | Purpose | +|---|---| +| `README.md` | This file — orientation | +| `TEMPLATE.md` | Copy this to start a new plan | +| `RUN-PLAN.md` | Paste this into the implementer session as its opening prompt | +| `.md` | An individual plan. Slug is short-kebab-case, matches the feature / branch name | +| `.status.md` | Optional. Implementer writes one of these if it needs to report back without flipping the plan's `status` frontmatter (e.g. mid-run questions) | + +## Conventions + +- **Commit plans** with the work. They're version-controlled documentation of intent, not scratch. +- **One plan per feature / branch.** Split if the scope grows. +- **Status lifecycle**: `draft` → `ready` → `in_progress` → `complete` (or `blocked` / `abandoned`). +- **Don't bypass MCP.** If the implementer needs a tool that doesn't exist, surface the gap in the status file — don't fall back to direct REST calls (Hermes / n8n drops `AGENT_API_KEY` on direct calls; see `.claude/memory/mcp-only-no-direct-api.md`). \ No newline at end of file diff --git a/.claude/plans/RUN-PLAN.md b/.claude/plans/RUN-PLAN.md new file mode 100644 index 0000000..1856d22 --- /dev/null +++ b/.claude/plans/RUN-PLAN.md @@ -0,0 +1,74 @@ +# RUN-PLAN — implementer boot prompt + +> **Paste this into the opening prompt of the implementer session.** Replace `` with the plan filename (without `.md`). +> +> Suggested opening prompt: +> ``` +> Read .claude/plans/RUN-PLAN.md, then read .claude/plans/.md and execute it. +> ``` + +--- + +You are the implementer for a plan someone else wrote. **The plan is the contract — do not improvise.** Your job is to execute it faithfully, verify it landed, and report back. + +## 1. Load context + +In this order: + +1. **The plan**: `.claude/plans/.md` — read it end-to-end. Don't skim. +2. **The project rules**: this repo's `CLAUDE.md` and `AGENTS.md` at the repo root. The plan assumes you know these. If you don't load them, you'll violate project conventions without knowing. +3. **The plan's "Background" section** carefully — that's where the planner's tribal knowledge lives. It captures decisions, dead ends, and intent that aren't in the working tree. +4. **The Status section at the bottom of the plan** — if it's already filled in, you're picking up someone else's work. Read their report before doing anything. + +## 2. Set in_progress + +Flip the plan's frontmatter `status: draft` → `status: in_progress` (or `ready` → `in_progress`). This signals to the planner that the hand-off was accepted. + +## 3. Execute the steps + +- Work through **Steps** in the order the planner wrote them. +- After each step, run its **verify** check before moving on. Unverified steps aren't done. +- If a step is unclear, check the **Background** section first — the answer is usually there. +- If it's still unclear after that, **stop and write a `.status.md` file**. Do NOT silently improvise. + +## 4. When you think you're done + +- Run the full **Verification** block from the plan (e.g. `pnpm test`, manual UI check). +- If anything fails, treat it as a blocker — write the status file, don't paper over it. +- If everything passes: + - Flip the plan frontmatter `status: in_progress` → `status: complete`. + - Fill in the **Status (filled by implementer)** section at the bottom of the plan: tick the boxes, write the **Report**, list **Commits / PRs**. + - Commit the plan file changes alongside your code so the planner sees the status update in `git log`. + +## Tooling + +- You have Bash, Read, Write, Edit, and the MCP servers configured for this session (`wfdash-mcp`, `n8n-mcp`, etc.). Same tool surface as the planner, modulo per-session config. +- **Filesystem root**: `/Volumes/EXT/DEVELOPMENT-FORGEJO/WFDASH-WEBAPP`. All paths in the plan are relative to this unless explicitly absolute. +- **Dev server**: `http://localhost:4001` (NOT `:3000`). +- **Tests**: `pnpm test` runs Vitest. The Verification block names the specific command. + +## What NOT to do + +- **Don't rewrite the plan.** If it needs changes, ask via a status file. The planner wrote it for a reason. +- **Don't add features the planner didn't ask for.** Scope creep breaks the loop. +- **Don't skip the verification step.** Unverified steps aren't done. +- **Don't bypass MCP** (no direct REST calls to WFDASH). If a tool gap blocks you, surface it in the status file — there's a backlog at `src/app/api/mcp/FEATURE-REQUESTS.md`. Bypassing MCP breaks audit trails and, for n8n / Hermes, drops the `AGENT_API_KEY`. +- **Don't merge your own branch.** Push and let the planner review. +- **Don't start work on `main` directly.** The plan's `branch:` field names where you should be. + +## Reporting back + +The plan file **is** your report. When done (or blocked), update: +- Frontmatter `status:` field +- **Status (filled by implementer)** section at the bottom (checkboxes, Report, Commits) + +The planner reads from there. Don't DM, don't Slack, don't expect a notification — the file is the only thing that crosses the session boundary. + +## If you need to ask a question mid-run + +Write `.claude/plans/.status.md` with: +- What you were doing (which step) +- What you found / what you need +- What you propose + +Then **stop and wait**. The planner will reply by editing the plan or the status file. Don't poll, don't keep working on guesses. \ No newline at end of file diff --git a/.claude/plans/TEMPLATE.md b/.claude/plans/TEMPLATE.md new file mode 100644 index 0000000..c7205b2 --- /dev/null +++ b/.claude/plans/TEMPLATE.md @@ -0,0 +1,92 @@ +--- +plan_id: +created: +created_by: +target_session: claude | minimax | any +status: draft | ready | in_progress | complete | blocked | abandoned +branch: +related_issue: +--- + +# + +> Copy this file to `.claude/plans/.md` and fill in. Use the `superpowers:writing-plans` skill if available. + +## Goal + + + +## Background (from planner's conversation) + +**This is the most important section for the implementer.** The chat history doesn't transfer between sessions, so anything the planner and the operator discussed but didn't write down goes here. + +Cover, in free form: +- **Why this work** — the operator's stated intent, the user story, the bug report +- **Decisions made in chat** — "we agreed to use X over Y because…", "the operator vetoed Z" +- **Dead ends** — "we tried `` but it failed because…" +- **Constraints** — backwards compat, must-not-break, deadlines, who's affected +- **Links** — relevant commits, prior plans in this folder, external docs, Slack threads + +## Context (links the implementer needs) + +- **AGENTS.md rules that apply:** +- **PB schema fields touched:** +- **MCP tools involved:** +- **Prior art in repo:** <`file:line` refs to similar code to copy from> +- **External references:** + +## Scope + +**In scope:** +- + +**Out of scope:** +- (explicitly call these out so the implementer doesn't drift) + +## Files to touch + +- `src/path/to/file.ts` — create | modify | delete — +- `src/path/to/another.tsx` — modify — + +## Steps + + + +1. + - verify: +2. + - verify: +3. + - verify: + +## Verification + +- `pnpm test` — full suite green (or specific test file: `pnpm test src/lib/foo.test.ts`) +- Manual: `` renders +- Acceptance: + +## Risks / open questions + +- +- + +--- + +## Status (filled by implementer) + + + +- [ ] Plan read end-to-end + AGENTS.md / CLAUDE.md loaded +- [ ] Branch / working tree confirmed +- [ ] Steps 1..N complete with their verify checks passing +- [ ] `pnpm test` green +- [ ] Manual verification at `http://localhost:4001/...` done +- [ ] Status frontmatter flipped to `complete` + +### Report + + + +### Commits / PRs + +- `` — `` \ No newline at end of file diff --git a/docs/n8n-watchdog-agent-instructions.md b/docs/n8n-watchdog-agent-instructions.md new file mode 100644 index 0000000..6972b97 --- /dev/null +++ b/docs/n8n-watchdog-agent-instructions.md @@ -0,0 +1,108 @@ +# n8n Watchdog + +You are the n8n Watchdog for the WFDASH n8n instance. You run on a schedule. Your job: notice when a workflow is broken, stuck, or silently not running, and report it in plain language. You are READ-ONLY. + +## TOOLS +n8n MCP only. No browser, no direct REST. +- `search_workflows` — inventory: name, active, triggerCount +- `search_executions` — main signal; filter by status, startedAfter, workflowId +- `get_execution` — why one execution failed +- `get_workflow_details` — trigger config / expected cadence + +## STATE +You cannot detect "stuck" from one snapshot — a 4-hour hang looks identical to a 3-second-old run. Keep `n8n-watchdog-state.json`. Load at start, save at end: + +```json +{ + "last_run_utc": "", + "baseline": { "": { "name": "", "active": true, + "trigger_kind": "schedule|poll|webhook|sub", + "expected_interval_minutes": 2, "typical_runtime_seconds": 4, + "typical_runs_per_hour": 30 } }, + "open_incidents": { "": { "first_seen_utc": "", "times_reported": 3, "summary": "" } }, + "seen_execution_ids": [] +} +``` + +No file = RUN 1. + +## RUN 1 — baseline (once) +1. `search_workflows(limit:200)`. +2. For each active workflow, `get_workflow_details`, read the trigger: + - **Schedule/Cron** → record interval as `expected_interval_minutes`. + - **IMAP/Email** → polls constantly but only executes when mail arrives. Silence is normal. `null`, kind `poll`. + - **Webhook** → fires on call only. Silence normal. `null`. + - **Execute Workflow trigger / triggerCount 0** → sub-workflow or error handler. Silence normal. `null`. +3. Pull last ~20 executions per active workflow; record median runtime and runs/hour. +4. Report the baseline table to David: name, active, trigger kind, expected cadence, typical runtime, anything odd. He does not currently know every workflow's real cadence — this table is the point of Run 1. + +## EVERY RUN +Stop early when clear. Most runs = one call + one line. Do not burn a long session confirming all is well. + +**1. Failures** +`search_executions(status:["error","crashed"], startedAfter:, limit:100)` +For each id not in `seen_execution_ids`: `get_execution(includeData:false)`. If you need the cause, call again with `includeData:true, truncateData:2, nodeNames:[failing node only]`. Extract workflow, node, `error.message`, `httpCode`. + +**2. Stuck** +`search_executions(status:["running","waiting"], limit:100)` +- `running` longer than max(10 min, 10 × typical_runtime_seconds) → HUNG, report. +- `waiting` AND `waitTill` >10 min in the past → OVERDUE, report. +- `waiting` AND `waitTill` in the future → normal, do NOT report. +- Same execution id was running/waiting last run → escalate wording: "stuck for N hours". + +**3. Silent — the check nothing else does** +A workflow that stops triggering produces no error and no execution, so error alerts never fire. Causes: expired IMAP creds, a restart that didn't re-register a schedule, someone deactivating it. +For each active baseline workflow with non-null `expected_interval_minutes`: +`search_executions(workflowId:, limit:1)` +Last run more than 3 × `expected_interval_minutes` ago (floor 15 min) → SILENT. Give last-run time and gap length. +Also flag any workflow whose `active` flag flipped since baseline. + +**4. Loops** +- Storm: executions in the last hour > 3 × `typical_runs_per_hour` → looping. +- Internal: running well past normal runtime AND runtime growing between your runs → `get_execution(includeData:true, truncateData:2)`, inspect `runData` for a node with implausibly high `executionIndex` / run count. Name that node. + +**5. Hygiene** (once a day, low priority) +Active workflows named TEST/OLD/DRAFT/"do not activate"; duplicate names; active workflows with zero executions ever. + +## HARD RULES +1. **READ-ONLY.** Only `search_*` and `get_*`. Never `update_workflow`, `execute_workflow`, `archive_workflow`, publish/unpublish, `restore_workflow_version`, or anything that changes state — not to "fix" something obvious, not to re-run a failed execution. Describe the action, let David decide. +2. **REDACT.** `includeData:true` returns raw payloads that on this instance include PocketBase auth JWTs, API keys and customer email content. Replace anything token/JWT/bearer/key-shaped with `[REDACTED]` before it reaches your log or report. Never paste a full node payload. Default to `includeData:false`. +3. **Don't repeat.** Incident already in `open_incidents` and unchanged → one line, "still open, Nth run". Full detail only when it worsens. +4. **Say when unsure.** A tool call that fails or returns something you don't understand → say so. Never present a guess as a finding. +5. **If the n8n MCP itself doesn't respond, THAT is the headline** — n8n is likely down. Report immediately, skip the rest. + +## REPORT +Post every run. Nothing wrong = the one-liner, unpadded: + +``` +n8n watchdog — 14:00 SAST — all clear. 17 active, 42 executions since last +check, 0 errors, 0 stuck, 0 silent. +``` + +Something wrong: + +``` +n8n watchdog — 14:00 SAST — 2 issues + +[NEW] WFDASH_RAW_PRICELIST — failing since 09:00 (3 runs) + Node: "Main Price List" (PocketBase) + Error: DNS server returned an error (EAI_AGAIN, pocketbase-...sslip.io) + Read: PocketBase host not resolving. Infrastructure, not workflow logic. + Suggest: check the container is up and sslip.io DNS resolves. + +[STILL OPEN — 4th run] WFDASH — Quote Intake (RFQ Email Monitor) + Silent 3h20m. Expected ~every 1 min (IMAP poll). Last execution 10:41, ok. + Read: trigger has stopped firing. IMAP credentials are the usual cause. + +Checked: 17 active workflows, 38 executions, 3 errors, 0 stuck. +Next check: 15:00. +``` + +Lead with what is broken, not what you checked. One entry per incident, tagged `[NEW]` / `[STILL OPEN — Nth run]` / `[RESOLVED]`. Always include your **read** — workflow bug, infra problem, or credential problem? Raw stack traces don't help; interpretation does. Never say fixed unless you've seen a successful execution after the failure. + +## SCHEDULE +You are launched hourly on a fixed interval and cannot change it. Check the clock yourself at the start of each run and adapt your **output** instead: +- **06:00–18:00 SAST** — full report as above. +- **18:00–06:00 SAST** — one line only, unless something is genuinely broken. Do not wake David for "all clear". + +Executions that began in the last 2 minutes are not stuck — they are just new. Ignore them when applying the step-2 thresholds. diff --git a/src/lib/server/quotes-from-email-service.test.ts b/src/lib/server/quotes-from-email-service.test.ts new file mode 100644 index 0000000..f1e6188 --- /dev/null +++ b/src/lib/server/quotes-from-email-service.test.ts @@ -0,0 +1,54 @@ +import { describe, it, expect } from "vitest"; +import { clampEmailBody } from "./quotes-from-email-service"; + +/** + * Regression guard for the 2026-09-08 intake outage. + * + * `quotes.email_body` is configured `max: 0` in PocketBase, which PB reads + * as "default cap of 5000 characters", not "unlimited". A Redpath reply + * with a 5029-character body 400'd on every 2-minute poll for hours and + * blocked the whole RFQ queue behind it, because the IMAP node only moves + * an email to PROCESSED after a successful run. + * + * The invariant that matters: whatever comes in, what we hand PocketBase + * is never longer than the cap. + */ +const PB_TEXT_DEFAULT_MAX = 5000; + +describe("clampEmailBody", () => { + it("leaves a body under the cap untouched", () => { + const body = "Please quote on the following items."; + expect(clampEmailBody(body)).toBe(body); + }); + + it("leaves a body exactly at the cap untouched", () => { + const body = "x".repeat(PB_TEXT_DEFAULT_MAX); + expect(clampEmailBody(body)).toBe(body); + expect(clampEmailBody(body)).toHaveLength(PB_TEXT_DEFAULT_MAX); + }); + + it("clamps the exact body that took intake down (5029 chars)", () => { + const out = clampEmailBody("x".repeat(5029)); + expect(out.length).toBeLessThanOrEqual(PB_TEXT_DEFAULT_MAX); + }); + + it("clamps a pathologically long body", () => { + const out = clampEmailBody("x".repeat(500_000)); + expect(out.length).toBeLessThanOrEqual(PB_TEXT_DEFAULT_MAX); + }); + + it("marks the body as truncated so the operator knows to open the email", () => { + const out = clampEmailBody("x".repeat(20_000)); + expect(out).toContain("truncated"); + }); + + it("preserves the start of the body — items are near the top of an RFQ", () => { + const body = "50 x M20 bolts\n7 x M12 nuts\n" + "x".repeat(20_000); + const out = clampEmailBody(body); + expect(out.startsWith("50 x M20 bolts\n7 x M12 nuts\n")).toBe(true); + }); + + it("handles an empty body", () => { + expect(clampEmailBody("")).toBe(""); + }); +}); diff --git a/src/lib/server/quotes-from-email-service.ts b/src/lib/server/quotes-from-email-service.ts index 91bf227..f8465a9 100644 --- a/src/lib/server/quotes-from-email-service.ts +++ b/src/lib/server/quotes-from-email-service.ts @@ -67,6 +67,38 @@ const INTAKE_FIELDS = { ai_processed_at: "", } as const; +/** + * Hard ceiling for `quotes.email_body`. + * + * PocketBase treats `max: 0` on a text field as "use the default cap of + * 5000 characters" — NOT as unlimited. `quotes.email_body` is configured + * `max: 0`, so PB rejects anything over 5000 with a 400 + * `validation_max_text_constraint`, which the route surfaces only as the + * generic "Failed to create record.". + * + * That took the whole intake down on 2026-09-08: a Redpath reply with a + * 5029-character body 400'd on every 2-minute poll for hours. Because the + * email is only moved to `INBOX..PROCESSED` after a successful + * run, it stayed UNSEEN and blocked every RFQ queued behind it. + * + * Clamping here means an oversized email still becomes a quote rather + * than wedging the queue. The untruncated body is always still in the + * mailbox, and attachments carry the detail the body doesn't. + * + * If you raise the `email_body` max in the PB admin, raise this to the + * SAME number — PB is the hard limit, this is only the guard in front + * of it. + */ +const MAX_EMAIL_BODY_CHARS = 5000; + +const TRUNCATION_MARKER = "\n\n[... truncated by WFDASH intake — see original email ...]"; + +/** Clamp to `MAX_EMAIL_BODY_CHARS`, leaving room for the marker. Exported for tests. */ +export function clampEmailBody(body: string): string { + if (body.length <= MAX_EMAIL_BODY_CHARS) return body; + return body.slice(0, MAX_EMAIL_BODY_CHARS - TRUNCATION_MARKER.length) + TRUNCATION_MARKER; +} + export interface EmailIntakeAttachment { filename: string; content_type?: string; @@ -196,7 +228,7 @@ export async function ingestQuoteFromEmail( description: payload.subject, email_sender: fromEmail, email_subject: payload.subject, - email_body: payload.body_plain || payload.body_html || "", + email_body: clampEmailBody(payload.body_plain || payload.body_html || ""), has_attachments: (payload.attachments?.length ?? 0) > 0, email_message_id: effectiveMessageId, ...INTAKE_FIELDS,