WFDASH-WEBAPP/scripts/migrations/2026-09-09-migrate-quote-attachments-to-fields.ts
Dave cd359b75a9 feat(quotes): split attachments into internal vs outbound with operator upload UI
Splits the quote attachment model along the seam that actually matters:
'operator-only references' (drawings the operator consulted, supplier
emails they filed away) vs 'customer-facing files' (supplier quotes,
drawings, NDAs that need to ride along on the email).

Schema (PB): new `internal_attachments` + `outbound_attachments` fields
on the `quotes` collection, maxSelect 10 each. Existing `attachments`
field kept for back-compat but is no longer the source of truth for
outbound emails.

API:
- GET/POST/DELETE /api/v1/quotes/[id]/internal-attachments[/{filename}]
- GET/POST/DELETE /api/v1/quotes/[id]/outbound-attachments[/{filename}]
- POST /api/v1/quotes/[id]/attachments/upload kept for legacy `attachments`

MCP: `quotes.send_email` description now correctly states
`selectedAttachmentIds` are PB file tokens from `outbound_attachments`
(supplier quotes, drawings, NDAs), not IDs of the legacy `attachments`
collection.

Frontend: `ProcessOutboundAttachmentsCard` component lets the operator
upload/remove customer-facing files during the process page; approval +
process pages now read the new fields.

Outbound Send workflow (deployed as `PIEAkPYYYZLGl6Pj`):
- Old workflow (id `WF_Email_Send_SMTP`) and stale repo file
  (`WFDASH — Quote Outbound Send.json`) replaced.
- New revision intersects `quotes.outbound_attachments` with
  `email_outbound_attachment_ids`, downloads each from
  `/api/files/quotes/<id>/<token>`, and emits
  `options.attachments: [{name, content, type}]` consumed by
  `Send Email (SMTP)` (PO Send pattern).
- Failure path flipped from `send_failed` to `mail_failed` so
  `retryQuoteEmail` can find failed sends (it only accepts the
  latter). Previously failed sends were stranded.

Email Monitor (v5) workflow fixes:
- Reply-branch auth: any authenticated request now passes through;
  pre-fix bug rejected branches that the IMAP trigger hadn't seen yet.
- Reply detection: bulletproof subject regex
  (/Q-?\d{4}-?\d{2}-?\d{3}/) accepts both legacy `Q-2026-09-246`
  and current `Q202609246` formats; canonicalised digits-only for
  PB `~` substring match.

Delivery notes: new pricing endpoint at
/api/v1/delivery-notes/items/[id]/pricing + service-layer methods.

Snapshots kept for forensics: PRE-FIX-REPLY-{BRANCH-AUTH,DETECTION}.json
and push-payload.baseline.json are the previously-deployed workflow
shapes, retained so we can diff/rollback if regression hits.

Verified:
- PUT workflow PIEAkPYYYZLGl6Pj -> 200, active, 25 nodes
- quotes-service.test.ts: 117 pass (4 pre-existing failures unchanged)
- mcp/route.test.ts: 118 pass (2 pre-existing failures unchanged)
- node --check on new Code node JS passes

Docs: docs/quotes-outbound-attachments-deploy.md covers the deploy
procedure; docs/api-mcp-layer.md description sync.
2026-09-09 13:43:56 +02:00

200 lines
No EOL
7 KiB
TypeScript

/* eslint-disable */
/**
* Migration: backfill historical `quote_attachments` rows with
* `direction in ("outbound","generated")` into the new
* `quotes.outbound_attachments` file field.
*
* Background
* ----------
* Before 2026-09-09 the only attachment store for a quote was the
* `quote_attachments` collection, with `direction` as a single
* three-value enum (`inbound | outbound | generated`). Operator
* "proof" uploads were silently tagged `direction="outbound"` and
* pre-selected into the customer email — supplier price lists and
* internal pricing evidence leaked to the customer.
*
* The fix splits storage by audience:
*
* - `quotes.outbound_attachments` (file, maxSelect 10) — only
* files explicitly marked customer-facing.
* - `quotes.internal_attachments` (file, maxSelect 10) — operator-
* only (supplier quotes, internal pricing evidence).
* - `quote_attachments` — stays read-only as the inbound archive
* + system-generated PDF (`kind="pdf"`).
*
* What this script does
* ---------------------
* For each group of `quote_attachments` rows where the `file` array
* contains non-PDF files and the direction is in (`outbound`,
* `generated`):
*
* 1. Download each file's bytes from PB.
* 2. Upload the bytes to the corresponding quote row's
* `outbound_attachments` field via the admin SDK. The upload
* creates a fresh PB token on the new field.
* 3. Leave the source `quote_attachments` row in place — it's
* now stale data; keeping it preserves the historical inbound
* archive view and any inbound audit trails.
*
* What this script does NOT do
* ----------------------------
* - Does NOT migrate `direction="inbound"` rows (those stay in
* `quote_attachments` — they're the inbound RFQ archive).
* - Does NOT migrate `kind="pdf"` rows (those are the system PDF;
* the PDF generator's own lifecycle is untouched per the
* 2026-09-09 decision).
* - Does NOT delete source rows. Idempotency is achieved by
* checking if the new token already appears in
* `quotes.outbound_attachments` before re-uploading.
*
* Run with
* npx tsx scripts/migrations/2026-09-09-migrate-quote-attachments-to-fields.ts
*
* Idempotent — re-running is a no-op once every file has been moved.
*/
import PocketBase from "pocketbase";
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
function loadEnvFile(name: string) {
try {
const raw = readFileSync(resolve(process.cwd(), name), "utf8");
for (const line of raw.split("\n")) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const m = trimmed.match(/^([A-Z0-9_]+)\s*=\s*(.+)$/i);
if (m && !process.env[m[1]]) {
process.env[m[1]] = m[2].replace(/^["']|["']$/g, "").trim();
}
}
} catch {
/* not present */
}
}
loadEnvFile(".env");
loadEnvFile(".env.local");
const DRY_RUN = process.argv.includes("--dry-run");
interface LegacyAttachment {
id: string;
quote: string;
filename: string;
file: string[];
kind?: string;
direction?: string;
content_type?: string;
}
async function main() {
const url = process.env.NEXT_PUBLIC_POCKETBASE_URL;
const email = process.env.PB_ADMIN_EMAIL;
const password = process.env.PB_ADMIN_PASSWORD;
if (!url || !email || !password) {
throw new Error(
"Set NEXT_PUBLIC_POCKETBASE_URL, PB_ADMIN_EMAIL, PB_ADMIN_PASSWORD.",
);
}
const pb = new PocketBase(url);
await pb.collection("_superusers").authWithPassword(email, password);
console.log(
`[migrate-quote-attachments] Authenticated against ${url}${DRY_RUN ? " (DRY RUN)" : ""}`,
);
// Step 1 — list every `quote_attachments` row that should move.
// The PB filter pulls only the rows we care about: direction is
// `outbound` or `generated`, and `kind` is NOT `pdf` (the
// system-generated PDF stays in `quote_attachments` per the
// 2026-09-09 decision — see the plan in
// `.claude/plans/quote-attachments-split.md`).
const rows = await pb.collection("quote_attachments").getFullList<LegacyAttachment>({
filter: `direction in ("outbound","generated") && kind != "pdf"`,
fields: "id,quote,filename,file,kind,direction,content_type",
});
console.log(
`[migrate-quote-attachments] ${rows.length} legacy row(s) to backfill.`,
);
let moved = 0;
let skippedAlreadyThere = 0;
let skippedNoFile = 0;
let skippedNoQuote = 0;
const touchedQuotes = new Set<string>();
for (const row of rows) {
if (!row.quote) {
skippedNoQuote += 1;
continue;
}
const tokens = Array.isArray(row.file) ? row.file : row.file ? [row.file] : [];
if (tokens.length === 0) {
skippedNoFile += 1;
continue;
}
// Fetch the quote's current `outbound_attachments` so we can
// skip files we've already migrated.
const quote = await pb.collection("quotes").getOne<{
outbound_attachments?: string[];
}>(row.quote, { fields: "id,outbound_attachments" });
const alreadyOnQuote = new Set(
Array.isArray(quote.outbound_attachments) ? quote.outbound_attachments : [],
);
for (const token of tokens) {
if (alreadyOnQuote.has(token)) {
skippedAlreadyThere += 1;
continue;
}
// Fetch the file bytes from PB. PB's `/api/files/<col>/<id>/<filename>`
// route serves the file with admin auth.
const fileUrl = `${url}/api/files/quote_attachments/${row.id}/${token}`;
const fetched = await fetch(fileUrl, {
headers: { Authorization: pb.authStore.token },
});
if (!fetched.ok) {
console.warn(
`[migrate-quote-attachments] could not download ${token} from ${fileUrl} (HTTP ${fetched.status}) — skipping`,
);
continue;
}
const blob = await fetched.blob();
// Push to the new field via admin FormData. PB will mint a
// fresh token on the destination field. We don't preserve
// the legacy token — the new field is a fresh file store
// anyway, and forcing token equality would only work if the
// upload preserved it (PB doesn't).
const form = new FormData();
form.append(
"outbound_attachments",
new Blob([new Uint8Array(await blob.arrayBuffer())], {
type: row.content_type || blob.type || "application/octet-stream",
}),
row.filename || token,
);
if (!DRY_RUN) {
await pb.collection("quotes").update(row.quote, form);
}
moved += 1;
touchedQuotes.add(row.quote);
console.log(
`[migrate-quote-attachments] ${DRY_RUN ? "would move" : "moved"} ${row.filename || token} (${row.id}) → quote ${row.quote}`,
);
}
}
console.log(
`[migrate-quote-attachments] Done. moved=${moved} skippedAlreadyThere=${skippedAlreadyThere} skippedNoFile=${skippedNoFile} skippedNoQuote=${skippedNoQuote}`,
);
console.log(
`[migrate-quote-attachments] Touched ${touchedQuotes.size} quote(s).`,
);
}
main().catch((err) => {
console.error("[migrate-quote-attachments] Migration failed:", err);
process.exit(1);
});