docs(orders): record sweep checkpoint outcome, keep 90-day window

This commit is contained in:
Dave 2026-08-30 12:13:29 +02:00
parent e374513fba
commit cb2beb4016

View file

@ -233,17 +233,60 @@ exist as an MCP tool. Follow the `orders.set_quote` pattern in
`src/app/api/mcp/route.ts` — tool definition alongside it, handler forwarding
to the REST route.
### ⛔ CHECKPOINT — human runs a real sweep before Phase 2
### ✅ CHECKPOINT — DONE 2026-08-29. Sweep run, window confirmed at 90 days.
Run one batch against real data, read the `ageGaps` histogram, and decide
whether 90 days is right before building UI on top. Expect the amber pill to
appear across many historical orders at once on the first full sweep — that's
information, not damage, but it will look noisy for a day.
**Outcome: `FUZZY_WINDOW_DAYS` stays at 90. No change needed. Phase 2 is unblocked.**
Full backlog swept: **5,096 orders processed, 2,483 suggested, 2,613 no-match,
0 failed** — `remaining: 0`. Amber pills are now live across the historical
backlog (approved by the owner).
**Read this before you trust the sweep's `ageGaps` output.** The histogram
reported `d0_30: 100%`, which is an **artifact, not a finding**:
- **92% of matched backlog orders have a gap of exactly 0 days.** Their
`created` timestamps come from the legacy bulk migration — order and quote
were inserted at the same instant — so the gap measures migration timing,
not quote-to-order lead time.
- `quoteToOrderAgeDays` itself is **correct** (verified: returns `33` for a
real 33-day PB-format pair). Do not go looking for a bug in it.
- The real signal, from non-migrated orders: gaps run **1–33 days, p50 23,
p90 24, max 33. Zero beyond 90 days.** That's what justifies keeping 90 —
nearly 3× headroom against the worst observed real gap.
**Consequence worth knowing: for migrated records the date window constrains
nothing.** If an order and every candidate quote share a migration timestamp,
all of them fall inside any window. So those ~2,400 backlog suggestions were
scored with the window effectively disabled — only company / customer /
description / price carried them. Backlog amber pills deserve more operator
skepticism than new-order ones, whose timestamps are real and where the window
does its job.
**Two follow-ups, neither blocking Phase 2:**
1. Have the sweep report gap-0 matches in their own bucket so this histogram
can't mislead the next reader the way it nearly did here.
2. Check whether the migration preserved a real business date (there is a
`migration_meta` JSON field on `orders`) that the matcher could prefer over
`created` for migrated rows.
**Separate open question — do not conflate it with the window.** Match rate is
**48.7%**, well under the owner's ~85% expectation. The misses are not old
quotes, so widening the window will not help. Candidates: those orders
genuinely had no quote; the `MATCH_CONFIDENCE_THRESHOLD` of 0.4 is too strict;
or description drift exceeds what the Jaccard scorer tolerates. Diagnose
deliberately, don't tune blind.
**Operational note for anyone re-running the sweep:** `limit: 200` killed the
dev server outright (process gone, not a timeout). `limit: 50` is stable at
~14s per batch. The route advertises up to 500 with `maxDuration = 300`; that
ceiling is optimistic, because each order fans out into a candidate scan of up
to 200 quotes plus their line items, twice when the window widens. Consider
lowering the documented max.
```bash
curl -s -X POST http://localhost:4001/api/v1/orders/quote-match/sweep \
-H "authorization: Bearer $AGENT_API_KEY" \
-H 'content-type: application/json' -d '{"limit":10}'
-H 'content-type: application/json' -d '{"limit":50}'
```
(Dev server runs on **port 4001**, not 3000.)