docs(orders): record sweep checkpoint outcome, keep 90-day window
This commit is contained in:
parent
e374513fba
commit
cb2beb4016
1 changed files with 49 additions and 6 deletions
|
|
@ -233,17 +233,60 @@ exist as an MCP tool. Follow the `orders.set_quote` pattern in
|
|||
`src/app/api/mcp/route.ts` — tool definition alongside it, handler forwarding
|
||||
to the REST route.
|
||||
|
||||
### ⛔ CHECKPOINT — human runs a real sweep before Phase 2
|
||||
### ✅ CHECKPOINT — DONE 2026-08-29. Sweep run, window confirmed at 90 days.
|
||||
|
||||
Run one batch against real data, read the `ageGaps` histogram, and decide
|
||||
whether 90 days is right before building UI on top. Expect the amber pill to
|
||||
appear across many historical orders at once on the first full sweep — that's
|
||||
information, not damage, but it will look noisy for a day.
|
||||
**Outcome: `FUZZY_WINDOW_DAYS` stays at 90. No change needed. Phase 2 is unblocked.**
|
||||
|
||||
Full backlog swept: **5,096 orders processed, 2,483 suggested, 2,613 no-match,
|
||||
0 failed** — `remaining: 0`. Amber pills are now live across the historical
|
||||
backlog (approved by the owner).
|
||||
|
||||
**Read this before you trust the sweep's `ageGaps` output.** The histogram
|
||||
reported `d0_30: 100%`, which is an **artifact, not a finding**:
|
||||
|
||||
- **92% of matched backlog orders have a gap of exactly 0 days.** Their
|
||||
`created` timestamps come from the legacy bulk migration — order and quote
|
||||
were inserted at the same instant — so the gap measures migration timing,
|
||||
not quote-to-order lead time.
|
||||
- `quoteToOrderAgeDays` itself is **correct** (verified: returns `33` for a
|
||||
real 33-day PB-format pair). Do not go looking for a bug in it.
|
||||
- The real signal, from non-migrated orders: gaps run **1–33 days, p50 23,
|
||||
p90 24, max 33. Zero beyond 90 days.** That's what justifies keeping 90 —
|
||||
nearly 3× headroom against the worst observed real gap.
|
||||
|
||||
**Consequence worth knowing: for migrated records the date window constrains
|
||||
nothing.** If an order and every candidate quote share a migration timestamp,
|
||||
all of them fall inside any window. So those ~2,400 backlog suggestions were
|
||||
scored with the window effectively disabled — only company / customer /
|
||||
description / price carried them. Backlog amber pills deserve more operator
|
||||
skepticism than new-order ones, whose timestamps are real and where the window
|
||||
does its job.
|
||||
|
||||
**Two follow-ups, neither blocking Phase 2:**
|
||||
1. Have the sweep report gap-0 matches in their own bucket so this histogram
|
||||
can't mislead the next reader the way it nearly did here.
|
||||
2. Check whether the migration preserved a real business date (there is a
|
||||
`migration_meta` JSON field on `orders`) that the matcher could prefer over
|
||||
`created` for migrated rows.
|
||||
|
||||
**Separate open question — do not conflate it with the window.** Match rate is
|
||||
**48.7%**, well under the owner's ~85% expectation. The misses are not old
|
||||
quotes, so widening the window will not help. Candidates: those orders
|
||||
genuinely had no quote; the `MATCH_CONFIDENCE_THRESHOLD` of 0.4 is too strict;
|
||||
or description drift exceeds what the Jaccard scorer tolerates. Diagnose
|
||||
deliberately, don't tune blind.
|
||||
|
||||
**Operational note for anyone re-running the sweep:** `limit: 200` killed the
|
||||
dev server outright (process gone, not a timeout). `limit: 50` is stable at
|
||||
~14s per batch. The route advertises up to 500 with `maxDuration = 300`; that
|
||||
ceiling is optimistic, because each order fans out into a candidate scan of up
|
||||
to 200 quotes plus their line items, twice when the window widens. Consider
|
||||
lowering the documented max.
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://localhost:4001/api/v1/orders/quote-match/sweep \
|
||||
-H "authorization: Bearer $AGENT_API_KEY" \
|
||||
-H 'content-type: application/json' -d '{"limit":10}'
|
||||
-H 'content-type: application/json' -d '{"limit":50}'
|
||||
```
|
||||
|
||||
(Dev server runs on **port 4001**, not 3000.)
|
||||
|
|
|
|||
Loading…
Reference in a new issue