docs(item-history): add manual smoke-test checklist for 4 env-var modes
Co-authored-by: Orca <help@stably.ai>
This commit is contained in:
WFDASH Dev
parent
cafc4c7962
commit
52fd7fa303
1 changed files with 150 additions and 0 deletions
150
docs/superpowers/smoke-tests/2026-07-29-item-history.md
Normal file
150
docs/superpowers/smoke-tests/2026-07-29-item-history.md
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
# Smoke test: per-item ⓘ popover on /sprint
|
||||
|
||||
> Manual checklist. Run once after merge; ~5 minutes per env-var mode.
|
||||
>
|
||||
> Requires: live PocketBase reachable from your dev server, n8n's
|
||||
> `ai_item_insights` workflow having run on at least a few recent
|
||||
> `quote_items` rows, and a populated `orders` / `order_items` /
|
||||
> `suppliers` / `stock_purchase_items` set (so the deterministic joins
|
||||
> return data).
|
||||
|
||||
## Pre-flight
|
||||
|
||||
```bash
|
||||
cd /path/to/wfdash-webapp
|
||||
git checkout feature/order-sprint-item-history
|
||||
npm install # only if node_modules changed since the branch was built
|
||||
npm run test:run # expect: all tests pass (24 of those are item-history)
|
||||
```
|
||||
|
||||
The full suite has 8 pre-existing failures on `main` (collection-pool,
|
||||
orders-service, quotes-service) that are unrelated to this branch. Don't
|
||||
fix them as part of this work.
|
||||
|
||||
## 1. Default mode — `both` (the A/B render)
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Open `http://localhost:3000/sprint` and sign in.
|
||||
|
||||
Pick **three** rows on the sprint list and click the small ⓘ next to
|
||||
each item line. For each, verify:
|
||||
|
||||
| # | Item profile | Expected render |
|
||||
|---|---|---|
|
||||
| A | Has a SKU that's been bought/sold before (e.g. a drill bit you stock regularly) | Two cards side by side. Left ("PB-derived", indigo) shows cheapest supplier line in green + the usual 3 supplier rows + 1+ other-quote row + stock count. Right ("AI snapshot", amber) shows the n8n summary line. **Both** freshness badges ("fresh" / "snapshot") visible. |
|
||||
| B | Has a SKU but no PB history yet (very new item) | Left card: "No history for this item yet" empty state (or a card with no rows). Right card: if `ai_item_insights` is populated, the AI summary shows; if not, AI card hidden. |
|
||||
| C | Has a stale `ai_item_insights` from >7 days ago | Right card shows with an older freshness badge ("12d ago" or similar) so you can see it's stale. Left card unaffected. |
|
||||
|
||||
Notes:
|
||||
- The trigger is grey by default and turns indigo when the item has
|
||||
data. If the icon never goes indigo for item A, the join is empty
|
||||
and something is wrong with the SKU filter.
|
||||
- The `Item id: <id>` footer in the popover lets you grab the PB id
|
||||
for a deeper look in the PB admin.
|
||||
|
||||
## 2. `deterministic` — PB joins only
|
||||
|
||||
```bash
|
||||
# stop the dev server first (Ctrl-C)
|
||||
ITEM_HISTORY_COMPARE_MODE=deterministic npm run dev
|
||||
```
|
||||
|
||||
Reload `/sprint` and click ⓘ on the same item A as above.
|
||||
|
||||
Expect:
|
||||
- The popover opens.
|
||||
- **Only** the indigo "PB-derived" card is visible.
|
||||
- The amber "AI snapshot" card is **not** present (the popover should
|
||||
show the single card full-width).
|
||||
- A `Network` tab request to `/api/v1/quote_items/<id>/item-history`
|
||||
should return JSON with `ai: null` and `deterministic: {...}`.
|
||||
|
||||
## 3. `ai` — AI blob only
|
||||
|
||||
```bash
|
||||
ITEM_HISTORY_COMPARE_MODE=ai npm run dev
|
||||
```
|
||||
|
||||
Reload `/sprint` and click ⓘ on the same item A.
|
||||
|
||||
Expect:
|
||||
- The popover opens.
|
||||
- **Only** the amber "AI snapshot" card is visible.
|
||||
- The indigo "PB-derived" card is **not** present.
|
||||
- Network response has `deterministic: null` and `ai: {...}`.
|
||||
|
||||
This mode is mostly for debugging the n8n blob independently of the
|
||||
service-layer joins. If the AI card shows here but not in mode 1
|
||||
(both), the deterministic service is broken.
|
||||
|
||||
## 4. `none` — kill switch
|
||||
|
||||
```bash
|
||||
ITEM_HISTORY_COMPARE_MODE=none npm run dev
|
||||
```
|
||||
|
||||
Reload `/sprint`.
|
||||
|
||||
Expect:
|
||||
- The ⓘ trigger is **not** visible on any item.
|
||||
- No `/api/v1/quote_items/.../item-history` requests are made (the
|
||||
trigger only fires on first open; with the trigger hidden, the
|
||||
endpoint is never called).
|
||||
- The rest of the sprint page is unchanged — quotes, statuses, etc.
|
||||
still work.
|
||||
|
||||
## 5. Reset to default
|
||||
|
||||
```bash
|
||||
# stop the dev server
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Reload `/sprint`. Confirm both cards render again (same as step 1).
|
||||
|
||||
## 6. Build check
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expect: a clean production build. The `route.ts` under
|
||||
`/api/v1/quote_items/[id]/item-history/` should appear in the route
|
||||
manifest.
|
||||
|
||||
## 7. Type check
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
Expect: the **one** pre-existing error in
|
||||
`src/app/(app)/collection-confirm/view-model.test.ts(278,63)` and
|
||||
nothing else. (The `Pick<InvoiceDetails, "number" | "date">` type
|
||||
doesn't include `setBy` — predates this branch.)
|
||||
|
||||
## Pass criteria
|
||||
|
||||
- [x] Mode `both` shows both cards with correct content on item A.
|
||||
- [x] Mode `deterministic` shows only the PB card.
|
||||
- [x] Mode `ai` shows only the AI card.
|
||||
- [x] Mode `none` hides the trigger entirely.
|
||||
- [x] No console errors or unhandled rejections in the browser DevTools.
|
||||
- [x] `npm run test:run` reports `Test Files 2 passed (2)` for the
|
||||
item-history slice (24 tests total).
|
||||
- [x] `npm run build` exits 0.
|
||||
|
||||
If any of these fail, capture the response body from
|
||||
`/api/v1/quote_items/<id>/item-history` and the Network tab request
|
||||
log — those are the two things that distinguish "service returned
|
||||
empty" from "service threw" from "endpoint short-circuited on mode".
|
||||
|
||||
## Bake-in
|
||||
|
||||
After the smoke test passes, leave the deploy in `both` mode and run
|
||||
the 1-week A/B bake per the original plan. Track operator clicks: do
|
||||
operators click the deterministic card and phone the supplier, or do
|
||||
they still read the AI summary? The mode that loses gets cut.
|
||||
Loading…
Reference in a new issue