WFDASH-WEBAPP/.mavis/plans/qa-health-check.yaml

157 lines
7.5 KiB
YAML
Raw Normal View History

version: 1
plan:
name: 'WFDASH QA health check'
max_concurrency: 2
max_consecutive_failures: 2
max_cycles: 6
auto_reject_retries: 1
verifier_config:
default_verifiers: [verifier]
audit_sample_rate: 0.0
tasks:
- id: qa-audit
title: 'WFDASH full-stack QA health check'
prompt: |
Conduct a comprehensive QA health-check of the WFDASH Next.js app at
/Volumes/EXT/DEVELOPMENT/WFDASH_NEXTJS. The user asked: "look through my
app and see if everything looks in order." Produce an honest, evidence-
based verdict — do not pad with nits to look productive.
## Step 1 — Load the skill first
Load the `Ultimate Debugger and QA Engineer` skill and follow its
Part 3 workflow + Part 2 bug-report template (P0–P3). Do not skip this
step. The skill defines the report structure you must use.
## Step 2 — Read context before auditing
- `AGENTS.md` (root): CRITICAL note — "This is NOT the Next.js you know"
/ Next 16 has breaking changes. Verify any Next-16 API claim against
`node_modules/next/dist/docs/` when uncertain.
- `CLAUDE.md` — imports AGENTS.md.
- `package.json` — full dependency list.
- `docs/AGENT_INSTRUCTIONS.md` and `docs/SHIPPED.md` — project
conventions and what was recently shipped.
- Last 30 commits via `git log --oneline -30` to see what changed
recently (currently on branch `minimax_ui_rev1`).
Read agent memory if helpful (the user works with shadcn/ui + Base UI +
PocketBase — past lessons live there).
## Step 3 — Coverage matrix (do not skip any)
Audit, in order, with concrete file:line refs for each finding:
1. **Build & type health** — Run `npm run lint` and `npx tsc --noEmit`.
Report exact counts of errors and warnings, then group by file.
Don't speculate — use real output.
2. **Test health** — Run `npx vitest run --reporter=basic`. Report
pass/fail counts and any broken tests. Don't fix tests.
3. **Next.js 16 compliance** — Spot-check `src/app/**/page.tsx` and
`src/app/api/**/route.ts` for Next-16 patterns (async params,
async cookies, server actions shape, `use cache` vs
`unstable_cache`, route handler exports, `revalidate`). Flag
anything that looks like a pre-16 pattern.
4. **React 19 + shadcn/ui + Base UI** — Spot-check 3–5 components in
`src/components/` for: hooks-rules violations, missing `key` on
lists, controlled-input sync issues (see memory: "modal form state:
use a key, not a useEffect sync"), uncontrolled warnings,
shadcn/ui vs Base UI vs Radix primitive consistency.
5. **PocketBase integration** — Review `src/lib/pocketbase.ts`,
`src/lib/pb-helpers.ts`, `src/lib/server/`, and the API routes that
hit PB. Check filter syntax, expand usage, file URLs in no-auth
mode, 400/404 patterns (see memory: pocketbase).
6. **PocketBase migrations** — Look at `scripts/migrations/` and
`scripts/setup-pocketbase.ts`. Identify destructive migrations
without a backup plan, orphaned migration files, migrations that
drift from the current `pb_schema*.json` or `pb-exports/`.
7. **Security** — Scan for: secrets in code, .env committed, missing
auth on sensitive API routes, raw user input passed to PB filters
without sanitization, XSS via `dangerouslySetInnerHTML`, missing
CSP / security headers in `next.config.ts`.
8. **Performance** — Check `src/components/leaflet-markers.ts`,
`src/lib/pdf/*`, large bundle imports (e.g. `lucide-react` deep-
imports?), missing `dynamic()` for heavy client components, large
images in `public/`.
9. **Accessibility** — Spot-check 3 form components for: label
associations, error message wiring, focus management on dialogs,
keyboard escape on modals.
10. **E2E coverage gap** — Read `e2e/*.spec.ts`. Compare to the page
surface in `src/app/**/page.tsx`. List pages with no E2E coverage.
11. **Dead code / stale TODOs** — `grep -rn "TODO\|FIXME\|XXX\|HACK"
src/ scripts/` and check whether each is recent and actionable.
## Step 4 — Write the report
Save to `docs/qa-review.md` using the skill's Part 3 "Debugging Report"
structure:
- Summary (1 short paragraph — overall health)
- Issues Identified (P0 → P3 order)
- What's Working Well
- Recommendations
- Next Steps (which of these to tackle first)
For each issue: Severity, Location (file:line), Reproduction,
Root Cause, Recommended Fix, Verification.
## Constraints
- **Read-only audit.** Do NOT modify source code, do NOT delete test
data (no `rm`, no DELETE on PB records), do NOT commit anything.
- Do not run `npm run dev` or Playwright — static audit only.
- If a finding needs a deeper runtime repro you can't safely run,
mark it `[needs-runtime-verify]` and explain.
- Cap total findings to ~30 most actionable. Group trivial nits
under a single "Nits" section with file:line only.
- The user wants to know if "everything looks in order" — be honest.
If it does, say so clearly. Don't pad.
## Report back
Send a short message containing:
- Report path (docs/qa-review.md)
- Total findings by severity (e.g. "2 P1, 5 P2, 8 P3")
- Top 3 most critical issues
- One-line overall verdict: "in order" / "minor issues" / "needs fixes"
assigned_to: coder
verified_by: verifier
verify_prompt: |
Independently audit the QA report at `docs/qa-review.md`.
## Step 1 — Run the gates yourself (don't trust the producer's claim)
- `cd /Volumes/EXT/DEVELOPMENT/WFDASH_NEXTJS && npm run lint 2>&1 | tail -50`
- `cd /Volumes/EXT/DEVELOPMENT/WFDASH_NEXTJS && npx tsc --noEmit 2>&1 | tail -50`
- `cd /Volumes/EXT/DEVELOPMENT/WFDASH_NEXTJS && npx vitest run --reporter=basic 2>&1 | tail -60`
Compare actual results to the report's claims. If the report says
"lint clean" but `npm run lint` produced errors, that's a FAIL.
## Step 2 — Spot-check 5 findings
Pick 5 findings at random (mix of P0–P3 if available). For each:
- Open the cited file at the cited line.
- Verify the bug/issue is actually present as described.
- Verify the recommended fix is reasonable and not a band-aid.
- Note any obvious miss — e.g. a clear P0 issue the report skipped.
## Step 3 — Severity sanity
Common over-classification to flag:
- Cosmetic typo graded as P1.
- "Code smell" graded as P0 without runtime impact.
- Deprecated-but-still-working API graded as P0.
## Step 4 — Coverage check
Did the report cover: build, types, tests, Next 16 compliance, PB
integration, PB migrations, security, performance, a11y, E2E gaps,
dead code? If any of these are missing, FAIL with specifics.
## Output
Write verdict to `docs/qa-review.verify.md`:
```
VERDICT: PASS | FAIL
Gates I ran: lint, tsc, vitest (paste tail outputs)
Findings I spot-checked: 5 — all hold / N failed (list)
Severity sanity: pass / fail (notes)
Coverage: complete / missing (X)
Missed P0/P1 I found: (list, or "none")
Recommendation: accept | accept-with-fixes | retry
```
Be adversarial but fair. If the report is honest and the app
genuinely looks in order, PASS it.
max_retries: 2