Adds /dashboard/plant/map: a Leaflet view of every vehicle scanned from
the MEGA Clock kiosk, plotted at its last known position.
Schema
- vehicles.last_latitude / last_longitude / last_scanned_at /
last_location_accuracy_m denormalised off fuel_logs so the map page
renders every vehicle in a single query. Same best-effort pattern as
attention_dismissed_at; updated by /api/clock/vehicle-scan after every
successful scan. PB init defaults new number cols to 0 (not null) —
filter uses 'last_latitude > 0 || last_latitude < 0' to mean 'real fix'.
- Indexed via idx_vehicles_last_scanned.
Map UI
- Custom SVG markers per unit_type: car silhouette for fuel, tractor
silhouette for plant, 24x24, drop-shadowed.
- Clustering via react-leaflet-cluster over Leaflet.markercluster.
Cluster icons colour-ramp with size (green->sky->amber->rose) and
scale 32->54px.
- Three-button base-layer switcher: Street (OSM), Dark (ESRI Dark
Gray Canvas — CartoDB watermarks tiles without a key), Satellite
(ESRI World Imagery). Last choice persists in localStorage.
- Site pins: small purple map-marker divIcon, click flies the map to
zoom level 15.
- Empty-state is a floating chip on the map rather than a full-page
card so the map is always visible.
APIs
- /api/clock/vehicle-scan: stores the GPS fix on the fuel_logs row
AND denormalises onto the vehicle when present; missing-coord
scan leaves last_* untouched.
Sidebar: new Map entry under Plant group.
Updated the vehicle scan API to include optional GPS coordinates (latitude, longitude) and location accuracy. The API now validates these fields, ensuring both latitude and longitude are provided together and within valid ranges. Enhanced error handling for invalid inputs and updated the API reference documentation accordingly. Additionally, modified the FleetScanner component to capture GPS data and reflect its status in the UI.
dn-snapshot prints row counts across the delivery-note collections. The
two that matter are delivery_notes and delivery_note_items: the staged
pipeline must never move them on its own, so a reading before and after a
test says plainly whether anything reached the ledger.
dn-board renders the inbox board as text, which is how the pipeline can
be read over SSH or while the app is mid-deploy.
Both are read-only. The other scripts written while diagnosing this work
were one-off probes and are not kept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Picking the right order did nothing. The route stored the corrected
number, the re-run rebuilt the candidate from its page reads, and
po_number_raw went back to the original misread — so the operator
answered the same question again, having been told it was saved.
Nothing in dn_batch_notes distinguished a value a person chose from one
the model read, so the runner could not tell a correction from stale OCR
and overwrote both. A manual_overrides field now records the decision,
and the runner applies it before resolving rather than after, so the
corrected number is what the PO is matched on.
Only fields a run rebuilds are recorded this way. Quantity, unit and the
masses survive on their own; page ranges are the assembler's answer and
are corrected through the assembly route, which keeps the whole batch
consistent. An override must carry a value — clearing a box is not a
decision that a document has no number.
Verified end to end on a slip whose order number was read with a zero for
the letter O: the correction survived the run, resolved the order, moved
the match from fuzzy to exact, and left the document ready to post.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three faults stopped the staged pipeline from ingesting anything once it
met a live PocketBase rather than a test double.
PocketBase treats 0 as blank for a required number field, and two staged
fields legitimately hold zero: dn_batch_sources.ordinal is 0 for the
first source in a batch, and dn_batch_pages.region is 0 for a whole-page
read. Both creates were rejected with validation_required, so the first
file of every upload failed and no page ever persisted. The migration now
declares them optional, and pb-relax-dn-zero-fields.py relaxes instances
that already have them required.
The staged runner verifies its batch lease and commits the protected
write in one transaction, which needs PocketBase's batch API. That is off
by default, so the runner failed closed with
dn_pipeline_transactions_required and no deployment note explained why.
pb-enable-batch-api.py turns it on and raises the 3s transaction timeout,
which is tight for a commit carrying page images.
compare-dn-pipelines used top-level await while resolving as CJS, so the
command in the rollout runbook always failed to transform. Renamed .mts.
The tests missed all of this because they mock PocketBase, and the mock
accepts what the real server rejects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Duplication happens per NOTE, not per batch. The same stack of Vula
invoices was scanned three times in overlapping groupings — 13, 16 and 14
pages with three different file hashes, so upload-time hash dedup could
not see it. Each of those batches mixes notes that promoted, notes
duplicating another batch, and invoices unique to it: vdn074468 alone
carries four that exist nowhere else.
So no batch-level action is correct. Dismissing one wholesale either
discards unique invoices or zeroes a counter for notes already on the
ledger. Adds POST /api/dn-batches/[id]/notes/[noteId]/dismiss, which
stands one note down, refuses to touch a promoted one, and parks the
batch only once nothing is outstanding.
Also fixes the batch dismiss route, which wrote notes_promoted: 0
unconditionally. A batch that had promoted 8 of 13 notes would report
nothing while 8 delivery notes sat in the figures people pay from — the
same board-versus-ledger disagreement already fixed in run.ts. It now
counts what actually reached the ledger and says so in the reason.
929 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Updated the aliasing mechanism to utilize article numbers and barcodes for product identification, addressing issues with ambiguous material codes. The PageItem now requests the article number alongside the product code, ensuring accurate transcription of truncated descriptions. This change improves the resolution process by prioritizing product codes and refining the lookup strategy to reduce ambiguity in purchase orders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The alias table shipped keyed on material_code alone. Live data says
that cannot work:
- 16 of 24 purchase orders carry ONE material_code across every line.
MEGCPO9239 is "513095" sixteen times.
- Code "580070" spans 24 different products; "513080" spans 16.
An alias keyed on it would have resolved to every line on the order and
been permanently ambiguous — for Builders, the supplier whose truncated
descriptions need aliases most. I verified every PO item HAD a
material_code and never checked whether it discriminated.
Also found, by reading the source scan: the truncation is the supplier's
printer. Builders' form prints descriptions in a fixed ~20-character
column that wraps twice then stops ("Grip Fibreglass Meas" / "uring Tape
Orange a"). Extraction was faithful; nothing upstream can recover it.
But every line also carries an article number and a barcode, which are
never clipped and which we were discarding.
So:
- PageItem gains product_code, and the prompt asks for the Art No /
SKU / Prod Code column plus an explicit instruction to transcribe a
clipped description verbatim rather than guess its ending.
- Aliases record slip_product_code and po_description_key as well.
Lookup tries product code before description; resolution tries the
PO line description, then material_code ONLY when it narrows to a
single line on that order, then falls through to text matching.
- The resolve-item route no longer rejects a PO line for having no
material_code, which was based on the same wrong assumption.
dn_item_aliases had 0 rows, so there was nothing to migrate.
916 tests. Live regression unchanged: 25 matched, 3 ambiguous, 3 none,
zero wrong; 0 of 24 POs vulnerable to nonsense.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Batch 5qzcv6tvgb6cqks promoted 3 of 3 notes and still read needs_attention
because page 4 was a tax invoice. Stray pages become open questions that
can be dismissed, and every detected note must still promote.
Co-Authored-By: MiniMax <noreply@minimax.io>
Five of fourteen blocked batches were tax invoices or collection notes,
which have no delivery note number. Also stops reading a composite
reference like '22433700 / 2025344290' as three separate documents.
Co-Authored-By: MiniMax <noreply@minimax.io>
The script read dn_batch_notes.date, which does not exist on that
collection — the backfill would have silently set nothing while
reporting success. My plan specified the wrong field.
Now sources from delivery_notes.date_received via the batch link (11 of
34), falling back to the scanner-generated filename date (17), leaving 6
empty rather than inventing one. Dry-run by default, and it reports the
source per batch so the numbers can be checked before writing.
Applied: 17 existing fields preserved, uploaded_at added, 28 of 34
batches backfilled. -uploaded_at now sorts newest first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PocketBase's created is empty on every dn_batches record, making the
board's -created sort a no-op and the tick's oldest-first meaningless.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The earlier commit made the script include these fields for from-scratch
creates, but didn't repair the existing live dn_batch_notes collection.
The repair path (add_fields) covers that case so re-running the script
against an already-deployed PB collection also adds them.
The dn_batches and dn_batch_notes collections were created without
PB's standard 'created' / 'updated' date fields, which broke any
query that sorted on them (the tick route, the inbox page).
Live PB was patched inline by the Task 16 verification; this commit
makes the script that creates these collections include the fields
from the start so a fresh deployment doesn't trip on the same bug.
Rewrites qty_received into the purchase order line's unit, recovering the
slip's own unit from the condition_notes format the OCR callback wrote.
Rows with no recorded unit are left untouched and reported.
The new PWA icon matches the supplied reference: dark background, a
clock face with an orange upper-right arc and hands at 12 and 4
o'clock, sitting on an isometric cube whose two visible faces each
have the MEGA 'M' carved into the background colour, with two small
orange base wedges under the cube.
When the /clock URL is shared (Slack, iMessage, etc.) the static
1200x630 opengraph-image.png is picked up automatically and renders
the icon alongside the 'MEGA Clock' wordmark and tagline, via
openGraph + twitter metadata added to the route.
The migration's docstring pointed at a spec file that was never
written for this feature (the design was worked out via conversation,
not a separate spec doc) — points at the plan instead.
.pyc files are generated, not source -- machine/version-specific,
produce spurious diffs, and a routine __pycache__ cleanup can delete
tracked files as a side effect (hit this earlier in the session). Added
__pycache__/ to .gitignore.
Also fixed a boundary error in the plan's checkbox sweep: Task 17's own
6 steps were skipped when checking off Task 18's range. Task 17 has been
done and independently verified since commit 29ee2cc.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pb-bootstrap-schema.py -- the script a fresh PocketBase instance would
run against -- had fallen ten migrations behind and still described the
pre-restructure world. Rewrote it field-by-field against a live
production dump (17 collections, every field/required-flag/relation/
index verified). Found and fixed two pre-existing gaps unrelated to this
restructure while verifying: fuel_prices and income_claims were entirely
absent from the bootstrap script.
pb-relax-rules.py's collection list updated to match.
No local PocketBase binary is available in this environment, so the
plan's "bootstrap fresh and diff against production" verification step
was replaced with a manual field-by-field comparison against the live
dump -- same goal, different mechanism, documented as a substitution
rather than silently skipped.
Added site scoping to the one agent tool that still lacked it
(list_imports, backed by loadImportBatches) -- every other list_* tool
already had it. Preparation for the future per-user site-allowlist work,
not part of this restructure itself.
Rewrote docs/data-model.md and docs/data-flows.md end to end: the
site-attribution table, the collection reference, the five ingestion
pipelines, and the schema-drift section that was describing the exact
problem this task fixes. Every remaining data_import_batches reference
in both docs is now historical context, not a current description.
This is the last task of the ingestion restructure
(docs/superpowers/plans/2026-09-11-ingestion-restructure.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
timesheet_entries, fuel_logs and delivery_notes had no indexes at all --
fine at today's row counts, not once site count grows toward the 18-20
target. Added 8 indexes across fuel_logs, timesheet_entries,
delivery_notes, purchase_orders and plant_hours_log, unioned into each
collection's existing index array rather than replacing it.
Verified both pre-existing UNIQUE indexes (idx_po_number, load-bearing
for the placeholder-PO-number scheme; idx_plant_hours_log_qr_start,
prevents duplicate weekly imports) are byte-for-byte unchanged.
Idempotent: a second run touches nothing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The irreversible step of the ingestion restructure. Guards proved clean
(0 no-site rows across all 8 target collections, PLANT empty, all 27
migrated documents present, no row still depending on the old batch
link), full schema + row-data backups taken to /tmp/pb-backup-2026-09-15
first, then applied.
Dropped project_code from employees, vehicles, fixed_costs,
income_claims, purchase_orders, delivery_notes, fuel_logs and
plant_hours_log. Dropped qty_outstanding from purchase_order_items.
Deleted the PLANT collection (0 rows, 0 code references — a deprecated
alias for vehicles) and DATA_IMPORT_BATCHES (86 rows, superseded by the
five document collections built across Tasks 6-15).
PocketBase refused the first attempt: it will not delete a collection
while any relation still points at it, and fuel_logs, plant_hours_log
and timesheet_entries all still carried a batch_reference relation to
data_import_batches that Task 7's migration made obsolete but never
removed. Confirmed live that nothing in src/ reads or writes that field
any more (only three stale type declarations in pb-types.ts), extended
the drop script to remove it from all three collections, re-ran, and the
deletion succeeded.
Cleaned up the now-dead DataImportBatchRecord interface and BatchType
union from pb-types.ts; BatchStatus survives as the shared type every
document/queue collection's processing_status now uses independently.
Final collection count: 17, down from a peak of 20 mid-restructure (16
original + 3 new document collections - PLANT - data_import_batches -
2 renames counted once each). Every remaining collection's fields match
what code actually reads and writes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
project_code was a text copy of the parent site's code, denormalised
onto 8 child collections that each already have a real site relation.
Written once at insert, never updated -- docs/data-model.md documents
the drift risk. On purchase_orders it never even held a real project
code: production had requisition numbers, site names, and the PO's own
number in that field.
Every read site now resolves the code through site/expand.site/siteById
instead, reusing each page's existing site map where one already
existed. Every create() call stops writing the column. The chat agent's
list_imports tool no longer reads data_import_batches -- it now reads
timesheets, fuel_sheets, plant_hours_sheets, purchase_orders and
delivery_notes filtered by processing_status, which was necessary before
the pool can be dropped (Task 20 would otherwise 404 the agent).
Also closed a Task 17 gap found while doing this: the agent tools' vehicle
and fuel-log handlers still resolved project code from the now-removed
vehicles.project_code column.
Wrote scripts/migrations/2026-09-15-drop-legacy-columns.py, the one
irreversible step in this whole restructure. Its guard against orphaning
fuel_logs rows checked the wrong condition -- "fuel_sheet is empty" --
which produced a false abort: 73 fuel_logs rows have no fuel_sheet
because they predate batch tracking entirely and never had a
batch_reference to begin with, so dropping data_import_batches costs
them nothing. Fixed the guard to check what it actually claims to check:
a row that HAD a batch_reference and was never relinked. Verified live
that no such row exists on any of the three affected collections.
Script run in dry-run only; not yet applied.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tightened site from optional to required on fuel_logs, timesheet_entries,
delivery_notes and purchase_orders now that every route writes it and
every historical row has been backfilled. plant_hours_log was already
required (matches docs/data-model.md's documented invariant, confirmed
against production). timesheets (the document collection) was already
required from Task 6 -- a no-op here, not a regression.
fuel_sheets.site and plant_hours_sheets.site deliberately stay optional:
those sheets can legitimately span multiple sites.
Verified: creating a fuel_logs row with no site now gets a hard 400
(validation_required) from PocketBase itself.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Task 13 moved PO creation to upload time and needed a hash-dedup window
matching the spec's "same 30-day dedup window as today" (design doc
section 4.1). purchase_orders has no working created/updated autodate
field on this PocketBase instance, and unlike delivery_notes it had no
other timestamp field either -- the first cut of Task 13 correctly found
this, but responded by dropping the time window entirely rather than
adding one. That's a real behaviour change from spec, not just a
workaround: a PO file legitimately re-sent long after the original would
have been flagged a duplicate forever, with no way to age out.
Added purchase_orders.date_uploaded (additive, mirrors how
delivery_notes already uses its own date_received) and wired it through:
stamped at create, filtered on in the dedup query.
Also brought PurchaseOrderRecord in pb-types.ts in line with what Task 1
already put on the live schema and what the upload route now writes --
processing_status/attempts/last_error/next_retry_at/last_attempt_at/
file_hash/date_uploaded were live in production but absent from the
TypeScript interface. docs/data-model.md section 7 documents this exact
gap as a standing foot-gun; closing it now while it's fresh.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Migrates the 27 batches of type timesheets/fuel_logs/plant_hours out of
data_import_batches into the three document collections created in Task
6, preserving every source file and relinking every child row.
Status is derived from actual child count, not copied from the batch
row. Production proved the stored status cannot be trusted: every one of
these batches was written status=processed at creation time, before any
insert ran.
The bug is worse than the plan's own audit found. It predicted 18
phantom-processed batches, all plant_hours. Production has 21 -- three
timesheets batches are also phantom, including two byte-identical
re-uploads of the same wage sheet that both reported success and created
nothing. This hit payroll, not just plant-hour smoke tests.
Also: computed file_hash on migrated documents while the bytes were
already in hand (closes the same dedup gap Task 1 closed on delivery
notes and purchase orders), and split the schema-additive step
(--apply-fields) from the row migration (--apply) so the reversible part
can run independently of the irreversible one. The plan claimed
idempotency via a migrated_from_batch field that doesn't exist; replaced
with an explicit refuse-if-already-populated guard rather than pretend
resumability that isn't there.
27/27 migrated. 6/6 post-migration invariants verified: 3 fuel_sheets,
19 plant_hours_sheets, 5 timesheets, 1367/247/1 children relinked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three document collections replacing data_import_batches for the
spreadsheet importers. Each owns its source file and carries honest
outcome counters (rows_total/created/failed, row_errors) — the counts the
current importers compute and then discard.
site is required on timesheets (a wage sheet belongs to one site) and
optional on the other two, which legitimately span sites since the fuel
importer resolves a site per row. cascadeDelete is false on all three, so
deleting a site cannot destroy its source spreadsheets.
Widened source_file mimeTypes beyond the plan's three entries to match
the pool being replaced. Browsers label CSVs text/csv, application/csv or
text/plain interchangeably and the importers branch on file extension,
not MIME — so the narrower list would have parsed a CSV successfully then
rejected it on write. The 2026-08-24 migration exists because that
already happened once.
Plan corrected: its TypeScript edit added a second `expand` property to
an interface that already had one, which does not compile.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RECEIVED_ITEMS -> DELIVERY_NOTE_ITEMS (55 rows): they are the lines on a
delivery note; the old name described an event, not a thing.
TIMESHEETS -> TIMESHEET_ENTRIES (1367 rows): this collection is the time
log, not a timesheet. The name is freed for the uploaded-document
collection created in Task 6.
PocketBase stores relations by collection id, so every relation survived
untouched; row counts asserted intact on both renames.
Two call sites the plan missed: src/app/api/timesheets/import/route.ts
and src/lib/ocr-result-tier.ts. Deliberately preserved the "timesheets"
string literals in pb-types.ts and the import route — those are
data_import_batches.batch_type select values, not collection names, and
changing them would have violated the select constraint.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
