The Aug 2026 pump sheet from The Hills (and any future sheet from
the same source) uses different column names and a different date
format than the legacy CSV the import route was built for. Adapts
the route so the same endpoint accepts either Excel (.xlsx) or CSV,
maps the new column names onto the existing FUEL_LOGS schema, and
persists their 'Latest Consumption' column verbatim.
What's new
• SheetJS (xlsx) replaces PapaParse for parsing. Single code path
handles both .xlsx and .csv uploads.
• Resolves columns case-insensitively against an alias table so
legacy CSVs (vehicle_description, odo_reading) still work
alongside the new shape (Date Logged, Vehicle QR, Km's/ Hours,
Litres Pumped, Current Site, Latest Consumption).
• Date parser: DD/MM/YYYY first (with leading-space tolerance),
falls back to YYYY-MM-DD, then to Excel serial numbers.
• Vehicle match: QR code first (clean identifier in the source
data), falls back to description text for legacy CSV.
• Site match: by sites.name (one round-trip, cached for the
import). project_code is derived from the resolved site; falls
back to the vehicle's project_code.
• Latest Consumption persisted as a new FUEL_LOGS field
(latest_consumption, number, optional). Empty source cells
stay empty rather than getting coerced to 0.
PocketBase change
• scripts/migrations/2026-08-24-fuel-logs-latest-consumption.py
adds the new field. Idempotent; follows the pattern of the
2026-08-19 migration. Run it with PB_ADMIN_EMAIL +
PB_ADMIN_PASSWORD before importing the new sheets, or the
import will fail with 'latest_consumption is not a field'.
UI
• Import button now accepts .xlsx and CSV; SAMPLE_HEADERS +
SAMPLE_OPTIONAL show the new column expectations and the
DD/MM/YYYY date format hint.
• fuel-view.tsx shows a 'Consumption (src)' column on the row
table so operators can eyeball what their fuel team sent
against the app's recomputed variance.
Out of scope (intentionally untouched)
• vehicles, sites, data_import_batches and every other
collection — no schema changes there.
• PO / timesheets / delivery-notes imports.
• fuel/page.tsx business logic (variance calc, vehicle summaries).
Verified
• Dry-run against 'The Hills Diesel 1-24 August.xlsx': 134/134
rows parse cleanly, 0 errors. 12/134 rows correctly carry an
empty Latest Consumption (matches the source).
• vitest: 20/20 fuel-import-parse + 35/35 PO-schema tests pass.
• tsc --noEmit clean.
• One pre-existing failure in
src/app/api/purchase-orders/process-batch/route.test.ts
unrelated to fuel; confirmed failing on main before this change.
PO ingestion was rejecting valid line items with qty = 0 (base
corrections, pending test rows) because the schema strictly required
qty > 0. Loosen the guard to qty >= 0 while keeping rate, disc_percent
and vat_percent strict. Also tighten against non-finite numbers
(NaN, +/-Infinity) via Number.isFinite — the previous !(qty > 0) guard
let Infinity slip through.
- Schema: qty >= 0, finite; rate/disc/vat rules unchanged
- Tests: add positive case for qty = 0; add negative + NaN rejection
- 35/35 schema tests pass; tsc clean
PB-SDK puts the HTTP status on err.status (e.g. 404), not in
err.message. The previous /404/.test(err.message) check never matched
PB's actual 404 message ('The requested resource wasn't found.'), so
every 404 fell through to the generic 'lookup_failed' 500.
This wouldn't have caused the user's current 500 since their batch row
exists, but it would mask any future 'batch row not found' race as
'lookup_failed' and prevent the correct 404 + retry flow.
When the route returns 500 with 'lookup_failed', include the failing
step name and the underlying error string in the response body so we
can pinpoint which catch block is firing (get_batch / dedup / match_site).
Production diagnostic - safe to revert once root cause is found.
- Enhanced VehicleSummary and PlantRow interfaces to include qrCodeId and costPerHour properties.
- Updated FuelView and PlantView components to display QR code information in the respective tables.
- Adjusted data fetching in FuelPage and VehicleDetailPage to retrieve and handle new fields from the API.
- Modified site detail views to reflect the unified VEHICLES collection, ensuring accurate data representation for plant utilization.
This update improves the visibility of QR codes and cost information for better tracking and management of vehicles and equipment.
- Normalise po_number (strip PO- prefix, uppercase, collapse whitespace)
via the new normalizePoNumber helper on the dedup compare AND on the
stored purchase_order_number. All new POs go forward in canonical form.
- Add a clearFile option to finalizeBatchAsProcessed and call it with
{ clearFile: false } on the dedup path. The duplicate upload's bytes
stay on the batch row for audit / debugging; the original PO's file
is unchanged. The success path still moves the file from batch -> PO
and clears the batch (current canonical-home behaviour).
- Response surfaces both po_number (normalised, used for dedup/storage)
and po_number_extracted (the raw AI output) for downstream debugging.
Tests: 15 in this file; full suite still 88/88 green.
SHA-256 the file bytes, look for a recent (30-day) processed
data_import_batches row with the same hash, and short-circuit with
200 { duplicate: true, existing_batch_id, existing_po_id } when found.
Saves an AI extraction call + a cron tick for byte-identical re-uploads.
Stores file_hash on the new batch row so future re-uploads can match
against it.
Backed by a new index (idx_batches_file_hash on file_hash, status)
and a new file_hash column (text, 64 chars) on data_import_batches
applied via PB admin API.
Tests: 11 in this file (covering happy path, byte-identical re-uploads,
hash collisions, dedup-query failures, and existing edge cases).
Strips vendor prefixes (PO-, PO#, P.O., etc.), collapses whitespace,
uppercases. Used by the upload-route hash dedup and the process-batch
dedup-by-po_number check so OCR variants ('PO-1234' vs 'po 1234') fold
to the same canonical form.
Tests: 33 in this file (23 pre-existing for validatePurchaseOrderPayload,
matchSite, computeTotalValue + 10 new for normalizePoNumber).
The catch block on batch_create only logged the PB error server-side
and returned a generic 500 'batch_create_failed', which forced
operators to dig through Coolify logs to find the real cause (e.g.
mime type not allowed, missing field). Now the response includes
pb_status, pb_message, and pb_data so the actual reason is visible
in the browser's Network tab and the dashboard's error toast.
Co-Authored-By: Claude <noreply@anthropic.com>
Captures the async-queue redesign of the PO/DN import pipeline (see
spec + plan) and rewrites the n8n implementation doc to match the
new cron-driven architecture (workflow 005iI4jX5mq9DsTR). Also
gitignores cooler.env so the local-only deployment reference is no
longer tracked.
Co-Authored-By: Claude <noreply@anthropic.com>
Both Next.js /api/purchase-orders/process-batch and the n8n cron
workflow need a PocketBase admin token (generated via PocketBase
admin > Settings > Admins). The existing template only documented
email+password auth.
Add a new 'POCKETBASE (Next.js process-batch + n8n cron)' section
with the token variable, the generation instructions, and the
warning that Next.js env and the n8n workflow's hardcoded token
value must match.
Co-Authored-By: Claude <noreply@anthropic.com>
After Task 7's upload returns 202 { batch_id, status: 'queued' }, the
button now polls /api/data-import-batches/[id]/status every 2s, renders
a state pill (queued/processing/processed/failed/dead), and provides a
Retry button that calls POST /retry for failed/dead batches. Polling
stops on terminal states and on unmount; no double-submit.
Co-Authored-By: Claude <noreply@anthropic.com>
POST /api/purchase-orders/upload no longer forwards the file to n8n
synchronously. It now validates size/MIME, writes a data_import_batches
row (batch_type='purchase_orders', status='pending', attempts=0) with the
uploaded file attached, and returns 202 { batch_id, status: "queued",
filename } immediately. n8n's 5-minute cron claims pending rows and calls
back to /api/purchase-orders/process-batch; the browser polls
/api/data-import-batches/[id]/status.
Adds route tests covering validation guards, the queued 202 contract,
the batch-create failure path, and that no synchronous fetch to n8n
happens.
Co-Authored-By: Claude <noreply@anthropic.com>
n8n POSTs extracted PO JSON + batch_id after downloading the file
from PocketBase. The route validates the payload, dedups by po_number
(idempotent), resolves the site, moves the batch's file onto a new
purchase_orders row + items, and finalizes the batch with
status=processed, processed_record_id, processed_at, original_file
cleared.
Co-Authored-By: Claude <noreply@anthropic.com>
POST /api/data-import-batches/[id]/retry resets a failed or dead batch
back to pending (attempts=0, last_error/next_retry_at cleared) so the
next cron sweep picks it up. Non-retryable states return 409 with the
current status; legacy rows with no status default to processed.
Co-Authored-By: Claude <noreply@anthropic.com>
GET /api/data-import-batches/[id]/status returns the current batch
state for the upload button's 2-second polling loop. Returns 400 on
bad id format, 404 if missing, 500 on lookup failure. Defaults
missing status to 'processed' for legacy timesheets/fuel-logs rows.
Co-Authored-By: Claude <noreply@anthropic.com>
Extends the data_import_batches collection for the PO/DN async-import
queue redesign (see spec docs/superpowers/specs/2026-08-19-po-import-queue-redesign.md
and plan docs/superpowers/plans/2026-08-19-po-import-queue-implementation.md).
Adds 8 fields:
content_type, status (select: pending|processing|processed|failed|dead),
attempts, last_error, last_attempt_at, next_retry_at, processed_at,
processed_record_id
Extends the batch_type enum to include purchase_orders and delivery_notes.
Adds index idx_batches_pending on (status, batch_type, next_retry_at) for
the cron query.
Idempotent: safe to re-run.
Co-Authored-By: Claude <noreply@anthropic.com>
- Implement improved error handling for the purchase order upload process, ensuring that failures from n8n are properly surfaced to the user.
- Introduce a timeout mechanism for the n8n fetch request to handle potential delays more effectively.
- Refine the validation logic for the purchase order payload to ensure stricter adherence to expected formats and values.
- Purchase order upload route now returns 502 n8n_rejected when n8n
responds with a non-2xx status, instead of silently claiming success.
- Add a 10s AbortController timeout on the n8n fetch, returning 504
n8n_timeout on expiry (502 n8n_unreachable on other network errors).
- Extract ALLOWED_MIME and MAX_FILE_BYTES from upload/route.ts into
src/lib/purchase-order-schema.ts so the upcoming ingest route can
share the same validation constants.
- Updated the documentation to reflect the latest changes in the Purchase Order upload and n8n ingestion process.
- Clarified the data flow from browser to Next.js API endpoints and through to PocketBase.
- Ensured that the documentation aligns with the recent implementation and design specifications.
Adds design spec and n8n handoff doc for the new PO upload flow:
browser -> Next.js /api/purchase-orders/upload -> n8n OCR ->
Next.js /api/purchase-orders/ingest -> PocketBase via pbAdmin().
Co-Authored-By: Claude <noreply@anthropic.com>
- Introduced new data fetching for plant and income claims, enhancing the financial overview.
- Added calculations for daily costs and integrated weekly and monthly income vs expenses roll-ups.
- Implemented plant utilization metrics to track equipment efficiency.
- Updated the SiteDetailData interface and SiteDetailView component to accommodate new data structures and display improvements.
- Updated the Dashboard page to improve card styling with a warm-amber glow strip and adjusted icon presentation for better visual appeal.
- Expanded the Sidebar navigation by adding new entries for "Sites," "Workforce," "Plant & Equipment," "Fleet & Fuel," and "Purchasing," each with corresponding icons and descriptions to enhance user navigation and access to key features.
- Added `pocketbase` as a dependency in `package.json` and `package-lock.json`.
- Updated the `dev` script in `package.json` to run on port 6004 instead of 3000.
- Removed unused API routes: `src/app/api/health/route.ts`, `src/app/api/mcp/route.ts`, `src/app/api/upload/route.ts`, and their associated logic to streamline the codebase.
- Cleaned up `package-lock.json` by removing unnecessary peer dependencies and ensuring consistency in the dependency tree.
The previous commit dropped real data for The Hills ext 12
(sites, vehicles, employees, orders, plant hire rates, P&G
expenses) so the dashboard had something to render during design
iteration. With PB about to be wired, that data now belongs in
PocketBase rather than in the source tree.
This commit empties the data layer back to zero so the dashboard
falls into its empty-state UI on every page:
- src/lib/mock-data.ts SITES_DATA, SITES, SUPPLIERS, VEHICLES,
EMPLOYEES, ORDERS, FUEL_READINGS,
PLANT_HIRE_RATES, PG_EXPENSES → []
Header comment updated to point future
contributors at lib/pb.ts as the next
step.
- src/lib/uploads.ts UPLOAD_HISTORY → []. Acceptance rules
(mime / extensions / size limits) kept so
the Uploads page still shows format hints.
Type definitions, helper functions and date utilities are untouched
so all routes continue to render (HTTP 200 across the board). The
sidebar, dashboard, fuel, orders, employees, sites and uploads pages
all gracefully render their empty states.
When PocketBase is wired:
1. Install 'pocketbase' (npm)
2. Build src/lib/pb.ts with admin-authed server client + public
client
3. Replace the imports in pages like
import { SITES_DATA, VEHICLES, ... } from '@/lib/mock-data';
with calls like
import { getServerPB } from '@/lib/pb';
const records = await getServerPB().collection('sites').getList();
4. Keep the empty exports here as a final fallback so the dashboard
still works if PB is ever unreachable.
The sidebar was hidden until 1280px wide, which meant laptops and
small desktops between 1024-1280px (incl. iPads in landscape, many
Windows laptops, Chromebooks) saw the hamburger instead of the
persistent sidebar. That felt wrong on what was clearly a desktop.
Dropping to 'lg' (1024px):
- Phone portrait (< 768 px) → hamburger ✅
- Tablet portrait (768-1024 px) → hamburger ✅
- iPad landscape / laptop (1024 px+) → sidebar ✅
- 932 px iPhone landscape cap is below 1024 px, so the hamburger
still kicks in for all landscape phones.
Changed 4 occurrences: 3x 'xl:hidden' → 'lg:hidden', 1x
'hidden xl:block' → 'hidden lg:block' on the <aside> element.
Adds the four variables the PocketBase integration will use:
- NEXT_PUBLIC_POCKETBASE_URL — client-safe PB URL (no
NEXT_PUBLIC_ on the auth values!)
- POCKETBASE_ADMIN_EMAIL — server-only admin login
- POCKETBASE_ADMIN_PASSWORD — server-only admin password
- POCKETBASE_ADMIN_TOKEN — optional pre-issued token
The real .env.local stays out of git (.env.local is gitignored);
developers copy this file to .env.local and fill in their credentials.
Data wired from the site cost sheet, fuel log, wage schedule and two
PO PDFs the user shared:
- src/lib/types.ts VehicleType expanded (Roller, TLB,
Pickup, Water Truck, Compressor,
Mixer) so plant types map cleanly to
the source data. Vehicle gains
optional kmPerL and expectedLPerHour
fields for the mixed L/h + km/L
allowance model.
- src/lib/mock-data.ts SITES_DATA[0] = The Hills ext 12,
contract value R24,988,663, contract
code C26/09, 8-month duration.
SITES, SUPPLIERS updated.
VEHICLES populated with 47 entries
covering rollers, TLBs, excavators,
graders, loaders, tipper trucks,
water trucks, pickups, mixers and
auxiliary equipment (compressors,
crushing machine).
EMPLOYEES populated with 15 permanent
+ 53 casuals, all at the project's
45h/week standard.
PLANT_HIRE_RATES populated with 32
entries matching the cost sheet
(R185–R680/h depending on equipment).
PG_EXPENSES populated with all 6 P&G
monthly lines (container, security,
salaries, overheads, insurance,
site facilities = R567,403.50/m).
ORDERS populated with MEGCPO9199
(PPE, Delivered) and MEGCPO9165
(aggregates, Partial).
FUEL_READINGS starts empty — drop
CSV-imported rows in once the depot
system is wired.
- src/lib/uploads.ts UPLOAD_HISTORY seeded with the GRN
attached to MEGCPO9199.
- src/lib/burn.ts New helper. burnBreakdown() joins
plant hire rates with the vehicles
list, multiplies each hire rate by
the standard 45h/week, adds weekly
P&G share, and surfaces both weekly
and monthly totals.
- src/lib/sites.ts summarizeSite + summarizeAllSites
now take PlantHireRate[] and
PgExpense[] so the site summary
exposes weeklyBurn / monthlyBurn /
weeklyPlantHire / monthlyPg etc.
- src/app/dashboard/page.tsx New 'Burn rate — weekly vs monthly'
panel. Two side-by-side cards show
weekly + monthly totals with plant
hire vs P&G breakdown, plus a
'Cost breakdown' card on the right.
- src/app/dashboard/sites/* Pass the new hireRates + pgExpenses
arrays through to the summary helpers.
Notes:
- The 'Spent to date' KPI on the Sites list / detail now sums PO
value (since there are no fuel readings yet). Once depot data is
ingested, it will pick up the actual R25.65/L burn automatically.
- Tank capacities + odometer start values are sensible estimates
for the equipment types — adjust per-unit when you have real
fleet specs.
- Weekly hours per employee default to 45h so the workforce
trends look populated; replace with timesheet imports as soon as
those land.
Wipe all seeded mock data so the user can drop in real numbers:
- src/lib/mock-data.ts SITES_DATA, SITES, SUPPLIERS, VEHICLES,
EMPLOYEES, ORDERS, FUEL_READINGS → []
Type definitions and helper functions
preserved; the FUEL_READINGS generator is
preserved as a commented block ready to
re-enable.
- src/lib/uploads.ts UPLOAD_HISTORY → []. Acceptance rules
(mime / extensions / size limits per kind)
kept so the UI still shows format hints.
While verifying empty states, caught a divide-by-zero on the Sites
list page: 'Workforce this week' KPI rendered as 'NaN% of total
contract value' when there are zero sites. Guard the divide.
All pages now render in their empty-state UI — KPIs show 0, charts
draw empty axes, tables say 'No X match the current filters', upload
history cards say 'No uploads yet'. Typecheck clean, all routes
return 200, no runtime errors.
The previous commit used http:// for the Umami script. Any browser
loading the dashboard over HTTPS would silently drop the HTTP
request as mixed content, so the tracker would never fire in
production. Switch to https:// — the SSLip.io subdomain is
already served over TLS.
