Updated the PurchasingPage component to include received items data, allowing for accurate fulfillment percentage calculations based on authoritative data. The implementation now sums received quantities per purchase order item, replacing the previous denormalized field. Additionally, the needs-review delivery notes component has been improved with a modal for reviewing slips and assigning purchase orders, enhancing user interaction. The SlipPreview component has been integrated for better visual representation of delivery slips.
Updated the OCR result processing to include validation of matched PO items before determining the verification tier. This change ensures that if the local PO items fetch is empty but n8n supplied matched IDs, the tier will downgrade to draft, requiring user review. Additionally, the `resolveOcrTier` function has been modified to account for item verifiability, improving the accuracy of tier assignments based on confidence levels and PO resolution status. Unit tests have been added to cover these new scenarios.
Updated the OCR result processing to include a more robust PO matching mechanism. The new implementation prioritizes exact matches, followed by normalized comparisons, and finally employs a Levenshtein-based fuzzy match for improved accuracy. This change ensures that delivery notes are correctly linked to purchase orders, even when OCR introduces minor errors. Additionally, the behavior for handling confidence levels and PO resolution has been clarified in the documentation, enhancing overall processing reliability.
OCR is noisy. Instead of exact-then-fuzzy-fallback, always fetch
all ~15 POs from PocketBase, normalise both sides (strip non-
alphanumeric, lowercase), score each with Levenshtein similarity,
and return the best match above 0.75 threshold. Single-char
insert/delete (MEGCP09239 vs MEGCPO9239) scores ~0.9 — matches.
Completely different POs (similarity <0.5) don't match.
Plus the existing 'return 200 with empty items' fallback if no
match — keeps the workflow loop alive even when OCR is bad.
OCR sometimes mis-reads PO numbers (e.g. 'MEGCP09239' instead of
'MEGCPO9239') — when that happens the PO lookup 404'd, which
terminated the n8n workflow loop even with onError set on the
HTTP node.
Return 200 with {po: null, items: [], note: 'po_not_found'} instead.
The downstream 'Match items to PO' Code node already treats an
empty items array as 'no PO to match against' and leaves every
OCR item unmatched. The OCR result route then marks the DN as
needs-review and the user can fix the PO link manually.
OCR sometimes prefixes PO numbers with '#' from the slip header
(e.g. "#MEGCPO9165"). The workflow URL would otherwise break on
the # being a URL fragment separator, and the exact-match PB
filter would also fail (the fuzzy fallback worked, but a clean
sanitise is cheaper).
TS strict-mode rejects direct cast from RecordModel because the
two types don't sufficiently overlap. The fix is the standard
'as unknown as' double-cast. Was previously caught only by the
Docker build's stricter typecheck, not by the local dev
typecheck (which was polluted by .next/dev/types stale cache).
GET /api/purchase-orders/by-number/[poNumber]/items
- Auth via shared x-ingest-secret
- Returns { po: {id, purchase_order_number, supplier}, items: [{id, description, qty, qty_outstanding, unit}] }
- Exact lookup first, then fuzzy fallback (strip non-alphanumeric + lowercase)
- Used by the Delivery Note OCR workflow to enrich the LLM extraction
with PO context so each line item can be linked back to a
purchase_order_items row.
PB's filter parser only accepts && as the boolean AND operator; the
literal word AND returns HTTP 400 'Something went wrong while processing
your request'. The rest of the codebase already uses && (see
purchase-orders/upload/route.ts:81).
Caused updateBatchRow to silently no-op after every successful OCR
run — the batch row never transitioned to 'processed'.
- upload: create data_import_batches row when DN is uploaded
(batch_type=delivery_notes, processed_record_id=<dnId>)
- ocr-result: mark the batch processed on success, failed/dead on
failure (uses nextBatchState from batch-state.ts for retries)
- new /api/delivery-notes/[id]/slip proxy route: lets n8n fetch the
slip file via shared secret instead of embedding the PocketBase
admin JWT in workflow headers (which was leaking in the workflow JSON)
Backfill: existing pending DN ekko6xkhynoekas needs a manually
created data_import_batches row until it's reprocessed.
- Updated the delivery note upload API to include site ID for better project association.
- Modified the delivery note schema to support additional fields such as processing status, OCR confidence, and received by name.
- Enhanced the UI components for uploading delivery notes to allow file drop functionality and improved user feedback during file selection.
- Refactored purchasing page to integrate delivery notes that require review, providing better visibility and management of pending items.
- Added new KPIs to track active purchase orders and fulfillment percentages, improving overall dashboard insights.
- Updated the timesheet import route to support optional metadata fields for site linking and auto-creation of missing employees.
- Enhanced parsing to detect multiple file formats, including Hills Casual Matrix and long-format CSV/XLSX.
- Modified response structure to include detected format and created employees in the JSON response.
- Improved UI components to allow project selection and options for creating new workers during import.
- Refactored import logic to ensure a seamless user experience when handling timesheet uploads.
- Integrated inline editing functionality for vehicle descriptions, allowing users to edit directly within the FuelView.
- Added a Topbar with EditModeToggle and ImportFuelLogsButton for improved user interaction.
- Implemented EditModeBanner to provide feedback during editing sessions.
- Updated FleetMobileCard and FleetRow components to support inline editing, enhancing the overall user experience.
- Refactored FuelView to accept title and subtitle props for better contextual information.
- Improved the layout of the detail drawer using flexbox for better responsiveness.
- Implemented a scrollable container for the timesheet table to prevent overflow.
- Ensured a clean and organized appearance for displaying employee details.
- Adjusted the DialogContent to use flex layout and set a maximum height for better responsiveness.
- Wrapped the timesheet table in a scrollable div to prevent overflow and enhance user experience.
- Ensured the layout maintains a clean and organized appearance while displaying employee details.
- Introduced a new optional field `job_start_date` in the site schema to track project start dates.
- Updated the SitesPage component to utilize `job_start_date` for determining site status and fallback logic.
- Enhanced the SitesView component to display the new status "upcoming" for sites with future start dates.
- Modified the SiteDetailPage to include `job_start_date` in the site details and added an EditSiteButton for site management.
- Updated the site templates to include `job_start_date` for better pre-filling in the site creation form.
- Updated the import route to accept both long-format CSV and Hills-style wide-format XLSX files.
- Implemented a new parsing mechanism that detects file type and processes accordingly.
- Enhanced error handling to provide more informative messages during file parsing failures.
- Modified the response structure to include total rows processed and detailed error reporting.
- Updated the UI components to reflect changes in file selection and user instructions for importing timesheets.
- Updated FuelAttentionItem type to include qrCodeId for better tracking.
- Enhanced loadOverview function to fetch and include qrCodeId in vehicle data.
- Modified UI components in dashboard and fuel-view to display QR code information when available, improving visibility for users.
- Introduced a new section for flagged vehicles based on fuel variance, providing alerts and warnings for each vehicle.
- Updated data fetching to include fuel logs and compute variance for better tracking of fuel consumption.
- Improved site-level aggregation of fuel issues, allowing for a clearer overview of vehicles requiring attention.
- Refactored utility functions for fuel variance calculations and streamlined the data structure for better maintainability.
- Enhanced UI components to display alerts and warnings effectively, ensuring better visibility for users.
- Introduced a new section for flagged vehicles requiring attention, sorted by alert severity.
- Added pagination for fleet vehicles with a separate fleet page size.
- Updated UI components to improve visibility of flagged vehicles and their statuses.
- Adjusted vehicle detail view layout for better readability of project and QR code information.
- Ensured mobile-friendly display of vehicle data with responsive design adjustments.
When the new fuel-import route started accepting .xlsx uploads, the
real xlsx MIME type (application/vnd.openxmlformats-officedocument
.spreadsheetml.sheet) hit PB's mimeTypes allowlist on data_import_
batches.original_file and got rejected with:
validation_invalid_mime_type: mime type must be one of:
application/pdf, image/jpeg, image/png, image/webp, text/csv,
text/plain, application/vnd.ms-excel, application/csv.
The old CSV-only route never hit this (text/csv was already on the
list), so the audit-trail batch attach worked fine.
Adds the OOXML xlsx MIME to the allowlist via the same migration
pattern as 2026-08-19 and 2026-08-24-fuel-logs. Idempotent: re-run
is a no-op once applied. No app-code changes - the route was
already sending the correct MIME for the file extension.
Verified: batch create succeeds against live PB with the exact
xlsx payload the route sends.
When PB rejects a create, the SDK's ClientResponseError has:
err.message = 'Failed to create record.'
err.data = { <fieldName>: { message: 'The string did not match...' } }
We were throwing away err.data and only logging err.message, so
operators saw the generic top-level message with no clue which field
was rejecting (latest_consumption? site? vehicle? batch_reference?).
Two changes:
1. Add formatPbError() that flattens err.data into a single string
like 'Failed to create record. (latest_consumption: The string
did not match the expected pattern.)'.
2. Wrap the data_import_batches create in a try/catch so batch-
level failures (e.g. a field name mismatch after the migration)
surface as a useful 400 instead of bubbling as a 500.
No behaviour change on success. Tests + tsc clean.
PB's date field validator runs an internal regex that requires a
full ISO 8601 datetime (e.g. '2026-08-24T00:00:00.000Z'), not a
bare 'YYYY-MM-DD' string. The new parser returns YYYY-MM-DD which
is correct for our internal use, but the route was passing it
straight through to PB and getting rejected with:
'The string did not match the expected pattern.'
Anchor at UTC midnight (the pump sheet is date-only) before
writing to PB. Same shape as the original route's
'new Date(date).toISOString()' output.
Triggered by importing 'The Hills Diesel 1-24 August.xlsx'
through the new xlsx route after the migration ran.
The Aug 2026 pump sheet from The Hills (and any future sheet from
the same source) uses different column names and a different date
format than the legacy CSV the import route was built for. Adapts
the route so the same endpoint accepts either Excel (.xlsx) or CSV,
maps the new column names onto the existing FUEL_LOGS schema, and
persists their 'Latest Consumption' column verbatim.
What's new
• SheetJS (xlsx) replaces PapaParse for parsing. Single code path
handles both .xlsx and .csv uploads.
• Resolves columns case-insensitively against an alias table so
legacy CSVs (vehicle_description, odo_reading) still work
alongside the new shape (Date Logged, Vehicle QR, Km's/ Hours,
Litres Pumped, Current Site, Latest Consumption).
• Date parser: DD/MM/YYYY first (with leading-space tolerance),
falls back to YYYY-MM-DD, then to Excel serial numbers.
• Vehicle match: QR code first (clean identifier in the source
data), falls back to description text for legacy CSV.
• Site match: by sites.name (one round-trip, cached for the
import). project_code is derived from the resolved site; falls
back to the vehicle's project_code.
• Latest Consumption persisted as a new FUEL_LOGS field
(latest_consumption, number, optional). Empty source cells
stay empty rather than getting coerced to 0.
PocketBase change
• scripts/migrations/2026-08-24-fuel-logs-latest-consumption.py
adds the new field. Idempotent; follows the pattern of the
2026-08-19 migration. Run it with PB_ADMIN_EMAIL +
PB_ADMIN_PASSWORD before importing the new sheets, or the
import will fail with 'latest_consumption is not a field'.
UI
• Import button now accepts .xlsx and CSV; SAMPLE_HEADERS +
SAMPLE_OPTIONAL show the new column expectations and the
DD/MM/YYYY date format hint.
• fuel-view.tsx shows a 'Consumption (src)' column on the row
table so operators can eyeball what their fuel team sent
against the app's recomputed variance.
Out of scope (intentionally untouched)
• vehicles, sites, data_import_batches and every other
collection — no schema changes there.
• PO / timesheets / delivery-notes imports.
• fuel/page.tsx business logic (variance calc, vehicle summaries).
Verified
• Dry-run against 'The Hills Diesel 1-24 August.xlsx': 134/134
rows parse cleanly, 0 errors. 12/134 rows correctly carry an
empty Latest Consumption (matches the source).
• vitest: 20/20 fuel-import-parse + 35/35 PO-schema tests pass.
• tsc --noEmit clean.
• One pre-existing failure in
src/app/api/purchase-orders/process-batch/route.test.ts
unrelated to fuel; confirmed failing on main before this change.
PO ingestion was rejecting valid line items with qty = 0 (base
corrections, pending test rows) because the schema strictly required
qty > 0. Loosen the guard to qty >= 0 while keeping rate, disc_percent
and vat_percent strict. Also tighten against non-finite numbers
(NaN, +/-Infinity) via Number.isFinite — the previous !(qty > 0) guard
let Infinity slip through.
- Schema: qty >= 0, finite; rate/disc/vat rules unchanged
- Tests: add positive case for qty = 0; add negative + NaN rejection
- 35/35 schema tests pass; tsc clean
PB-SDK puts the HTTP status on err.status (e.g. 404), not in
err.message. The previous /404/.test(err.message) check never matched
PB's actual 404 message ('The requested resource wasn't found.'), so
every 404 fell through to the generic 'lookup_failed' 500.
This wouldn't have caused the user's current 500 since their batch row
exists, but it would mask any future 'batch row not found' race as
'lookup_failed' and prevent the correct 404 + retry flow.
When the route returns 500 with 'lookup_failed', include the failing
step name and the underlying error string in the response body so we
can pinpoint which catch block is firing (get_batch / dedup / match_site).
Production diagnostic - safe to revert once root cause is found.
- Enhanced VehicleSummary and PlantRow interfaces to include qrCodeId and costPerHour properties.
- Updated FuelView and PlantView components to display QR code information in the respective tables.
- Adjusted data fetching in FuelPage and VehicleDetailPage to retrieve and handle new fields from the API.
- Modified site detail views to reflect the unified VEHICLES collection, ensuring accurate data representation for plant utilization.
This update improves the visibility of QR codes and cost information for better tracking and management of vehicles and equipment.
- Normalise po_number (strip PO- prefix, uppercase, collapse whitespace)
via the new normalizePoNumber helper on the dedup compare AND on the
stored purchase_order_number. All new POs go forward in canonical form.
- Add a clearFile option to finalizeBatchAsProcessed and call it with
{ clearFile: false } on the dedup path. The duplicate upload's bytes
stay on the batch row for audit / debugging; the original PO's file
is unchanged. The success path still moves the file from batch -> PO
and clears the batch (current canonical-home behaviour).
- Response surfaces both po_number (normalised, used for dedup/storage)
and po_number_extracted (the raw AI output) for downstream debugging.
Tests: 15 in this file; full suite still 88/88 green.
SHA-256 the file bytes, look for a recent (30-day) processed
data_import_batches row with the same hash, and short-circuit with
200 { duplicate: true, existing_batch_id, existing_po_id } when found.
Saves an AI extraction call + a cron tick for byte-identical re-uploads.
Stores file_hash on the new batch row so future re-uploads can match
against it.
Backed by a new index (idx_batches_file_hash on file_hash, status)
and a new file_hash column (text, 64 chars) on data_import_batches
applied via PB admin API.
Tests: 11 in this file (covering happy path, byte-identical re-uploads,
hash collisions, dedup-query failures, and existing edge cases).
Strips vendor prefixes (PO-, PO#, P.O., etc.), collapses whitespace,
uppercases. Used by the upload-route hash dedup and the process-batch
dedup-by-po_number check so OCR variants ('PO-1234' vs 'po 1234') fold
to the same canonical form.
Tests: 33 in this file (23 pre-existing for validatePurchaseOrderPayload,
matchSite, computeTotalValue + 10 new for normalizePoNumber).
The catch block on batch_create only logged the PB error server-side
and returned a generic 500 'batch_create_failed', which forced
operators to dig through Coolify logs to find the real cause (e.g.
mime type not allowed, missing field). Now the response includes
pb_status, pb_message, and pb_data so the actual reason is visible
in the browser's Network tab and the dashboard's error toast.
Co-Authored-By: Claude <noreply@anthropic.com>
Captures the async-queue redesign of the PO/DN import pipeline (see
spec + plan) and rewrites the n8n implementation doc to match the
new cron-driven architecture (workflow 005iI4jX5mq9DsTR). Also
gitignores cooler.env so the local-only deployment reference is no
longer tracked.
Co-Authored-By: Claude <noreply@anthropic.com>
Both Next.js /api/purchase-orders/process-batch and the n8n cron
workflow need a PocketBase admin token (generated via PocketBase
admin > Settings > Admins). The existing template only documented
email+password auth.
Add a new 'POCKETBASE (Next.js process-batch + n8n cron)' section
with the token variable, the generation instructions, and the
warning that Next.js env and the n8n workflow's hardcoded token
value must match.
Co-Authored-By: Claude <noreply@anthropic.com>
