Anthropic adapter: image content block before text.
OpenAI-compatible adapter: image_url with a data: URL.
Route normalizer carries images through the trust boundary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The status endpoint reports a PB record id, which is meaningless to the
chat upload card. Resolve it to the readable order number when the
batch type is purchase_orders, and leave the label null otherwise.
Failure to resolve the PO (e.g. it was deleted) is swallowed — the
batch status still stands and the raw id is still in the response as
the fallback.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The browser sends assistant turns as {role, text} with no toolCalls, so
both adapters threw 'Cannot read properties of undefined (reading
length)' on the second message. The ChatEntry type guarantees nothing
across a JSON boundary.
Normalizes history at the route (the trust boundary) and makes both
adapters defensive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Returns 404 unless ENABLE_CHAT=true. No authentication yet — must stay
disabled in production until phase 4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
formatPbError walked err.data but PB's ClientResponseError wraps the
field-level error object at err.data.data. The outer err.data has
just three keys (data, message, status), all 'invalid' on a failed
insert — useless to operators. Read err.data.data (and
err.data.data.fieldErrors if present) so the operator sees WHICH field
rejected the row.
Task 8's implementer dropped warnings[] in favour of a mismatched
count, but the design spec mandates warnings[]. Task 11's
ImportPlantHoursButton consumes warnings[] and crashes at runtime
when the response is parsed. Restore warnings[] (one entry per
mismatched-unit-type row, plus any schema-level warnings from
validatePlantHoursRow) while keeping the useful mismatched count.
Lets the inline-edit flow on /dashboard/plant set the vehicle's
unit_type (fuel|plant) so the plant-hours import route's type-mismatch
warnings can be resolved by reclassifying the vehicle.
Adds a route.test.ts mirroring the attention-dismiss precedent: 11
cases covering happy paths for unit_type='plant'/'fuel', invalid
unit_type dropped (with/without other valid fields), existing
description+project_code behaviour preserved, JSON parse errors,
empty-payload errors, and PB write failures.
Pure orchestration (importPlantHours) takes parsed source rows +
vehicle/site lookups and produces prepared plant_hours_log records
+ per-row errors. The route handles multipart upload, batch creation,
PB lookups, and insert-with-error-capture. Mismatched unit_type
vehicles are imported as audit-only records; PB dup-key failures
are recorded as 'duplicate' errors instead of failing the batch.
- Normalise date diff keys via parseDate()/slice() so imported rows
aren't all dumped as exceptions (date format mismatch silently
mis-classified every row as not_imported).
- Branch CSV vs XLSX when re-parsing the source file (type:'buffer'
doesn't work for CSV uploads).
- Validate batchId against /^[A-Za-z0-9]{15}$/ before interpolating
into the PB filter to close a filter-injection sink.
- Lift headers.indexOf out of the row loop, gate console.log, convert
dynamic imports to static, fix variable shadow.
buildPlantHoursExceptionCsv() emits a 15-column CSV with the 13
template columns + reason + operator_note, RFC-4180-quoted. The
GET /api/plant-hours/exception-report/[batchId] route re-parses the
source file from data_import_batches.original_file, diffs against
the imported plant_hours_log rows for that batch, and emits the CSV
of rows that did NOT import.
Emits the canonical xlsx with a sample row derived from the first
vehicle in PocketBase. Falls back to a hardcoded sample if the
vehicles collection is empty.
Sets or clears vehicles.attention_dismissed_at via pbAdmin().
15-char PB id pattern, 404 when PB reports the record is missing,
500 on any other write failure. Test mocks pbAdmin() so no live PB
needed.
Validates { supplier } (trimmed, 1-200 chars) and PATCHes the
purchase_orders.supplier field via pbAdmin(). 400/404/500 with
explicit error codes; mirrors the retry-route pattern.
Updated the OCR result processing to include validation of matched PO items before determining the verification tier. This change ensures that if the local PO items fetch is empty but n8n supplied matched IDs, the tier will downgrade to draft, requiring user review. Additionally, the `resolveOcrTier` function has been modified to account for item verifiability, improving the accuracy of tier assignments based on confidence levels and PO resolution status. Unit tests have been added to cover these new scenarios.
Updated the OCR result processing to include a more robust PO matching mechanism. The new implementation prioritizes exact matches, followed by normalized comparisons, and finally employs a Levenshtein-based fuzzy match for improved accuracy. This change ensures that delivery notes are correctly linked to purchase orders, even when OCR introduces minor errors. Additionally, the behavior for handling confidence levels and PO resolution has been clarified in the documentation, enhancing overall processing reliability.
OCR is noisy. Instead of exact-then-fuzzy-fallback, always fetch
all ~15 POs from PocketBase, normalise both sides (strip non-
alphanumeric, lowercase), score each with Levenshtein similarity,
and return the best match above 0.75 threshold. Single-char
insert/delete (MEGCP09239 vs MEGCPO9239) scores ~0.9 — matches.
Completely different POs (similarity <0.5) don't match.
Plus the existing 'return 200 with empty items' fallback if no
match — keeps the workflow loop alive even when OCR is bad.
OCR sometimes mis-reads PO numbers (e.g. 'MEGCP09239' instead of
'MEGCPO9239') — when that happens the PO lookup 404'd, which
terminated the n8n workflow loop even with onError set on the
HTTP node.
Return 200 with {po: null, items: [], note: 'po_not_found'} instead.
The downstream 'Match items to PO' Code node already treats an
empty items array as 'no PO to match against' and leaves every
OCR item unmatched. The OCR result route then marks the DN as
needs-review and the user can fix the PO link manually.
OCR sometimes prefixes PO numbers with '#' from the slip header
(e.g. "#MEGCPO9165"). The workflow URL would otherwise break on
the # being a URL fragment separator, and the exact-match PB
filter would also fail (the fuzzy fallback worked, but a clean
sanitise is cheaper).
TS strict-mode rejects direct cast from RecordModel because the
two types don't sufficiently overlap. The fix is the standard
'as unknown as' double-cast. Was previously caught only by the
Docker build's stricter typecheck, not by the local dev
typecheck (which was polluted by .next/dev/types stale cache).
GET /api/purchase-orders/by-number/[poNumber]/items
- Auth via shared x-ingest-secret
- Returns { po: {id, purchase_order_number, supplier}, items: [{id, description, qty, qty_outstanding, unit}] }
- Exact lookup first, then fuzzy fallback (strip non-alphanumeric + lowercase)
- Used by the Delivery Note OCR workflow to enrich the LLM extraction
with PO context so each line item can be linked back to a
purchase_order_items row.
PB's filter parser only accepts && as the boolean AND operator; the
literal word AND returns HTTP 400 'Something went wrong while processing
your request'. The rest of the codebase already uses && (see
purchase-orders/upload/route.ts:81).
Caused updateBatchRow to silently no-op after every successful OCR
run — the batch row never transitioned to 'processed'.
- upload: create data_import_batches row when DN is uploaded
(batch_type=delivery_notes, processed_record_id=<dnId>)
- ocr-result: mark the batch processed on success, failed/dead on
failure (uses nextBatchState from batch-state.ts for retries)
- new /api/delivery-notes/[id]/slip proxy route: lets n8n fetch the
slip file via shared secret instead of embedding the PocketBase
admin JWT in workflow headers (which was leaking in the workflow JSON)
Backfill: existing pending DN ekko6xkhynoekas needs a manually
created data_import_batches row until it's reprocessed.
- Updated the delivery note upload API to include site ID for better project association.
- Modified the delivery note schema to support additional fields such as processing status, OCR confidence, and received by name.
- Enhanced the UI components for uploading delivery notes to allow file drop functionality and improved user feedback during file selection.
- Refactored purchasing page to integrate delivery notes that require review, providing better visibility and management of pending items.
- Added new KPIs to track active purchase orders and fulfillment percentages, improving overall dashboard insights.
- Updated the timesheet import route to support optional metadata fields for site linking and auto-creation of missing employees.
- Enhanced parsing to detect multiple file formats, including Hills Casual Matrix and long-format CSV/XLSX.
- Modified response structure to include detected format and created employees in the JSON response.
- Improved UI components to allow project selection and options for creating new workers during import.
- Refactored import logic to ensure a seamless user experience when handling timesheet uploads.
- Integrated inline editing functionality for vehicle descriptions, allowing users to edit directly within the FuelView.
- Added a Topbar with EditModeToggle and ImportFuelLogsButton for improved user interaction.
- Implemented EditModeBanner to provide feedback during editing sessions.
- Updated FleetMobileCard and FleetRow components to support inline editing, enhancing the overall user experience.
- Refactored FuelView to accept title and subtitle props for better contextual information.
- Introduced a new optional field `job_start_date` in the site schema to track project start dates.
- Updated the SitesPage component to utilize `job_start_date` for determining site status and fallback logic.
- Enhanced the SitesView component to display the new status "upcoming" for sites with future start dates.
- Modified the SiteDetailPage to include `job_start_date` in the site details and added an EditSiteButton for site management.
- Updated the site templates to include `job_start_date` for better pre-filling in the site creation form.
- Updated the import route to accept both long-format CSV and Hills-style wide-format XLSX files.
- Implemented a new parsing mechanism that detects file type and processes accordingly.
- Enhanced error handling to provide more informative messages during file parsing failures.
- Modified the response structure to include total rows processed and detailed error reporting.
- Updated the UI components to reflect changes in file selection and user instructions for importing timesheets.
When PB rejects a create, the SDK's ClientResponseError has:
err.message = 'Failed to create record.'
err.data = { <fieldName>: { message: 'The string did not match...' } }
We were throwing away err.data and only logging err.message, so
operators saw the generic top-level message with no clue which field
was rejecting (latest_consumption? site? vehicle? batch_reference?).
Two changes:
1. Add formatPbError() that flattens err.data into a single string
like 'Failed to create record. (latest_consumption: The string
did not match the expected pattern.)'.
2. Wrap the data_import_batches create in a try/catch so batch-
level failures (e.g. a field name mismatch after the migration)
surface as a useful 400 instead of bubbling as a 500.
No behaviour change on success. Tests + tsc clean.
PB's date field validator runs an internal regex that requires a
full ISO 8601 datetime (e.g. '2026-08-24T00:00:00.000Z'), not a
bare 'YYYY-MM-DD' string. The new parser returns YYYY-MM-DD which
is correct for our internal use, but the route was passing it
straight through to PB and getting rejected with:
'The string did not match the expected pattern.'
Anchor at UTC midnight (the pump sheet is date-only) before
writing to PB. Same shape as the original route's
'new Date(date).toISOString()' output.
Triggered by importing 'The Hills Diesel 1-24 August.xlsx'
through the new xlsx route after the migration ran.
The Aug 2026 pump sheet from The Hills (and any future sheet from
the same source) uses different column names and a different date
format than the legacy CSV the import route was built for. Adapts
the route so the same endpoint accepts either Excel (.xlsx) or CSV,
maps the new column names onto the existing FUEL_LOGS schema, and
persists their 'Latest Consumption' column verbatim.
What's new
• SheetJS (xlsx) replaces PapaParse for parsing. Single code path
handles both .xlsx and .csv uploads.
• Resolves columns case-insensitively against an alias table so
legacy CSVs (vehicle_description, odo_reading) still work
alongside the new shape (Date Logged, Vehicle QR, Km's/ Hours,
Litres Pumped, Current Site, Latest Consumption).
• Date parser: DD/MM/YYYY first (with leading-space tolerance),
falls back to YYYY-MM-DD, then to Excel serial numbers.
• Vehicle match: QR code first (clean identifier in the source
data), falls back to description text for legacy CSV.
• Site match: by sites.name (one round-trip, cached for the
import). project_code is derived from the resolved site; falls
back to the vehicle's project_code.
• Latest Consumption persisted as a new FUEL_LOGS field
(latest_consumption, number, optional). Empty source cells
stay empty rather than getting coerced to 0.
PocketBase change
• scripts/migrations/2026-08-24-fuel-logs-latest-consumption.py
adds the new field. Idempotent; follows the pattern of the
2026-08-19 migration. Run it with PB_ADMIN_EMAIL +
PB_ADMIN_PASSWORD before importing the new sheets, or the
import will fail with 'latest_consumption is not a field'.
UI
• Import button now accepts .xlsx and CSV; SAMPLE_HEADERS +
SAMPLE_OPTIONAL show the new column expectations and the
DD/MM/YYYY date format hint.
• fuel-view.tsx shows a 'Consumption (src)' column on the row
table so operators can eyeball what their fuel team sent
against the app's recomputed variance.
Out of scope (intentionally untouched)
• vehicles, sites, data_import_batches and every other
collection — no schema changes there.
• PO / timesheets / delivery-notes imports.
• fuel/page.tsx business logic (variance calc, vehicle summaries).
Verified
• Dry-run against 'The Hills Diesel 1-24 August.xlsx': 134/134
rows parse cleanly, 0 errors. 12/134 rows correctly carry an
empty Latest Consumption (matches the source).
• vitest: 20/20 fuel-import-parse + 35/35 PO-schema tests pass.
• tsc --noEmit clean.
• One pre-existing failure in
src/app/api/purchase-orders/process-batch/route.test.ts
unrelated to fuel; confirmed failing on main before this change.
PB-SDK puts the HTTP status on err.status (e.g. 404), not in
err.message. The previous /404/.test(err.message) check never matched
PB's actual 404 message ('The requested resource wasn't found.'), so
every 404 fell through to the generic 'lookup_failed' 500.
This wouldn't have caused the user's current 500 since their batch row
exists, but it would mask any future 'batch row not found' race as
'lookup_failed' and prevent the correct 404 + retry flow.
When the route returns 500 with 'lookup_failed', include the failing
step name and the underlying error string in the response body so we
can pinpoint which catch block is firing (get_batch / dedup / match_site).
Production diagnostic - safe to revert once root cause is found.
- Normalise po_number (strip PO- prefix, uppercase, collapse whitespace)
via the new normalizePoNumber helper on the dedup compare AND on the
stored purchase_order_number. All new POs go forward in canonical form.
- Add a clearFile option to finalizeBatchAsProcessed and call it with
{ clearFile: false } on the dedup path. The duplicate upload's bytes
stay on the batch row for audit / debugging; the original PO's file
is unchanged. The success path still moves the file from batch -> PO
and clears the batch (current canonical-home behaviour).
- Response surfaces both po_number (normalised, used for dedup/storage)
and po_number_extracted (the raw AI output) for downstream debugging.
Tests: 15 in this file; full suite still 88/88 green.
SHA-256 the file bytes, look for a recent (30-day) processed
data_import_batches row with the same hash, and short-circuit with
200 { duplicate: true, existing_batch_id, existing_po_id } when found.
Saves an AI extraction call + a cron tick for byte-identical re-uploads.
Stores file_hash on the new batch row so future re-uploads can match
against it.
Backed by a new index (idx_batches_file_hash on file_hash, status)
and a new file_hash column (text, 64 chars) on data_import_batches
applied via PB admin API.
Tests: 11 in this file (covering happy path, byte-identical re-uploads,
hash collisions, dedup-query failures, and existing edge cases).
The catch block on batch_create only logged the PB error server-side
and returned a generic 500 'batch_create_failed', which forced
operators to dig through Coolify logs to find the real cause (e.g.
mime type not allowed, missing field). Now the response includes
pb_status, pb_message, and pb_data so the actual reason is visible
in the browser's Network tab and the dashboard's error toast.
Co-Authored-By: Claude <noreply@anthropic.com>
POST /api/purchase-orders/upload no longer forwards the file to n8n
synchronously. It now validates size/MIME, writes a data_import_batches
row (batch_type='purchase_orders', status='pending', attempts=0) with the
uploaded file attached, and returns 202 { batch_id, status: "queued",
filename } immediately. n8n's 5-minute cron claims pending rows and calls
back to /api/purchase-orders/process-batch; the browser polls
/api/data-import-batches/[id]/status.
Adds route tests covering validation guards, the queued 202 contract,
the batch-create failure path, and that no synchronous fetch to n8n
happens.
Co-Authored-By: Claude <noreply@anthropic.com>
n8n POSTs extracted PO JSON + batch_id after downloading the file
from PocketBase. The route validates the payload, dedups by po_number
(idempotent), resolves the site, moves the batch's file onto a new
purchase_orders row + items, and finalizes the batch with
status=processed, processed_record_id, processed_at, original_file
cleared.
Co-Authored-By: Claude <noreply@anthropic.com>
POST /api/data-import-batches/[id]/retry resets a failed or dead batch
back to pending (attempts=0, last_error/next_retry_at cleared) so the
next cron sweep picks it up. Non-retryable states return 409 with the
current status; legacy rows with no status default to processed.
Co-Authored-By: Claude <noreply@anthropic.com>
GET /api/data-import-batches/[id]/status returns the current batch
state for the upload button's 2-second polling loop. Returns 400 on
bad id format, 404 if missing, 500 on lookup failure. Defaults
missing status to 'processed' for legacy timesheets/fuel-logs rows.
Co-Authored-By: Claude <noreply@anthropic.com>
- Purchase order upload route now returns 502 n8n_rejected when n8n
responds with a non-2xx status, instead of silently claiming success.
- Add a 10s AbortController timeout on the n8n fetch, returning 504
n8n_timeout on expiry (502 n8n_unreachable on other network errors).
- Extract ALLOWED_MIME and MAX_FILE_BYTES from upload/route.ts into
src/lib/purchase-order-schema.ts so the upcoming ingest route can
share the same validation constants.
- Updated the documentation to reflect the latest changes in the Purchase Order upload and n8n ingestion process.
- Clarified the data flow from browser to Next.js API endpoints and through to PocketBase.
- Ensured that the documentation aligns with the recent implementation and design specifications.
