MEGAINFRA-WEBAPP/scripts/migrations
DEls a3e4702159 refactor(schema): resolve project_code through site, not a stored copy
project_code was a text copy of the parent site's code, denormalised
onto 8 child collections that each already have a real site relation.
Written once at insert, never updated -- docs/data-model.md documents
the drift risk. On purchase_orders it never even held a real project
code: production had requisition numbers, site names, and the PO's own
number in that field.

Every read site now resolves the code through site/expand.site/siteById
instead, reusing each page's existing site map where one already
existed. Every create() call stops writing the column. The chat agent's
list_imports tool no longer reads data_import_batches -- it now reads
timesheets, fuel_sheets, plant_hours_sheets, purchase_orders and
delivery_notes filtered by processing_status, which was necessary before
the pool can be dropped (Task 20 would otherwise 404 the agent).

Also closed a Task 17 gap found while doing this: the agent tools' vehicle
and fuel-log handlers still resolved project code from the now-removed
vehicles.project_code column.

Wrote scripts/migrations/2026-09-15-drop-legacy-columns.py, the one
irreversible step in this whole restructure. Its guard against orphaning
fuel_logs rows checked the wrong condition -- "fuel_sheet is empty" --
which produced a false abort: 73 fuel_logs rows have no fuel_sheet
because they predate batch tracking entirely and never had a
batch_reference to begin with, so dropping data_import_batches costs
them nothing. Fixed the guard to check what it actually claims to check:
a row that HAD a batch_reference and was never relinked. Verified live
that no such row exists on any of the three affected collections.

Script run in dry-run only; not yet applied.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:01:14 +02:00
..
2026-08-19-extend-data-import-batches.py feat(migration): add async-queue fields to data_import_batches 2026-08-19 19:19:17 +02:00
2026-08-24-data-import-batches-allow-xlsx.py fix(data-import-batches): allow xlsx MIME on original_file 2026-08-24 22:17:45 +02:00
2026-08-24-fuel-logs-latest-consumption.py feat(fuel): drop-in xlsx/csv import for pump sheets 2026-08-24 22:03:08 +02:00
2026-08-28-vehicles-attention-dismissed.py feat(fuel): PB schema migration for attention_dismissed_at 2026-08-28 12:26:23 +02:00
2026-08-28-vehicles-unit-type-optional.py fix(pb): make vehicles.unit_type optional 2026-08-28 15:26:33 +02:00
2026-08-29-fixed-costs-recurring-backfill.py fix(schema): ensure is_recurring=false writes are preserved 2026-08-28 13:43:38 +02:00
2026-08-31-data-import-batches-allow-pdf-images.py fix(pb): allow PDF and image MIME on data_import_batches.original_file 2026-08-31 12:20:41 +02:00
2026-09-11-add-document-fields.py feat(schema): add file_hash and queue fields to DNs and POs 2026-09-11 10:29:10 +02:00
2026-09-11-backfill-entity-file-hashes.py fix(migration): separate download failures from missing files 2026-09-11 10:39:54 +02:00
2026-09-11-backfill-fuel-log-sites.py fix(fuel): backfill site on 147 orphaned fuel logs 2026-09-11 10:44:19 +02:00
2026-09-11-backfill-timesheet-sites.py fix(timesheets): stamp site on log rows, ending read-time drift 2026-09-11 10:48:51 +02:00
2026-09-12-add-po-date-uploaded.py fix(purchasing): restore the 30-day PO dedup window with a real timestamp 2026-09-11 14:17:42 +02:00
2026-09-12-create-document-collections.py feat(schema): add timesheets, fuel_sheets, plant_hours_sheets documents 2026-09-11 11:09:14 +02:00
2026-09-12-migrate-batches-to-documents.py feat(migration): move batch files into document collections 2026-09-11 13:06:40 +02:00
2026-09-12-rename-collections.py refactor(schema): rename received_items and timesheets to their real names 2026-09-11 11:01:40 +02:00
2026-09-12-widen-sheet-mimetypes.py feat(schema): add timesheets, fuel_sheets, plant_hours_sheets documents 2026-09-11 11:09:14 +02:00
2026-09-14-require-site.py feat(schema): require site on every ingested collection 2026-09-11 15:32:48 +02:00
2026-09-15-drop-legacy-columns.py refactor(schema): resolve project_code through site, not a stored copy 2026-09-11 16:01:14 +02:00