David/MEGAINFRA-WEBAPP
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
MEGAINFRA-WEBAPP/scripts/site-model/README.md
DEls 5141f7783a feat(site-map): Cedar Gate stormwater and sewer pipes and manholes from the long sections 
Extract 23 stormwater structures / 22 pipes and 18 sewer manholes / 17 pipes with sizes, inverts, ground levels and positions from the C3277 long-section tables (superseded sewer copies dropped by matching the issued PDFs). New model folder 2026-10-21. Map 3 draws them as round pipes and manholes, adds status popups for pipes and manholes, and reuses Map 2's services panel; water stays plan routes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 05:04:42 +02:00

3.9 KiB
Raw Blame History

Site model pipeline (Phase 1, run by hand)

Turns a consultant's drawings into the five JSON files the Site map page loads (bushveld.json, services.json, earthworks.json, schedules.json, surface.json). Specific to Civil Concepts' layer names on Bushveld Joy Road (C3092); other drawing sets need the layer maps at the top of each script adjusted.

Tools

  • LibreDWG (brew install libredwg) → dwg2dxf
  • poppler (brew install poppler) → pdftotext, pdftocairo
  • Python 3 with numpy

Build (Bushveld Joy example)

  1. Convert DWGs: dwg2dxf -y -o out.dxf in.dwg (road layout M910-001, signs & markings M910-002, studs M910-005, stormwater LS M921-001).
  2. Run in this order, all writing into one output folder OUT: 
    python3 -I extract_bushveld.py <road-layout.dxf> "<SO DATA folder>" OUT/bushveld.json
python3 -I extract_extras.py <drawings folder> <converted DXF folder> OUT
python3 -I extract_surface.py <road-layout.dxf> OUT
python3 -I extract_xs.py <cross-section.pdf> <road-layout.dxf> OUT
    (extract_xs.py validates against surface.json, so run it last.)
  3. Upload: python3 upload.py --site <site id> --label "<text>" [--dry-run] OUT.

Checks printed by the scripts

  • extract_bushveld: areas per work type and the 20 m chainage sections.
  • extract_surface: contour-only surface vs peg levels (median / 90th percentile).
  • extract_xs: rebuilt alignment length vs the drawing (674.412 m for Bushveld Joy) and cross-section vs design-surface agreement.
  • upload: progress keys recorded on the site that the new model no longer has.

Tests

python3 -m unittest discover -s scripts/site-model -p "test_*.py"

Cedar Gate (C3277 / Equestria Ext. 289)

The 2026-10-20 delivery contains two projects. Use the C3277 archive and Cedar Gate Contours Chris.dwg; exclude the Nellmapius drawings.

  1. Convert the bound C3277-910-001 Rev A road layout, C3277-920to940 Rev B services layout, and Chris ground-contour DWG to DXF with dwg2dxf.
  2. With Python packages numpy scipy shapely>=2.1 ezdxf available, run: 
    python3 scripts/site-model/extract_cedar_gate.py DXF_FOLDER OUTPUT_FOLDER
  3. Check validation.json, the source hashes in sources.json, surface coverage, and the rendered model. The initial reviewed output is in reports/site-models/cedar-gate/2026-10-20.
  4. Upload with the normal uploader using Cedar Gate site ID u6n5fye4czinw6o.

The extractor checks all four alignment lengths against drawing labels, conserves hatch areas in disjoint 20 m planning sections, measures single kerb lines once, and keeps surveyed ground separate from the finished-level surface. Service routes are plan overlays without inferred inverts, pipe sizes or trench volumes. External lower pavement layers, earthwork volumes and setting-out pegs are deliberately unquantified pending reconciliation with the relevant source data.

Cedar Gate pipes and manholes (added 2026-10-21)

extract_cedar_gate.py only has plan routes for services. The C3277-921-001 long sections hold the manholes, kerb inlets, pipe sizes, invert and ground levels, and a coordinate list for every structure (the sewer sheets 931-001/002 are bound into the same DWG). To add them to a built model folder:

dwg2dxf -y "C3277-921-001 REVA  -  INT SW LS - 2026-08-11 - bound.dwg"      # rename to a short .dwg first
python3 -I scripts/site-model/extract_cedar_gate_services.py LONGSECTION.dxf reports/site-models/cedar-gate/2026-10-20 \
  reports/site-models/cedar-gate/2026-10-21 "<921 SW LS>.pdf" "<931-001 SR LS>.pdf" "<931-002 SR LS>.pdf"
python3 -m unittest scripts/site-model/test_cedar_gate_services.py

The DWG also holds superseded copies of three sewer profiles; the issued PDFs decide which copy is kept (services-extraction.json lists what was kept and dropped). The external SW-M1 line is on the DWG only. Water is still plan routes only. Upload with the normal uploader.