David/MEGAINFRA-WEBAPP
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
MEGAINFRA-WEBAPP/scripts/site-model
History
DEls 36da7f9f20 feat(site-model): Nellmapius Ext. 25 roads, kerbs and benchmarks from the 100_ROADS and 400_KERBING drawings 
Road areas (5,221 m²) in 20 m sections along the ring roads and the Igwali Street spur, 1.74 km of kerb measured once, stand lines, and the four survey benchmarks with an indicative ground plane. No services or ground levels are placed; the supplied survey and sewer sheets cannot yield them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 05:29:17 +02:00
..
alignment.py
dxf_reader.py
extract_bushveld.py
extract_cedar_gate.py feat: update management comparisons and Cedar Gate site model 2026-10-20 12:23:44 +02:00
extract_cedar_gate_services.py feat(site-map): Cedar Gate stormwater and sewer pipes and manholes from the long sections 2026-10-21 05:04:42 +02:00
extract_cedar_gate_xs.py feat(site-map): Maps page with previews, double-click progress popup, Cedar Gate cut & fill 2026-10-20 12:55:51 +02:00
extract_extras.py
extract_nellmapius.py feat(site-model): Nellmapius Ext. 25 roads, kerbs and benchmarks from the 100_ROADS and 400_KERBING drawings 2026-10-21 05:29:17 +02:00
extract_surface.py
extract_xs.py
model_keys.py feat(site-map): model upload script with progress-key parity 2026-10-14 07:06:13 +02:00
README.md feat(site-model): Nellmapius Ext. 25 roads, kerbs and benchmarks from the 100_ROADS and 400_KERBING drawings 2026-10-21 05:29:17 +02:00
test_alignment.py
test_cedar_gate_services.py feat(site-map): Cedar Gate stormwater and sewer pipes and manholes from the long sections 2026-10-21 05:04:42 +02:00
test_model_keys.py feat(site-map): model upload script with progress-key parity 2026-10-14 07:06:13 +02:00
test_nellmapius.py feat(site-model): Nellmapius Ext. 25 roads, kerbs and benchmarks from the 100_ROADS and 400_KERBING drawings 2026-10-21 05:29:17 +02:00
test_upload.py feat(site-map): model upload script with progress-key parity 2026-10-14 07:06:13 +02:00
upload.py feat: update management comparisons and Cedar Gate site model 2026-10-20 12:23:44 +02:00

README.md

Site model pipeline (Phase 1, run by hand)

Turns a consultant's drawings into the five JSON files the Site map page loads (bushveld.json, services.json, earthworks.json, schedules.json, surface.json). Specific to Civil Concepts' layer names on Bushveld Joy Road (C3092); other drawing sets need the layer maps at the top of each script adjusted.

Tools

  • LibreDWG (brew install libredwg) → dwg2dxf
  • poppler (brew install poppler) → pdftotext, pdftocairo
  • Python 3 with numpy

Build (Bushveld Joy example)

  1. Convert DWGs: dwg2dxf -y -o out.dxf in.dwg (road layout M910-001, signs & markings M910-002, studs M910-005, stormwater LS M921-001).
  2. Run in this order, all writing into one output folder OUT: 
    python3 -I extract_bushveld.py <road-layout.dxf> "<SO DATA folder>" OUT/bushveld.json
python3 -I extract_extras.py <drawings folder> <converted DXF folder> OUT
python3 -I extract_surface.py <road-layout.dxf> OUT
python3 -I extract_xs.py <cross-section.pdf> <road-layout.dxf> OUT
    (extract_xs.py validates against surface.json, so run it last.)
  3. Upload: python3 upload.py --site <site id> --label "<text>" [--dry-run] OUT.

Checks printed by the scripts

  • extract_bushveld: areas per work type and the 20 m chainage sections.
  • extract_surface: contour-only surface vs peg levels (median / 90th percentile).
  • extract_xs: rebuilt alignment length vs the drawing (674.412 m for Bushveld Joy) and cross-section vs design-surface agreement.
  • upload: progress keys recorded on the site that the new model no longer has.

Tests

python3 -m unittest discover -s scripts/site-model -p "test_*.py"

Cedar Gate (C3277 / Equestria Ext. 289)

The 2026-10-20 delivery contains two projects. Use the C3277 archive and Cedar Gate Contours Chris.dwg; exclude the Nellmapius drawings.

  1. Convert the bound C3277-910-001 Rev A road layout, C3277-920to940 Rev B services layout, and Chris ground-contour DWG to DXF with dwg2dxf.
  2. With Python packages numpy scipy shapely>=2.1 ezdxf available, run: 
    python3 scripts/site-model/extract_cedar_gate.py DXF_FOLDER OUTPUT_FOLDER
  3. Check validation.json, the source hashes in sources.json, surface coverage, and the rendered model. The initial reviewed output is in reports/site-models/cedar-gate/2026-10-20.
  4. Upload with the normal uploader using Cedar Gate site ID u6n5fye4czinw6o.

The extractor checks all four alignment lengths against drawing labels, conserves hatch areas in disjoint 20 m planning sections, measures single kerb lines once, and keeps surveyed ground separate from the finished-level surface. Service routes are plan overlays without inferred inverts, pipe sizes or trench volumes. External lower pavement layers, earthwork volumes and setting-out pegs are deliberately unquantified pending reconciliation with the relevant source data.

Cedar Gate pipes and manholes (added 2026-10-21)

extract_cedar_gate.py only has plan routes for services. The C3277-921-001 long sections hold the manholes, kerb inlets, pipe sizes, invert and ground levels, and a coordinate list for every structure (the sewer sheets 931-001/002 are bound into the same DWG). To add them to a built model folder:

dwg2dxf -y "C3277-921-001 REVA  -  INT SW LS - 2026-08-11 - bound.dwg"      # rename to a short .dwg first
python3 -I scripts/site-model/extract_cedar_gate_services.py LONGSECTION.dxf reports/site-models/cedar-gate/2026-10-20 \
  reports/site-models/cedar-gate/2026-10-21 "<921 SW LS>.pdf" "<931-001 SR LS>.pdf" "<931-002 SR LS>.pdf"
python3 -m unittest scripts/site-model/test_cedar_gate_services.py

The DWG also holds superseded copies of three sewer profiles; the issued PDFs decide which copy is kept (services-extraction.json lists what was kept and dropped). The external SW-M1 line is on the DWG only. Water is still plan routes only. Upload with the normal uploader.

Nellmapius Ext. 25 (added 2026-10-21)

extract_nellmapius.py reads the converted 100_ROADS and 400_KERBING DWGs (not the Cedar Gate files) and writes a model with road areas in 20 m sections, kerbs, stand/context lines, the four SVR survey benchmarks and an indicative ground plane through them. Nothing else in the supplied set can be placed: the detail contour plan has its contour labels drawn as outlines (no readable levels), the sewer long section is sheet 3 of 3, and the services layout carries no levels. Do not use the model for earthworks or levels.

dwg2dxf -y 100_ROADS.dwg; dwg2dxf -y 400_KERBING.dwg      # LibreDWG
python3 -I scripts/site-model/extract_nellmapius.py 100_ROADS.dxf 400_KERBING.dxf reports/site-models/nellmapius-ext-25/2026-10-21
python3 -m unittest scripts/site-model/test_nellmapius.py
python3 scripts/site-model/upload.py --site dmqybquea190z22 --label "Nellmapius Ext. 25 — roads and kerbs" reports/site-models/nellmapius-ext-25/2026-10-21