2026-10-14 05:04:34 +00:00
|
|
|
# Site model pipeline (Phase 1, run by hand)
|
|
|
|
|
|
|
|
|
|
Turns a consultant's drawings into the five JSON files the Site map page loads
|
|
|
|
|
(`bushveld.json`, `services.json`, `earthworks.json`, `schedules.json`, `surface.json`).
|
|
|
|
|
Specific to Civil Concepts' layer names on Bushveld Joy Road (C3092); other drawing sets
|
|
|
|
|
need the layer maps at the top of each script adjusted.
|
|
|
|
|
|
|
|
|
|
## Tools
|
|
|
|
|
- LibreDWG (`brew install libredwg`) → `dwg2dxf`
|
|
|
|
|
- poppler (`brew install poppler`) → `pdftotext`, `pdftocairo`
|
|
|
|
|
- Python 3 with numpy
|
|
|
|
|
|
|
|
|
|
## Build (Bushveld Joy example)
|
|
|
|
|
1. Convert DWGs: `dwg2dxf -y -o out.dxf in.dwg` (road layout M910-001, signs & markings M910-002,
|
|
|
|
|
studs M910-005, stormwater LS M921-001).
|
|
|
|
|
2. Run in this order, all writing into one output folder `OUT`:
|
|
|
|
|
```
|
|
|
|
|
python3 -I extract_bushveld.py <road-layout.dxf> "<SO DATA folder>" OUT/bushveld.json
|
|
|
|
|
python3 -I extract_extras.py <drawings folder> <converted DXF folder> OUT
|
|
|
|
|
python3 -I extract_surface.py <road-layout.dxf> OUT
|
|
|
|
|
python3 -I extract_xs.py <cross-section.pdf> <road-layout.dxf> OUT
|
|
|
|
|
```
|
|
|
|
|
(`extract_xs.py` validates against `surface.json`, so run it last.)
|
|
|
|
|
3. Upload: `python3 upload.py --site <site id> --label "<text>" [--dry-run] OUT`.
|
|
|
|
|
|
|
|
|
|
## Checks printed by the scripts
|
|
|
|
|
- extract_bushveld: areas per work type and the 20 m chainage sections.
|
|
|
|
|
- extract_surface: contour-only surface vs peg levels (median / 90th percentile).
|
|
|
|
|
- extract_xs: rebuilt alignment length vs the drawing (674.412 m for Bushveld Joy) and
|
|
|
|
|
cross-section vs design-surface agreement.
|
|
|
|
|
- upload: progress keys recorded on the site that the new model no longer has.
|
|
|
|
|
|
|
|
|
|
## Tests
|
|
|
|
|
`python3 -m unittest discover -s scripts/site-model -p "test_*.py"`
|
2026-10-20 10:23:44 +00:00
|
|
|
|
|
|
|
|
## Cedar Gate (C3277 / Equestria Ext. 289)
|
|
|
|
|
|
|
|
|
|
The 2026-10-20 delivery contains two projects. Use the C3277 archive and
|
|
|
|
|
`Cedar Gate Contours Chris.dwg`; exclude the Nellmapius drawings.
|
|
|
|
|
|
|
|
|
|
1. Convert the bound C3277-910-001 Rev A road layout, C3277-920to940 Rev B
|
|
|
|
|
services layout, and Chris ground-contour DWG to DXF with `dwg2dxf`.
|
|
|
|
|
2. With Python packages `numpy scipy shapely>=2.1 ezdxf` available, run:
|
|
|
|
|
```sh
|
|
|
|
|
python3 scripts/site-model/extract_cedar_gate.py DXF_FOLDER OUTPUT_FOLDER
|
|
|
|
|
```
|
|
|
|
|
3. Check `validation.json`, the source hashes in `sources.json`, surface coverage,
|
|
|
|
|
and the rendered model. The initial reviewed output is in
|
|
|
|
|
`reports/site-models/cedar-gate/2026-10-20`.
|
|
|
|
|
4. Upload with the normal uploader using Cedar Gate site ID `u6n5fye4czinw6o`.
|
|
|
|
|
|
|
|
|
|
The extractor checks all four alignment lengths against drawing labels, conserves
|
|
|
|
|
hatch areas in disjoint 20 m planning sections, measures single kerb lines once,
|
|
|
|
|
and keeps surveyed ground separate from the finished-level surface. Service
|
|
|
|
|
routes are plan overlays without inferred inverts, pipe sizes or trench volumes.
|
|
|
|
|
External lower pavement layers, earthwork volumes and setting-out pegs are
|
|
|
|
|
deliberately unquantified pending reconciliation with the relevant source data.
|
2026-10-21 03:04:42 +00:00
|
|
|
|
|
|
|
|
### Cedar Gate pipes and manholes (added 2026-10-21)
|
|
|
|
|
|
|
|
|
|
`extract_cedar_gate.py` only has plan routes for services. The C3277-921-001 long sections hold the
|
|
|
|
|
manholes, kerb inlets, pipe sizes, invert and ground levels, and a coordinate list for every structure
|
|
|
|
|
(the sewer sheets 931-001/002 are bound into the same DWG). To add them to a built model folder:
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
dwg2dxf -y "C3277-921-001 REVA - INT SW LS - 2026-08-11 - bound.dwg" # rename to a short .dwg first
|
|
|
|
|
python3 -I scripts/site-model/extract_cedar_gate_services.py LONGSECTION.dxf reports/site-models/cedar-gate/2026-10-20 \
|
|
|
|
|
reports/site-models/cedar-gate/2026-10-21 "<921 SW LS>.pdf" "<931-001 SR LS>.pdf" "<931-002 SR LS>.pdf"
|
|
|
|
|
python3 -m unittest scripts/site-model/test_cedar_gate_services.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The DWG also holds superseded copies of three sewer profiles; the issued PDFs decide which copy is kept
|
|
|
|
|
(`services-extraction.json` lists what was kept and dropped). The external SW-M1 line is on the DWG only.
|
|
|
|
|
Water is still plan routes only. Upload with the normal uploader.
|
2026-10-21 03:29:17 +00:00
|
|
|
|
|
|
|
|
### Nellmapius Ext. 25 (added 2026-10-21)
|
|
|
|
|
|
|
|
|
|
`extract_nellmapius.py` reads the converted `100_ROADS` and `400_KERBING` DWGs (not the Cedar Gate files) and writes
|
|
|
|
|
a model with road areas in 20 m sections, kerbs, stand/context lines, the four SVR survey benchmarks and an
|
|
|
|
|
indicative ground plane through them. Nothing else in the supplied set can be placed: the detail contour plan
|
|
|
|
|
has its contour labels drawn as outlines (no readable levels), the sewer long section is sheet 3 of 3, and the
|
|
|
|
|
services layout carries no levels. Do not use the model for earthworks or levels.
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
dwg2dxf -y 100_ROADS.dwg; dwg2dxf -y 400_KERBING.dwg # LibreDWG
|
|
|
|
|
python3 -I scripts/site-model/extract_nellmapius.py 100_ROADS.dxf 400_KERBING.dxf reports/site-models/nellmapius-ext-25/2026-10-21
|
|
|
|
|
python3 -m unittest scripts/site-model/test_nellmapius.py
|
|
|
|
|
python3 scripts/site-model/upload.py --site dmqybquea190z22 --label "Nellmapius Ext. 25 — roads and kerbs" reports/site-models/nellmapius-ext-25/2026-10-21
|
|
|
|
|
```
|
