MEGA-BOQ-WEBAPP/README.md

# MEGA INFRA · BOQ Converter

Drop in **our** Candy PDF and **their** Excel. Get back their workbook
populated, using their own formulas and layout, plus a proof report of exactly
what was written, what was already correct, and where the two documents
disagree.

Anything you disagree with, you settle **in the app** and regenerate. The
consultant never has to leave the screen.

Runs entirely on your machine. Nothing is uploaded anywhere.

![the app](docs/app-screenshot.png)

## Settling a disagreement in the app

Where the two documents differ, the app writes **nothing** — your value stands.
Each dispute is one collapsed row. Open it and choose:

- **Keep yours (as it is)** — the default, your file is untouched
- **Use our PDF's value** — take the figure from the Candy export
- **or type a value** — anything else, coerced to a number when it looks like one

Press **Apply & regenerate** and the choices are written straight into the
workbook; the row then moves to "settled" and drops off the open list. The
PAYMENT column, your formulas and your formatting are never touched.

![disputes](docs/app-disputes.png)

## Design system — MEGA INFRASTRUCTURE

Palette sampled from the dashboard. All tokens live in **`static/theme.css`**
as CSS custom properties, so the brand can be retuned in one place without
touching Python.

| group | tokens |
|---|---|
| surfaces | `--canvas #1A202C` · `--topbar #1A212D` · `--surface #262A37` · `--surface-inset #252A36` · `--surface-raise #363B49` · `--hairline #2F3442` |
| status fills | `--success-fill #22383E` / `--success-edge #1B534A` · `--warn-fill #3A373C` · `--danger-fill #3A2D38` |
| accents | `--orange #FEA944` · `--orange-soft #F1A962` · `--orange-deep #DF8441` · `--green #00CD8A` · `--red #F04141` |
| text | `--text #E1E1E1` · `--text-muted #999999` · `--text-warm #B3AC99` · `--text-link #00CD8A` |

Warm neutrals are deliberate — `#E1E1E1` sits on a warm axis and pairs with
`#B3AC99`; pure `#EEEEEE` would read cold next to `#FEA944`.

**On the accessibility flag:** `#F04141` measures 3.77:1 on the card surface, so
it is used only for the severity edge and large marks. Any red *text* uses
`--red-soft #FF6B6B` at 5.15:1, which clears AA.

Type is **Fraunces** for headings and **Nunito** for body.

### Icons are real SVG, not emoji

Thirteen icons live in `boq_engine/theme.py` as inline SVG — 24x24, 1.75
stroke, round caps, so the set looks drawn by one hand. They inherit
`currentColor`, so they pick up the palette for free, and being inline they
need no network and no icon font.

They sit in the brand mark, the uploader labels, the KPI tiles, the pills and
the section headings — everywhere HTML actually renders.

**Not** on buttons or tab labels. Streamlit renders those as *plain text*, so
an inline SVG there leaks to the user as literal `<svg …>` markup, and emoji
read as emoticons. Those labels are deliberately typographic, which looks
more corporate anyway.

### Two implementation notes worth knowing

**The stylesheet is injected with `st.html`, not `st.markdown`.** Markdown
injection duplicated the style block on every rerun, and one malformed
selector silently killed every rule after it — the page rendered unstyled with
no error anywhere. Serving it from `static/` was tried and doesn't work:
Streamlit only serves its own hashed assets from that route, and an arbitrary
path falls through to the SPA shell and returns `index.html` as `text/html`.
`st.html` inserts the sheet into the page, isn't iframed, and replaces cleanly.

**Buttons and tab labels are plain text.** See the icon note above — this is
the one place Streamlit refuses to render HTML, and it fails loudly.

**Status messages are quiet pills, not alert blocks.** A filled
`st.warning` bar shouts at the reader for something that is background
information. `theme.pill()` sits at the same volume as the legend it belongs
next to. Genuine errors — source lines that have nowhere to go, subtotals that
do not reconcile — still use real alerts, because those are the thing to act on.

**Dispute choices are real Streamlit widgets.** Raw HTML `<select>` was tried
first and silently does nothing — the value never reaches Python, because
`st.markdown(unsafe_allow_html=True)` has no channel back to session state.
Keys are derived from the cell (`choice::<sheet>::<row>::<field>`) so a choice
survives a rerun and can be read back to build the override map.

## The rule that matters most

Where the two documents disagree, **the consultant's value is left alone and
the difference is reported.** The app never overwrites their work to match ours.

Every finding carries a `severity` decided in the data layer, not guessed from
the sentence text — so a red card always means "a person must choose":

| severity | meaning | on your sample |
|---|---|---|
| `decision` | the two documents genuinely disagree | **1** — overheads qty 15 000 vs 1 500 000 |
| `source_gap` | our PDF has no figure; theirs is better | 13 — all of Section 4 |
| `source_defect` | our PDF printed something corrupt | 2 — the `?` litre sign |
| `resolved` | the consultant settled it in the app | 0 until they do |
| `note` | worth knowing, no action implied | totals written |

A cell with an override is written instead of left alone, and the dispute drops
to `resolved` so it stops being flagged.

Two cases where that mattered:

- Candy prints the litre sign as `?`. Copying it into their file would have
  *introduced* a defect that was not there. Their `ℓ` was kept.
- Our PDF has all of Section 4 as "Rate only" (qty 0) — cable ducts and
  directional drilling were never quantified. Their template has real
  provisional quantities (100 m³ of trench, 200 m of 110 mm pipe, 70 m of
  drilling). Writing our `0` would have destroyed better data.

## Results on the sample

| | |
|---|---|
| Values written | 84 |
| Already correct (verified) | 98 |
| Source lines with nowhere to go | **0** |
| Disagreements reported, none overwritten | 16 |

Their workbook's own `COMBINED SUMMARY` recomputes to **R5,696,934.93** —
matching the Candy PDF's grand total to the cent, through their own formulas
and their own VAT line.

## Things the source BOQ gets wrong

Reported, not fixed — these are in the PDF:

1. **1.2 Scheduled Time-Related Items — R31,500.00 unexplained.** Its priced
   lines total R1,053,000.00 against a printed R1,084,500.00. Every other
   1.2.x line is blank/INCL. Worth a phone call.
2. **1.6 Temporary Works** — printed total swallows `1.5 Protection of Services`
   (R144,908.18), which the document numbers *after* it.
3. **2.1 Earthworks** — printed total swallows `2.2 Selected Sub Grade`
   (R146,983.50).
4. Item `4..2.5` (double dot), `2.5.4` used twice, two sections numbered
   `SECTION 6`, units drifting between `m2`/`m²`/`m3`/`m³`/`?`.

## Layout

```
static/theme.css         design tokens + all styling (edit this to retune)
.streamlit/config.toml   base Streamlit theme colours
boq_engine/extract.py    PDF -> tree            (deterministic, no model)
boq_engine/template.py   layout detect + align + write + totals
boq_engine/validate.py   arithmetic + structure checks
boq_engine/profiles.py   per-consultant template profiles
boq_engine/theme.py      injects the stylesheet, builds finding cards
boq_engine/populate.py   tree -> standalone xlsx + csv
app.py                   Streamlit UI
run.py                   CLI: PDF -> xlsx + csv
```

## Known limits

- Column bands are tuned for A4 portrait Candy output. A different print layout
  needs `BANDS` in `extract.py` re-measured.
- Only one consultant's file has been seen. Alignment is near 1:1 for this one;
  a structurally different template may need its own profile tuning.
- Multi-sheet templates are handled one BOQ-shaped sheet at a time; a template
  that splits one section across two sheets would need a different strategy.
- Description joins can drop a space where Candy's print engine split a word
  (`Fixed-ChargeItems`). Cosmetic.