kanban-dev-webapp/docs/superpowers/specs/2026-09-16-phase-2-commit-ingestion-and-dashboard-design.md

Phase 2 — Commit Ingestion and the Project Dashboard

Date: 2026-09-16 Status: Draft for review Phase: 2 of 5 Depends on: Phase 1 (complete)

---

1. Purpose

Turn a project's git history into something a client can read.

Two outcomes:

• A per-project dashboard showing what has actually been built — commits, features shipped, active days, effort, and where the work went. This is the "proof I'm doing my job" surface.
• Commits attached to cards, so a feature card carries the history of the work done on it. Phase 3 then translates that history into plain English.

This phase does no AI work. It produces the data that Phases 3 and 4 depend on.

2. Prior art: the Hermes build log

The developer already produced a version of this dashboard by hand, from the WFDASH-WEBAPP history, and tested it in conversation. That artifact is the best available specification of what is wanted, and several of its decisions are adopted here directly.

Adopted:

From the build log	Why
Total commits, features shipped, fixes, active period, longest session	The metric set that read well
Activity calendar (commits per day, a year-grid heatmap)	The single most legible element
Hours by month, as bars	Shows the shape of a project over time
Commits by type, from the conventional-commit prefix	Cheap to compute, immediately meaningful
Feature areas by commit volume, from the conventional-commit scope	The same scope signal Phase 1 uses for card linking
Workdays, not calendar days	"60 of 75 workdays (80%)" is the same truth as "60 of 108 calendar days (56%)", told competently
A visible methodology note	Load-bearing here — see §3

Corrected:

Problem in the build log	Correction
Reported "6 contributors": Claude, Dave, David, Mika, SJCoetz, WFDASH Dev — all one person plus AI agents	Every git identity collapses to one developer. The dashboard says 1 developer. See §6.
Two different hour figures (164.6 h, then 218.7 h) from two undocumented methods	One method, defined in §5.3, with its constants shown on the page
Monetary framing (internal cost vs outsourced equivalent)	No monetary figure appears anywhere in this product. See §3.

3. Audience and disclosure

The Hermes build log was written for the developer's employer, to justify effort internally. This dashboard is for clients, and the incentives are different.

Two decisions follow, both settled:

Clients see the effort numbers. Hours, sessions and longest run are shown to client managers, not withheld. The developer chose full transparency deliberately, against a recommendation to keep effort on the owner's view. The consequence is that the figures must survive scrutiny:

• Every effort figure carries the word estimated in the UI, not only in a footnote.
• A Methodology panel is part of the dashboard, not a link to elsewhere. It states the session gap, the padding constants, and in plain words what the number does and does not represent.
• Constants are per project and recorded. Changing them retroactively rewrites history a client has already seen, which reads as massaging the numbers. When they change, the dashboard says so.

No money, anywhere. No hourly rate, no cost-to-date, no outsourced-equivalent comparison, in any view including the owner's. Cost framing is a pitch for a particular audience and belongs in a document written by hand, not in a live dashboard a client can open.

4. Ingestion

4.1 Transport

Forgejo webhook plus an hourly reconciliation poll. Both, not either.

• The webhook fires on push; commits appear in seconds.
• The hourly poll asks the Gitea-compatible API what it missed. This covers webhook delivery failures, and it is the same code path as the initial backfill.

Webhooks alone silently lose data whenever the app restarts mid-push, and silent loss is the worst failure mode here: the dashboard is still plausible, just wrong.

4.2 Webhook authenticity

Forgejo signs each delivery with HMAC-SHA256 over the raw body, using a secret configured on the webhook, sent as X-Gitea-Signature.

The handler must verify that signature against a per-project secret before parsing anything, using a constant-time comparison. An unverified webhook endpoint lets anyone who finds the URL invent commits — and invented commits on a client-facing progress dashboard are worse than no dashboard.

The endpoint is unauthenticated by necessity (Forgejo has no session), so the signature is the only control.

4.3 Backfill

A project links to a repository by repo_owner / repo_name, already captured in Phase 1. Backfill pages the full commit history through the Forgejo API on demand from the owner's admin view.

WFDASH-WEBAPP has 787 commits, so backfill must be paginated, resumable, and idempotent on sha.

4.4 Idempotency

sha is unique per project. Every write is an upsert. A commit arriving by both webhook and poll must produce one row, not two — double-counting inflates every figure on the dashboard simultaneously and consistently, which makes it hard to spot.

5. Derived metrics

5.1 Conventional commits

The developer's history is already well-formed: feat(stock): …, fix(quotes): …, with branches like feat/quote-attachments-split. Parsing type(scope): subject yields both the type breakdown and the feature-area breakdown for free.

Commits that do not parse are counted as type other and are not dropped — a silently shrinking denominator is worse than an untidy bar.

Merge commits are excluded from every count and from session clustering.

This was verified rather than assumed. On WFDASH-WEBAPP today:

all commits:      787
excluding merges: 779
merge commits:      8

The Hermes build log reported 776 on 8 September; three non-merge commits have landed since. So the build log excluded merges, and it was right to: a merge is not a unit of work, it lands at whatever moment a branch happened to be integrated, and it drags session boundaries around. A merge commit is still stored — it is real history — but carries type = "merge" and is filtered out of totals, type breakdowns, and effort.

This is the kind of decision that must be settled once and written down. Two dashboards disagreeing by eight commits, for reasons nobody can reconstruct, is exactly what destroys a client's confidence in the rest of the numbers.

5.2 Active days and workdays

• Active day: a calendar day with at least one commit.
• Workday: Monday to Friday, excluding South African public holidays.

The dashboard leads with active days as a fraction of workdays, not calendar days, because the calendar figure implies absence where there was a weekend.

Public holidays come from a static per-year table covering the project's span. Computing Easter-derived dates was considered and rejected — a table is auditable, and the failure mode of a wrong table is one visibly odd day rather than a plausible-looking systematic drift.

5.3 Effort estimation

Git records no duration. Effort is inferred, and this section is the answer given when a client asks how a number was produced.

Commits are ordered by author timestamp and grouped into sessions: any two consecutive commits less than SESSION_GAP_MINUTES apart belong to the same session.

For each session:

session_minutes = (last_commit - first_commit)
                + PRE_SESSION_MINUTES      // work before the first commit
                + POST_SESSION_MINUTES     // work after the last commit

Defaults, per project and overridable:

Constant	Default	Reasoning
SESSION_GAP_MINUTES	120	Two hours between commits is a break, not a pause
PRE_SESSION_MINUTES	45	Reading, reproducing, planning before the first commit lands
POST_SESSION_MINUTES	20	Testing, merging, deploying after the last one

The padding is why the Hermes build log's first attempt (164.6 h, unpadded) undercounted relative to its second (218.7 h). A session that ends at its last commit assumes work stops the instant code is committed, which is never true.

What this number is not. It is not a timesheet, it is not billable, and it cannot see work that produced no commit — design, meetings, debugging that ended in a revert, or a day spent reading someone else's code. The dashboard says so in those words.

Derived from sessions: total estimated hours, session count, longest single session (with its commit count and date), and hours per month.

5.4 Per-card effort

A card's commits are its own sessions, computed the same way. A card shows its commit count, its first and last commit, and its estimated hours.

Sessions are computed within a card's commits, not sliced from project sessions. A single session that touched three cards contributes its padding to each, so per-card hours sum to more than project hours. That is stated where per-card hours appear, rather than being quietly reconciled — the alternative is apportioning padding arbitrarily and defending that instead.

6. Authorship

Every git identity on a project maps to one developer. The dashboard reports 1 developer and does not list contributor names.

This is a correction, not a simplification. The build log's "6 contributors" — Claude, Dave, David, Mika, SJCoetz, WFDASH Dev — were one person plus the AI agents they drive. Presenting agent identities as team members states something false to a paying client, and it is the kind of false that is difficult to explain when questioned.

Raw author_name and author_email are still stored on every commit, so the claim can be audited later and so a genuinely new contributor is recoverable.

Known limitation, accepted: if a second real person ever commits to a linked repository, this collapses them into the developer and the dashboard under-reports the team. That is correct today and wrong the moment it is not. Revisit before adding anyone to a repository.

7. Commit-to-card linking

Settled during the Phase 1 design. First match wins:

1. Explicit [#42] in the commit message. An override for when auto-linking is wrong, not a daily habit.
2. The card's scope_rule matching the commit's conventional-commit scope — feat(stock): links to the card owning scope stock. Carries most commits with no change to how the developer works.
3. No match → an Unassigned tray in the UI, where commits are dragged onto a card manually.

Two constraints that would otherwise rot silently:

• A scope rule claims commits only from the card's start_date forward — set by the accept flow in Phase 1c. Without it, a new "Stock pricing v2" card retroactively swallows a year of old feat(stock) commits and claims work it did not do.
• One commit links to at most one card. Where two live cards claim the same scope, the most recently active card wins. Otherwise commits double-count and per-card progress overstates reality.

Re-linking is idempotent and re-runnable: changing a card's scope_rule re-runs matching for that project rather than requiring a re-ingest.

8. Data model additions

New PocketBase collections, following Phase 1 conventions. Applied through scripts/setup-remote.ts, which is the single source of truth for schema.

commits

Field	Type	Notes
project	relation → projects, cascade
sha	text, required	Unique per project
short_sha	text	For display
message	text	Full message, first line used for display
subject	text	First line
type	text	feat, fix, … or other
scope	text	Conventional-commit scope, "" when absent
author_name	text	Raw, unmapped — see §6
author_email	text	Raw, unmapped
authored_at	date	The commit's own timestamp, not ingest time
card	relation → cards	Nullable. Empty means unassigned.
link_source	select	explicit | scope | manual | none
url	url	Link back to Forgejo

Indexes on (project, authored_at), (project, sha) unique, (card).

project_metrics

A computed cache, rebuilt on ingest. The dashboard reads this rather than aggregating 787 rows per page view.

Field	Type	Notes
project	relation → projects, cascade, unique
computed_at	date
totals	json	Commits, by type, by scope
activity	json	Commits per day, for the heatmap
sessions	json	Count, total minutes, longest session
monthly	json	Estimated hours per month
constants	json	The §5.3 values used for this computation

constants is stored rather than read live, so a figure on screen can always be explained by the settings that produced it.

Additions to projects

Field	Type	Notes
webhook_secret	text	Per project. Server-only; never sent to a browser.
session_gap_minutes	number	Default 120
pre_session_minutes	number	Default 45
post_session_minutes	number	Default 20
last_ingest_at	date	Shown on the dashboard so staleness is visible

9. Permissions

Consistent with Phase 1, and constrained by the same PocketBase limitation.

Collection	Read	Write
commits	Owner, and client managers for their own client's projects	Server-side only (superuser)
project_metrics	Same	Server-side only (superuser)

projects.webhook_secret must never reach a browser. Because PocketBase rules are record-level and cannot hide one field, the dashboard and admin views read projects through a server component that strips it, and the secret is only ever read server-side by the webhook handler.

The webhook endpoint itself is unauthenticated and protected by the HMAC signature in §4.2.

10. The dashboard

Route: /p/[clientSlug]/[projectSlug]/dashboard, reachable from the board.

Sections, in order:

1. KPI row — commits, features shipped, estimated hours, active days as a fraction of workdays. Number, label, and a sparkline where one helps. This is the pattern from the developer's second reference image, and this is the page it was always meant for.
2. Activity calendar — a day-grid heatmap of commits per day across the project's span. The strongest element in the Hermes log.
3. Estimated hours by month — bars.
4. Commits by type and feature areas by scope — two breakdowns, horizontal bars, folded past six with an "other" row.
5. Recent activity — the last N commits, subject and date, linked to Forgejo.
6. Methodology — always rendered, never collapsed by default. States the constants, what the estimate includes, and what it cannot see.

Freshness is shown. last_ingest_at appears on the page. A dashboard that is quietly twelve hours stale, with no indication, is worse than one that says so.

Charts are hand-written SVG, not a charting library. Three chart types are needed, all simple, and they must follow the Phase 1b design tokens in both themes — which is precisely what chart libraries make awkward. This also avoids adding a dependency to render four bars.

11. Decisions deliberately deferred

• AI humanisation of commit messages — Phase 3. This phase stores raw messages and computes numbers; it translates nothing.
• The client intake wizard — Phase 4, and it depends on this phase's velocity data to ground its date proposals.
• Cross-project roadmap — Phase 5.
• Author aliasing UI — not built. Everything collapses to one developer per §6. Needed only when a second real person contributes.
• Per-card commit history in the card detail view — the linking is built here, the card-detail rendering of it lands with Phase 3, where those commits finally have readable summaries to show.

12. Acceptance criteria

1. A push to a linked Forgejo repository appears in commits within seconds, and a delivery with a wrong or missing signature is rejected.
2. Backfilling WFDASH-WEBAPP ingests all 787 commits, is resumable, and running it twice produces no duplicates.
3. Commits link to cards by explicit tag and by scope rule, honouring start_date bounding and one-commit-one-card.
4. Unlinked commits are listed in an Unassigned tray and can be attached to a card by hand.
5. The dashboard shows the §10 sections, matching a hand-computed figure for at least one project on commits, features, and active days.
6. Every effort figure is labelled as estimated, and the methodology panel states the constants actually used.
7. No monetary figure appears in any view.
8. The dashboard reports 1 developer regardless of how many git identities appear in the history.
9. webhook_secret appears in no browser payload.

13. Suggested plan split

This phase is too large for one implementation plan. Suggested:

• Plan 2a — ingestion. Schema, Forgejo API client, signature verification, webhook handler, reconciliation poll, backfill, conventional-commit parsing. Verifiable by ingesting a real repository and counting rows.
• Plan 2b — linking. The precedence chain, start_date bounding, one-commit-one-card arbitration, re-linking, the Unassigned tray.
• Plan 2c — metrics and dashboard. Session clustering, workday calculation, the metrics cache, and the page. The pure-logic parts here — sessions, workdays, conventional-commit parsing — are the most testable code in the entire project and should be built test-first.