From 3f9bbb783a2c456b3947d3e8a3d803d42003d133 Mon Sep 17 00:00:00 2001 From: Dave Date: Wed, 16 Sep 2026 11:41:57 +0200 Subject: [PATCH] docs(plan): Phase 2a implementation plan for commit ingestion Eleven tasks covering conventional-commit parsing, HMAC webhook verification, the Forgejo client, a shared idempotent ingest core, the webhook and reconciliation endpoints, and the backfill admin panel. The Forgejo API shape, pagination headers and merge counts are measured against the live instance rather than recalled, and the plan says so, so the implementer builds against facts. Surfaces one unresolved issue for decision: projects.webhook_secret is readable by a client manager straight from PocketBase, because rules are record-level and cannot hide a field. Co-Authored-By: Claude Opus 5 --- .../2026-09-16-phase-2a-commit-ingestion.md | 1786 +++++++++++++++++ 1 file changed, 1786 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-16-phase-2a-commit-ingestion.md diff --git a/docs/superpowers/plans/2026-09-16-phase-2a-commit-ingestion.md b/docs/superpowers/plans/2026-09-16-phase-2a-commit-ingestion.md new file mode 100644 index 0000000..0eb4742 --- /dev/null +++ b/docs/superpowers/plans/2026-09-16-phase-2a-commit-ingestion.md @@ -0,0 +1,1786 @@ +# Phase 2a — Commit Ingestion Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Get every commit from a project's Forgejo repository into the database — by webhook for immediacy, by hourly reconciliation for reliability, and by backfill for history — parsed into type and scope. + +**Architecture:** A pure parsing layer with no I/O, a thin Forgejo API client, and one shared `ingestCommits` core that the webhook, the poll and the backfill all call. Writes are upserts keyed on `(project, sha)`, so the same commit arriving down three paths produces one row. The webhook endpoint is unauthenticated by necessity and is protected by HMAC signature verification. + +**Tech Stack:** Next.js 16 route handlers · PocketBase 0.30 · Forgejo 8.0.3 (Gitea 1.22 API) · vitest + +**Spec:** `docs/superpowers/specs/2026-09-16-phase-2-commit-ingestion-and-dashboard-design.md` §4, §5.1, §8 + +**Prerequisite:** Phase 1 complete. + +--- + +## The Forgejo API, verified + +These are measured against the live instance, not recalled from documentation. +Build against them. + +**Version:** `{"version":"8.0.3+gitea-1.22.0"}` — the Gitea 1.22 API shape. + +**Endpoint:** `GET /api/v1/repos/{owner}/{repo}/commits` + +**Query parameters that matter:** + +| Parameter | Use | +|---|---| +| `page`, `limit` | Paginate. `limit` caps at 50 on a default install. | +| `stat=false`, `files=false`, `verification=false` | Trim the payload. Each commit otherwise carries its full file list. | +| `sha` | Branch or commit to start from. Omit for the default branch. | + +**Pagination headers** — present and reliable, so there is no need to guess when +to stop: + +``` +X-Total-Count: 787 +X-Page: 2 +X-PageCount: 787 +X-HasMore: true +``` + +**Response shape**, confirmed against a real commit: + +```json +{ + "sha": "1bc0da40e773daebaa056baf01d03b703b6833f2", + "html_url": "http://…/David/WFDASH-WEBAPP/commit/1bc0da40…", + "created": "2026-09-16T11:04:39+02:00", + "commit": { + "message": "feat(customers): lift active/inactive toggle…\n\nfull body…\n", + "author": { "name": "Dave", "email": "david.els1@outlook.com", "date": "2026-09-16T11:04:39+02:00" }, + "committer": { "name": "Dave", "email": "david.els1@outlook.com", "date": "2026-09-16T11:04:39+02:00" } + }, + "parents": [ { "sha": "…" } ], + "stats": { "total": 104, "additions": 69, "deletions": 35 } +} +``` + +Two things to note: + +- **`commit.author.date` is the authoring time** and is what effort estimation + must use. `created` and `commit.committer.date` shift on rebase and would make + a rebased branch look like one enormous session. +- **`parents.length > 1` means a merge commit.** That is the authoritative test — + not a "Merge branch" string match, which catches a commit that merely mentions + merging. Verified on WFDASH-WEBAPP: 787 commits, 779 non-merge, 8 merges. + +**Authentication:** the instance currently serves this endpoint anonymously. The +client still sends `Authorization: token ` when the env var is +set, so a private repository works without a code change. + +--- + +## File structure + +``` +src/ +├── lib/ +│ └── git/ +│ ├── conventional.ts # message -> type / scope / subject +│ ├── conventional.test.ts +│ ├── signature.ts # webhook HMAC verification +│ ├── signature.test.ts +│ ├── forgejo.ts # API client +│ └── ingest.ts # shared upsert core +└── app/ + ├── api/ + │ ├── webhooks/forgejo/[projectId]/route.ts + │ └── cron/reconcile/route.ts + ├── actions/ + │ └── ingest.ts # owner: backfill, rotate secret + └── admin/projects/ + └── [projectId]/page.tsx # repo linking + webhook setup +``` + +--- + +## Task 1: Conventional-commit parsing + +Pure, no I/O, test-first. Drives both the type breakdown and the scope-based +card linking in Plan 2b. + +**Files:** +- Create: `src/lib/git/conventional.ts` +- Test: `src/lib/git/conventional.test.ts` + +- [ ] **Step 1: Write the failing tests** + +`src/lib/git/conventional.test.ts`: + +```ts +import { describe, it, expect } from "vitest"; +import { parseCommitMessage } from "@/lib/git/conventional"; + +describe("parseCommitMessage", () => { + it("pulls out type, scope and subject", () => { + expect(parseCommitMessage("feat(stock): inline qty editing")).toEqual({ + type: "feat", + scope: "stock", + subject: "inline qty editing", + breaking: false, + }); + }); + + it("handles a missing scope", () => { + expect(parseCommitMessage("fix: correct the total")).toEqual({ + type: "fix", + scope: "", + subject: "correct the total", + breaking: false, + }); + }); + + it("flags a breaking change with and without a scope", () => { + expect(parseCommitMessage("feat!: drop v1 API").breaking).toBe(true); + expect(parseCommitMessage("feat(api)!: drop v1").breaking).toBe(true); + expect(parseCommitMessage("feat(api)!: drop v1").scope).toBe("api"); + }); + + it("reads only the first line", () => { + const message = + "feat(customers): lift the toggle\n\nA long body that\nmentions fix(other): things\n"; + expect(parseCommitMessage(message)).toEqual({ + type: "feat", + scope: "customers", + subject: "lift the toggle", + breaking: false, + }); + }); + + it("lowercases the type and the scope", () => { + const parsed = parseCommitMessage("Feat(Stock): thing"); + expect(parsed.type).toBe("feat"); + expect(parsed.scope).toBe("stock"); + }); + + it("classifies an unparseable message as other, keeping the subject", () => { + expect(parseCommitMessage("just some words")).toEqual({ + type: "other", + scope: "", + subject: "just some words", + breaking: false, + }); + }); + + it("does not accept a space before the parenthesis", () => { + // "feat (stock): x" is not a conventional commit. Accepting it would make + // the scope breakdown depend on invisible whitespace. + expect(parseCommitMessage("feat (stock): x").type).toBe("other"); + }); + + it("accepts any single lowercase word as a type, but not a phrase", () => { + // "note" is not a conventional type, but the parser cannot tell an unknown + // type from prose shaped like one, and it must not police a whitelist — + // see the note after Step 4. + expect(parseCommitMessage("note: this is not conventional").type).toBe("note"); + // A space before the colon means it is a sentence, not a prefix. + expect(parseCommitMessage("TODO tomorrow: finish").type).toBe("other"); + }); + + it("rejects a type containing non-letters", () => { + expect(parseCommitMessage("feat2(x): thing").type).toBe("other"); + }); + + it("trims and caps an over-long subject", () => { + const parsed = parseCommitMessage(`feat(x): ${"s".repeat(400)}`); + expect(parsed.subject).toHaveLength(200); + }); + + it("returns other for an empty message", () => { + expect(parseCommitMessage("").type).toBe("other"); + expect(parseCommitMessage(" \n ").subject).toBe(""); + }); + + it("handles a scope containing a hyphen or slash", () => { + expect(parseCommitMessage("fix(delivery-notes): x").scope).toBe("delivery-notes"); + expect(parseCommitMessage("fix(api/v1): x").scope).toBe("api/v1"); + }); +}); +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +```bash +npm test +``` + +Expected: FAIL — `Failed to resolve import "@/lib/git/conventional"`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/git/conventional.ts`: + +```ts +/** + * Conventional-commit parsing. + * + * Two dashboard figures depend on this — the type breakdown and the feature + * areas by scope — and Plan 2b links commits to cards by scope. So the rule + * has to be strict and predictable rather than forgiving: a parser that + * sometimes accepts "feat (stock):" makes the scope chart depend on invisible + * whitespace, and nobody would ever find that. + * + * Anything that does not match is type "other" and is counted, never dropped. + * A silently shrinking denominator is worse than an untidy bar. + */ + +export interface ParsedCommit { + /** Lowercased conventional type, or "other". */ + type: string; + /** Lowercased scope, or "" when absent. */ + scope: string; + /** First line after the prefix, trimmed and capped. */ + subject: string; + breaking: boolean; +} + +export const MAX_SUBJECT_LENGTH = 200; + +/** + * type, optional (scope), optional !, colon, space. + * The type is letters only — "feat2" is not a type, it is prose. + */ +const CONVENTIONAL = /^([a-zA-Z]+)(?:\(([^)]+)\))?(!?):\s+(.*)$/; + +export function parseCommitMessage(message: string): ParsedCommit { + const firstLine = (message ?? "").split("\n")[0].trim(); + + const match = CONVENTIONAL.exec(firstLine); + if (!match) { + return { + type: "other", + scope: "", + subject: firstLine.slice(0, MAX_SUBJECT_LENGTH), + breaking: false, + }; + } + + const [, type, scope, bang, subject] = match; + + return { + type: type.toLowerCase(), + scope: (scope ?? "").toLowerCase().trim(), + subject: subject.trim().slice(0, MAX_SUBJECT_LENGTH), + breaking: bang === "!", + }; +} +``` + +- [ ] **Step 4: Run the tests to verify they pass** + +```bash +npm test +``` + +Expected: PASS — 12 tests in `conventional.test.ts`. + +Note the `"note: this is not conventional"` case returns type `note`, not +`other`. That is correct and intended: the parser cannot tell an unknown type +from prose that happens to be shaped like one, and inventing a whitelist of +"real" types would silently reclassify any new prefix the developer starts +using. Unknown types land in the chart's "other" bucket at render time, where +the folding rule already lives. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/git +git commit -m "feat(git): strict conventional-commit parsing" +``` + +--- + +## Task 2: Webhook signature verification + +The only thing standing between the ingest endpoint and anyone who finds its +URL. Pure, test-first. + +**Files:** +- Create: `src/lib/git/signature.ts` +- Test: `src/lib/git/signature.test.ts` + +- [ ] **Step 1: Write the failing tests** + +`src/lib/git/signature.test.ts`: + +```ts +import { describe, it, expect } from "vitest"; +import { verifyWebhookSignature, signatureHeader } from "@/lib/git/signature"; + +// Computed independently with: +// crypto.createHmac('sha256', SECRET).update(BODY, 'utf8').digest('hex') +const SECRET = "whsec_test_2a6f9c"; +const BODY = '{"ref":"refs/heads/main","commits":[{"id":"abc123"}]}'; +const VALID = "9a27e804fac4e850127587433949087f44f3d30cc9b92a0bb956dc5a794e073a"; + +describe("verifyWebhookSignature", () => { + it("accepts a correct signature", () => { + expect(verifyWebhookSignature(BODY, SECRET, VALID)).toBe(true); + }); + + it("rejects a signature for a different body", () => { + expect(verifyWebhookSignature(`${BODY} `, SECRET, VALID)).toBe(false); + }); + + it("rejects a signature made with a different secret", () => { + expect(verifyWebhookSignature(BODY, "wrong-secret", VALID)).toBe(false); + }); + + it("rejects a missing or empty signature", () => { + expect(verifyWebhookSignature(BODY, SECRET, null)).toBe(false); + expect(verifyWebhookSignature(BODY, SECRET, "")).toBe(false); + }); + + it("rejects a signature of the wrong length without throwing", () => { + // Node's timingSafeEqual throws on length mismatch. An uncaught throw here + // becomes a 500, which tells an attacker their guess had the wrong shape. + expect(() => verifyWebhookSignature(BODY, SECRET, "abc")).not.toThrow(); + expect(verifyWebhookSignature(BODY, SECRET, "abc")).toBe(false); + }); + + it("rejects non-hex characters without throwing", () => { + const sameLength = "z".repeat(VALID.length); + expect(() => verifyWebhookSignature(BODY, SECRET, sameLength)).not.toThrow(); + expect(verifyWebhookSignature(BODY, SECRET, sameLength)).toBe(false); + }); + + it("refuses to verify when no secret is configured", () => { + // A project without a secret must fail closed. Treating "no secret" as + // "no check required" turns every unconfigured project into an open endpoint. + expect(verifyWebhookSignature(BODY, "", VALID)).toBe(false); + }); + + it("accepts an upper-case hex signature", () => { + expect(verifyWebhookSignature(BODY, SECRET, VALID.toUpperCase())).toBe(true); + }); +}); + +describe("signatureHeader", () => { + it("reads the Forgejo header", () => { + const headers = new Headers({ "x-forgejo-signature": VALID }); + expect(signatureHeader(headers)).toBe(VALID); + }); + + it("falls back to the Gitea header", () => { + const headers = new Headers({ "x-gitea-signature": VALID }); + expect(signatureHeader(headers)).toBe(VALID); + }); + + it("returns null when neither is present", () => { + expect(signatureHeader(new Headers())).toBeNull(); + }); +}); +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +```bash +npm test +``` + +Expected: FAIL — `Failed to resolve import "@/lib/git/signature"`. + +- [ ] **Step 3: Write the implementation** + +`src/lib/git/signature.ts`: + +```ts +import { createHmac, timingSafeEqual } from "node:crypto"; + +/** + * Webhook authenticity. + * + * The ingest endpoint cannot be authenticated the normal way — Forgejo has no + * session with us — so this signature is the only control on it. Without it, + * anyone who learns the URL can post invented commits, and invented commits on + * a progress dashboard a client reads are worse than having no dashboard. + * + * Forgejo signs the RAW request body with HMAC-SHA256 under the webhook's + * secret and sends the hex digest. Verify before parsing: JSON.parse on an + * unverified body is already doing work for an attacker. + */ + +/** Forgejo sends its own header and the Gitea-compatible one. Accept either. */ +export function signatureHeader(headers: Headers): string | null { + return ( + headers.get("x-forgejo-signature") ?? + headers.get("x-gitea-signature") ?? + null + ); +} + +export function verifyWebhookSignature( + rawBody: string, + secret: string, + provided: string | null +): boolean { + // Fail closed. A project with no secret configured is not a project that + // skips the check — it is a project whose webhook must not work yet. + if (!secret) return false; + if (!provided) return false; + + const expected = createHmac("sha256", secret) + .update(rawBody, "utf8") + .digest("hex"); + + const a = Buffer.from(expected, "hex"); + const b = Buffer.from(provided.toLowerCase(), "hex"); + + // Buffer.from silently truncates invalid hex, so compare lengths explicitly. + // timingSafeEqual throws on a length mismatch, and an uncaught throw becomes + // a 500 that tells an attacker their guess had the wrong shape. + if (a.length !== b.length || a.length === 0) return false; + + return timingSafeEqual(a, b); +} +``` + +- [ ] **Step 4: Run the tests to verify they pass** + +```bash +npm test +``` + +Expected: PASS — 11 tests in `signature.test.ts`. + +- [ ] **Step 5: Commit** + +```bash +git add src/lib/git/signature.ts src/lib/git/signature.test.ts +git commit -m "feat(git): timing-safe webhook signature verification" +``` + +--- + +## Task 3: Schema + +**Files:** +- Modify: `scripts/setup-remote.ts` +- Modify: `src/lib/pb/types.ts` + +**No migration file is written for this collection.** `scripts/setup-remote.ts` +is the single source of truth for schema, and `pb_migrations/` never executes — +see `pocketbase/pb_migrations/README.md`. A migration mirroring this collection +would be sixty lines of code that can never run and can silently disagree with +the code that does. + +- [ ] **Step 1: Add the `commits` collection to `setup-remote.ts`** + +Insert before the `console.log("Applying API rules...")` line, following the +shape of the existing `getOrCreateCollection` calls: + +```ts + // commits + const projectsIdForCommits = (await pb.collections.getOne("projects")).id; + const cardsIdForCommits = (await pb.collections.getOne("cards")).id; + await getOrCreateCollection(pb, { + name: "commits", + type: "base", + listRule: AUTHED, + viewRule: AUTHED, + createRule: null, + updateRule: null, + deleteRule: null, + fields: [ + { + name: "project", + type: "relation", + required: true, + collectionId: projectsIdForCommits, + cascadeDelete: true, + minSelect: 0, + maxSelect: 1, + }, + { name: "sha", type: "text", required: true, max: 64 }, + { name: "short_sha", type: "text", required: false, max: 12 }, + { name: "message", type: "text", required: false, max: 50000 }, + { name: "subject", type: "text", required: false, max: 300, presentable: true }, + { name: "type", type: "text", required: false, max: 30 }, + { name: "scope", type: "text", required: false, max: 60 }, + { name: "author_name", type: "text", required: false, max: 200 }, + { name: "author_email", type: "text", required: false, max: 200 }, + { name: "authored_at", type: "date", required: true }, + { name: "is_merge", type: "bool", required: false }, + { name: "additions", type: "number", required: false }, + { name: "deletions", type: "number", required: false }, + { + name: "card", + type: "relation", + required: false, + collectionId: cardsIdForCommits, + cascadeDelete: false, + minSelect: 0, + maxSelect: 1, + }, + { + name: "link_source", + type: "select", + required: false, + maxSelect: 1, + values: ["explicit", "scope", "manual", "none"], + }, + { name: "url", type: "url", required: false }, + { name: "created", type: "autodate", onCreate: true, onUpdate: false }, + { name: "updated", type: "autodate", onCreate: true, onUpdate: true }, + ], + indexes: [ + "CREATE UNIQUE INDEX idx_commits_project_sha ON commits (project, sha)", + "CREATE INDEX idx_commits_project_date ON commits (project, authored_at)", + "CREATE INDEX idx_commits_card ON commits (card)", + ], + }); +``` + +The unique index on `(project, sha)` is what makes ingestion idempotent. A +commit arriving by webhook, by poll and by backfill must produce one row — if it +produces three, every figure on the dashboard inflates together and +consistently, which is very hard to notice. + +- [ ] **Step 2: Add the project fields** + +In the same file, extend the projects collection. Add a helper beside +`ensureUsersFields` — projects already exists by the time this runs, so the +fields must be added rather than declared: + +```ts +async function ensureProjectIngestFields(pb: PocketBase): Promise { + const projects = await pb.collections.getOne("projects"); + const have = new Set(projects.fields.map((f: { name: string }) => f.name)); + + const additions = [ + { name: "webhook_secret", type: "text", required: false, max: 80 }, + { name: "last_ingest_at", type: "date", required: false }, + ]; + + const fields = [...(projects.fields as unknown[])]; + let changed = false; + + for (const field of additions) { + if (!have.has(field.name)) { + fields.push({ ...field, presentable: false, hidden: false, system: false }); + changed = true; + } + } + + if (changed) { + await pb.collections.update(projects.id, { fields }); + console.log(" + projects fields: webhook_secret, last_ingest_at"); + } else { + console.log(" = projects ingest fields already present"); + } +} +``` + +Call it immediately after `await ensureUsersFields(pb);`. + +The session-tuning constants from the spec (`session_gap_minutes` and the two +padding values) are **not** added here. They are first read in Plan 2c, and a +field nothing reads is a field nobody maintains. + +- [ ] **Step 3: Apply the rules** + +In the rules block at the end of `main()`, add: + +```ts + await applyRules(pb, "commits", { + listRule: scopedRead("project.client"), + viewRule: scopedRead("project.client"), + createRule: null, + updateRule: null, + deleteRule: null, + }); +``` + +Client managers read their own client's commits — they see the dashboard. +Nobody writes from a browser; ingestion is server-side only. + +- [ ] **Step 4: Add the record type** + +In `src/lib/pb/types.ts`: + +```ts +export type CommitLinkSource = "explicit" | "scope" | "manual" | "none"; + +export interface CommitRecord extends BaseRecord { + project: string; + sha: string; + short_sha: string; + message: string; + subject: string; + type: string; + scope: string; + author_name: string; + author_email: string; + /** Authoring time, not commit or ingest time. See the API notes. */ + authored_at: string; + is_merge: boolean; + additions: number; + deletions: number; + card: string; + link_source: CommitLinkSource; + url: string; +} +``` + +And extend `ProjectRecord`: + +```ts + /** Server-only. Must never appear in a browser payload. */ + webhook_secret: string; + last_ingest_at: string; +``` + +- [ ] **Step 5: Apply and verify** + +```bash +set -a && source .env.local && set +a && npm run setup-remote +``` + +Expected: the script reports creating `commits` and adding the two projects +fields. Confirm in the PocketBase admin UI that the unique index on +`(project, sha)` exists. + +- [ ] **Step 7: Commit** + +```bash +git add scripts/setup-remote.ts pocketbase/pb_migrations src/lib/pb/types.ts +git commit -m "feat(schema): commits collection and project ingest fields" +``` + +--- + +## Task 4: Forgejo API client + +**Files:** +- Create: `src/lib/git/forgejo.ts` +- Modify: `.env.example` + +- [ ] **Step 1: Write the client** + +`src/lib/git/forgejo.ts`: + +```ts +import "server-only"; + +/** + * Forgejo REST client, against the Gitea 1.22 API shape. + * + * Everything here is measured against the live instance rather than recalled: + * the pagination headers, the query parameters, and the response fields are + * all confirmed. See the header of the Phase 2a plan. + */ + +export interface ForgejoCommit { + sha: string; + html_url: string; + commit: { + message: string; + author: { name: string; email: string; date: string }; + committer: { name: string; email: string; date: string }; + }; + parents: Array<{ sha: string }>; + stats?: { total: number; additions: number; deletions: number }; +} + +export interface CommitPage { + commits: ForgejoCommit[]; + page: number; + pageCount: number; + total: number; + hasMore: boolean; +} + +/** Forgejo caps limit at 50 on a default install; asking for more silently caps. */ +export const PAGE_SIZE = 50; + +function baseUrl(): string { + const url = process.env.FORGEJO_URL; + if (!url) throw new Error("FORGEJO_URL is not set"); + return url.replace(/\/+$/, ""); +} + +function headers(): HeadersInit { + const token = process.env.FORGEJO_TOKEN; + return { + accept: "application/json", + // The instance currently serves commits anonymously. Sending a token when + // one exists means a private repository works with no code change. + ...(token ? { authorization: `token ${token}` } : {}), + }; +} + +/** + * One page of commits, newest first. + * + * stat/files/verification are disabled: each commit otherwise carries its full + * file list, which for a backfill of several hundred commits is a great deal + * of payload for data this phase does not use. + */ +export async function fetchCommitPage( + owner: string, + repo: string, + page: number +): Promise { + const url = + `${baseUrl()}/api/v1/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/commits` + + `?page=${page}&limit=${PAGE_SIZE}&stat=false&files=false&verification=false`; + + const response = await fetch(url, { headers: headers(), cache: "no-store" }); + + if (!response.ok) { + throw new Error( + `Forgejo returned ${response.status} for ${owner}/${repo} page ${page}` + ); + } + + const commits = (await response.json()) as ForgejoCommit[]; + + return { + commits, + page: Number(response.headers.get("x-page") ?? page), + pageCount: Number(response.headers.get("x-pagecount") ?? 1), + total: Number(response.headers.get("x-total-count") ?? commits.length), + hasMore: response.headers.get("x-hasmore") === "true", + }; +} + +/** Confirms a repository exists and is readable before a backfill is started. */ +export async function repoExists(owner: string, repo: string): Promise { + const url = `${baseUrl()}/api/v1/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}`; + const response = await fetch(url, { headers: headers(), cache: "no-store" }); + return response.ok; +} +``` + +- [ ] **Step 2: Add the environment variables** + +Append to `.env.example`: + +``` +# Forgejo, for commit ingestion (Phase 2) +FORGEJO_URL=https://forgejo.example.com +# Optional. Only needed for private repositories. +FORGEJO_TOKEN= +# Shared secret for the hourly reconciliation endpoint. Generate a long random one. +CRON_SECRET=replace-me +``` + +Set the real values in `.env.local`. + +- [ ] **Step 3: Verify against the live instance** + +```bash +set -a && source .env.local && set +a && node -e " +fetch(process.env.FORGEJO_URL.replace(/\/+\$/,'') + '/api/v1/repos/David/WFDASH-WEBAPP/commits?limit=1&stat=false&files=false') + .then(r => { console.log('status', r.status, 'total', r.headers.get('x-total-count')); return r.json(); }) + .then(a => console.log('first sha', a[0].sha, '|', a[0].commit.message.split('\n')[0])); +" +``` + +Expected: `status 200 total 787` and the newest commit's subject. + +- [ ] **Step 4: Commit** + +```bash +git add src/lib/git/forgejo.ts .env.example +git commit -m "feat(git): Forgejo commit API client" +``` + +--- + +## Task 5: The shared ingest core + +One function that all three entry points call. Keeping the upsert in one place +is what makes "the same commit down three paths produces one row" a property of +the system rather than a hope. + +**Files:** +- Create: `src/lib/git/ingest.ts` + +- [ ] **Step 1: Write the core** + +`src/lib/git/ingest.ts`: + +```ts +import "server-only"; + +import { getAdminPb } from "@/lib/pb/admin"; +import { parseCommitMessage } from "@/lib/git/conventional"; +import { fetchCommitPage, type ForgejoCommit } from "@/lib/git/forgejo"; +import type { CommitRecord, ProjectRecord } from "@/lib/pb/types"; + +export interface IngestResult { + created: number; + skipped: number; + pagesRead: number; +} + +/** + * Turn one Forgejo commit into a row. + * + * `commit.author.date` — the authoring time — not `created` and not + * `committer.date`. The latter two move on rebase, and a rebased branch would + * then look like one enormous coding session in Plan 2c's effort estimate. + * + * `parents.length > 1` is the authoritative merge test. Matching on a + * "Merge branch" message prefix would also catch a commit that merely talks + * about merging, and would miss a merge with a rewritten message. + */ +function toRecord(projectId: string, commit: ForgejoCommit) { + const parsed = parseCommitMessage(commit.commit.message); + const isMerge = (commit.parents?.length ?? 0) > 1; + + return { + project: projectId, + sha: commit.sha, + short_sha: commit.sha.slice(0, 7), + message: commit.commit.message.slice(0, 50000), + subject: parsed.subject, + // A merge is real history but is not a unit of work. Typing it here keeps + // every consumer from having to remember to exclude it. + type: isMerge ? "merge" : parsed.type, + scope: parsed.scope, + author_name: commit.commit.author.name, + author_email: commit.commit.author.email, + authored_at: commit.commit.author.date, + is_merge: isMerge, + additions: commit.stats?.additions ?? 0, + deletions: commit.stats?.deletions ?? 0, + link_source: "none" as const, + url: commit.html_url, + }; +} + +/** + * Does this PocketBase error mean "that sha is already stored"? + * + * PocketBase answers 400 for EVERY validation failure, so status alone cannot + * distinguish a duplicate from a genuinely malformed record. Treating all 400s + * as duplicates would silently drop a commit that failed for any other reason — + * an over-long field, say — count it as "skipped", and under-report the + * dashboard with no error anywhere. That is exactly the kind of quiet wrongness + * this phase must not have. + */ +function isDuplicateSha(error: unknown): boolean { + const e = error as { status?: number; data?: { data?: Record } }; + if (e?.status !== 400) return false; + return e?.data?.data?.sha?.code === "validation_not_unique"; +} + +/** + * Store a batch, skipping anything already present. + * + * Existing shas are read in one query rather than discovered through failed + * creates — a backfill would otherwise generate 787 deliberate errors. The + * unique-violation catch remains as a race backstop for the case where the + * webhook and the hourly poll process the same commit concurrently, which the + * pre-read cannot prevent. + */ +export async function storeCommits( + projectId: string, + commits: ForgejoCommit[] +): Promise<{ created: number; skipped: number }> { + if (commits.length === 0) return { created: 0, skipped: 0 }; + + const pb = await getAdminPb(); + + const filter = commits.map((c) => `sha = "${c.sha}"`).join(" || "); + const existing = await pb.collection("commits").getFullList({ + filter: `project = "${projectId}" && (${filter})`, + fields: "sha", + }); + const known = new Set(existing.map((r) => r.sha as string)); + + let created = 0; + let skipped = 0; + + for (const commit of commits) { + if (known.has(commit.sha)) { + skipped += 1; + continue; + } + + try { + await pb.collection("commits").create(toRecord(projectId, commit)); + created += 1; + } catch (error: unknown) { + if (isDuplicateSha(error)) { + skipped += 1; + continue; + } + // Anything else is a real problem. Let it surface: the webhook returns + // 500 so Forgejo shows the delivery as failed, and the backfill stops + // rather than reporting a total that quietly omits rows. + throw error; + } + } + + return { created, skipped }; +} + +/** + * Walk a repository from the newest commit backwards. + * + * `stopAfterKnownPages` decides the mode: + * - backfill passes Infinity and reads to the end + * - reconciliation passes 1 and stops once a whole page is already known + * + * The Forgejo commits endpoint has no `since` parameter, so "what is new" is + * necessarily expressed as "walk newest-first until everything is familiar". + * Stopping on a whole page rather than on the first known sha tolerates a + * commit arriving out of order between the webhook and the poll. + */ +export async function ingestProject( + project: Pick, + options: { stopAfterKnownPages: number } = { stopAfterKnownPages: 1 } +): Promise { + if (!project.repo_owner || !project.repo_name) { + throw new Error("This project has no repository linked"); + } + + let page = 1; + let created = 0; + let skipped = 0; + let consecutiveKnownPages = 0; + + for (;;) { + const result = await fetchCommitPage( + project.repo_owner, + project.repo_name, + page + ); + + if (result.commits.length === 0) break; + + const stored = await storeCommits(project.id, result.commits); + created += stored.created; + skipped += stored.skipped; + + if (stored.created === 0) { + consecutiveKnownPages += 1; + if (consecutiveKnownPages >= options.stopAfterKnownPages) break; + } else { + consecutiveKnownPages = 0; + } + + if (!result.hasMore || page >= result.pageCount) break; + page += 1; + } + + const pb = await getAdminPb(); + await pb + .collection("projects") + .update(project.id, { last_ingest_at: new Date().toISOString() }); + + return { created, skipped, pagesRead: page }; +} + +/** Every project with a repository linked. Used by the hourly reconciliation. */ +export async function projectsWithRepos(): Promise { + const pb = await getAdminPb(); + return pb.collection("projects").getFullList({ + filter: 'repo_owner != "" && repo_name != "" && status != "archived"', + }); +} + +``` + +- [ ] **Step 2: Verify the build** + +```bash +npm run build +``` + +Expected: build succeeds. + +- [ ] **Step 3: Commit** + +```bash +git add src/lib/git/ingest.ts +git commit -m "feat(git): shared idempotent ingest core" +``` + +--- + +## Task 6: The webhook endpoint + +**Files:** +- Create: `src/app/api/webhooks/forgejo/[projectId]/route.ts` + +- [ ] **Step 1: Write the handler** + +`src/app/api/webhooks/forgejo/[projectId]/route.ts`: + +```ts +import { getAdminPb } from "@/lib/pb/admin"; +import { signatureHeader, verifyWebhookSignature } from "@/lib/git/signature"; +import { ingestProject } from "@/lib/git/ingest"; +import { jsonError, jsonOk } from "@/lib/http/json"; +import type { ProjectRecord } from "@/lib/pb/types"; + +/** + * POST /api/webhooks/forgejo/:projectId + * + * Unauthenticated by necessity — Forgejo holds no session with us — so the + * HMAC signature is the only control on this endpoint. Order matters: + * + * 1. read the RAW body as text + * 2. verify the signature against this project's secret + * 3. only then parse + * + * Parsing before verifying is already doing work on behalf of whoever sent it. + * + * The payload's own commit list is deliberately ignored. It is convenient and + * it is a liability: trusting it means a forged-but-correctly-signed-by-a-leaked + * -secret request could inject arbitrary history. Instead the webhook is + * treated purely as a signal that something changed, and the commits are then + * pulled from the Forgejo API, which is authoritative. + */ +export async function POST( + request: Request, + context: { params: Promise<{ projectId: string }> } +): Promise { + const { projectId } = await context.params; + + // Raw text, before any parsing — the signature covers these exact bytes. + const rawBody = await request.text(); + + const pb = await getAdminPb(); + + let project: ProjectRecord; + try { + project = await pb.collection("projects").getOne(projectId); + } catch { + // Same answer as a bad signature, so the endpoint cannot be used to + // discover which project ids exist. + return jsonError("Unauthorized", 401); + } + + const provided = signatureHeader(request.headers); + + if (!verifyWebhookSignature(rawBody, project.webhook_secret, provided)) { + console.warn(`[webhook] rejected delivery for project ${projectId}`); + return jsonError("Unauthorized", 401); + } + + try { + const result = await ingestProject(project, { stopAfterKnownPages: 1 }); + console.log( + `[webhook] project=${projectId} created=${result.created} skipped=${result.skipped}` + ); + return jsonOk({ created: result.created, skipped: result.skipped }); + } catch (error) { + console.error(`[webhook] ingest failed for ${projectId}`, error); + // 500 so Forgejo marks the delivery failed and it is visible in its UI. + // The hourly reconciliation will pick the commits up regardless. + return jsonError("Ingest failed", 500); + } +} +``` + +- [ ] **Step 2: Verify the build** + +```bash +npm run build +``` + +Expected: build succeeds; the route list includes +`/api/webhooks/forgejo/[projectId]`. + +- [ ] **Step 3: Commit** + +```bash +git add src/app/api/webhooks +git commit -m "feat(api): signature-verified Forgejo webhook endpoint" +``` + +--- + +## Task 7: The reconciliation endpoint + +**Files:** +- Create: `src/app/api/cron/reconcile/route.ts` + +- [ ] **Step 1: Write the handler** + +`src/app/api/cron/reconcile/route.ts`: + +```ts +import { ingestProject, projectsWithRepos } from "@/lib/git/ingest"; +import { jsonError, jsonOk } from "@/lib/http/json"; + +/** + * POST /api/cron/reconcile — hourly catch-up across every linked project. + * + * Exists because webhooks fail. A delivery lost while the app restarts is gone + * for good, and the dashboard stays plausible while being wrong, which is the + * worst failure available here. This is also the same code path as backfill, + * so the reliable path is the one that gets exercised constantly. + * + * Triggered by a Coolify scheduled task: + * curl -X POST -H "Authorization: Bearer $CRON_SECRET" https:///api/cron/reconcile + * + * One project failing must not stop the rest — a repository that was renamed + * should not silently freeze every other project's dashboard. + */ +export async function POST(request: Request): Promise { + const secret = process.env.CRON_SECRET; + if (!secret) { + console.error("[cron] CRON_SECRET is not set; refusing to run"); + return jsonError("Not configured", 500); + } + + if (request.headers.get("authorization") !== `Bearer ${secret}`) { + return jsonError("Unauthorized", 401); + } + + const projects = await projectsWithRepos(); + const results: Array<{ project: string; created?: number; error?: string }> = []; + + for (const project of projects) { + try { + const result = await ingestProject(project, { stopAfterKnownPages: 1 }); + results.push({ project: project.slug, created: result.created }); + } catch (error) { + const message = error instanceof Error ? error.message : "unknown error"; + console.error(`[cron] ${project.slug} failed: ${message}`); + results.push({ project: project.slug, error: message }); + } + } + + console.log(`[cron] reconciled ${projects.length} projects`); + return jsonOk({ projects: results }); +} +``` + +- [ ] **Step 2: Verify the build** + +```bash +npm run build +``` + +Expected: build succeeds; the route list includes `/api/cron/reconcile`. + +- [ ] **Step 3: Commit** + +```bash +git add src/app/api/cron +git commit -m "feat(api): hourly reconciliation endpoint" +``` + +--- + +## Task 8: Owner actions — backfill and webhook setup + +**Files:** +- Create: `src/app/actions/ingest.ts` + +- [ ] **Step 1: Write the actions** + +`src/app/actions/ingest.ts`: + +```ts +"use server"; + +import { randomBytes } from "node:crypto"; +import { revalidatePath } from "next/cache"; + +import { getAdminPb } from "@/lib/pb/admin"; +import { requireSession } from "@/lib/auth/session"; +import { ingestProject } from "@/lib/git/ingest"; +import { repoExists } from "@/lib/git/forgejo"; +import type { ProjectRecord } from "@/lib/pb/types"; + +/** See the security note at the top of src/app/actions/cards.ts. */ +async function requireOwner() { + const session = await requireSession(); + if (session.role !== "owner") throw new Error("Not found"); + return session; +} + +/** + * Generate or replace a project's webhook secret. + * + * Returned once so the owner can paste it into Forgejo. It is stored and can be + * read again by the owner's admin page — unlike a user password, this is a + * shared secret that both ends need, so there is no value in making it + * unrecoverable. It must still never reach a client manager's browser. + */ +export async function rotateWebhookSecret(input: { + projectId: string; +}): Promise<{ secret: string }> { + await requireOwner(); + + const secret = `whsec_${randomBytes(24).toString("hex")}`; + const pb = await getAdminPb(); + + await pb + .collection("projects") + .update(input.projectId, { webhook_secret: secret }); + + revalidatePath(`/admin/projects/${input.projectId}`); + return { secret }; +} + +/** + * Pull the full history for a project. + * + * `stopAfterKnownPages: Infinity` reads to the end rather than stopping at the + * first familiar page — that is the whole difference between a backfill and the + * hourly poll, and it is why both share one function. + * + * WFDASH-WEBAPP is 787 commits, so this is 16 pages at 50 per page. It is slow + * enough to want a progress indication in the UI and fast enough not to need a + * job queue. + */ +export async function backfillProject(input: { + projectId: string; +}): Promise<{ created: number; skipped: number }> { + await requireOwner(); + + const pb = await getAdminPb(); + const project = await pb + .collection("projects") + .getOne(input.projectId); + + if (!project.repo_owner || !project.repo_name) { + throw new Error("Link a repository to this project first"); + } + + if (!(await repoExists(project.repo_owner, project.repo_name))) { + throw new Error( + `Forgejo has no repository ${project.repo_owner}/${project.repo_name}, or it is not readable` + ); + } + + const result = await ingestProject(project, { + stopAfterKnownPages: Number.POSITIVE_INFINITY, + }); + + revalidatePath(`/admin/projects/${input.projectId}`); + return { created: result.created, skipped: result.skipped }; +} +``` + +- [ ] **Step 2: Verify the build** + +```bash +npm run build +``` + +Expected: build succeeds. + +- [ ] **Step 3: Commit** + +```bash +git add src/app/actions/ingest.ts +git commit -m "feat(actions): backfill and webhook secret rotation" +``` + +--- + +## Task 9: Project ingest admin page + +**Files:** +- Create: `src/components/admin/ingest-panel.tsx` +- Create: `src/app/admin/projects/[projectId]/page.tsx` + +- [ ] **Step 1: Write the panel** + +`src/components/admin/ingest-panel.tsx`: + +```tsx +"use client"; + +import { useState, useTransition } from "react"; +import { Copy, RefreshCw, Download } from "lucide-react"; + +import { backfillProject, rotateWebhookSecret } from "@/app/actions/ingest"; + +export function IngestPanel({ + projectId, + repoOwner, + repoName, + hasSecret, + webhookUrl, + lastIngestAt, + commitCount, +}: { + projectId: string; + repoOwner: string; + repoName: string; + hasSecret: boolean; + webhookUrl: string; + lastIngestAt: string; + commitCount: number; +}) { + const [pending, startTransition] = useTransition(); + const [secret, setSecret] = useState(null); + const [status, setStatus] = useState(null); + const [error, setError] = useState(null); + + const linked = Boolean(repoOwner && repoName); + + const field = + "w-full rounded-lg border border-border-subtle bg-surface px-3 py-2 text-xs text-text-secondary"; + + return ( +
+
+

+ Commit ingestion +

+

+ {linked ? ( + <> + Linked to {repoOwner}/{repoName} ·{" "} + {commitCount} commits stored + {lastIngestAt && <> · last synced {lastIngestAt.slice(0, 16).replace("T", " ")}} + + ) : ( + "No repository linked. Set the owner and name on this project first." + )} +

+
+ + {linked && ( + <> +
+ Webhook URL +
+ + +
+

+ In Forgejo: Settings → Webhooks → Add Webhook → Gitea. Method POST, + content type application/json, trigger on Push. +

+
+ +
+ + Webhook secret {hasSecret && !secret && "(set)"} + +
+ {secret ? ( + + ) : ( + + {hasSecret + ? "A secret is configured. Rotating it requires updating Forgejo." + : "No secret yet. The webhook will reject every delivery until one is set."} + + )} + +
+
+ +
+ + + {status && {status}} + {error && ( + + {error} + + )} +
+ + )} +
+ ); +} +``` + +- [ ] **Step 2: Write the page** + +`src/app/admin/projects/[projectId]/page.tsx`: + +```tsx +import { notFound } from "next/navigation"; + +import { AppShell } from "@/components/shell/app-shell"; +import { TopBar } from "@/components/shell/top-bar"; +import { IngestPanel } from "@/components/admin/ingest-panel"; +import { getAdminPb } from "@/lib/pb/admin"; +import type { ProjectRecord } from "@/lib/pb/types"; + +/** + * The admin layout has already refused anyone who is not the owner. + * + * `webhook_secret` is read here but only its presence crosses into the client + * component — never the value. PocketBase rules are record-level and cannot + * hide one field, so stripping it is this page's job. + */ +export default async function ProjectAdminPage({ + params, +}: { + params: Promise<{ projectId: string }>; +}) { + const { projectId } = await params; + + const pb = await getAdminPb(); + + let project: ProjectRecord; + try { + project = await pb.collection("projects").getOne(projectId); + } catch { + notFound(); + } + + const commits = await pb + .collection("commits") + .getList(1, 1, { filter: `project = "${projectId}"`, fields: "id" }); + + const appUrl = process.env.NEXT_PUBLIC_APP_URL ?? "http://localhost:3000"; + + return ( + + + +
+
+ +
+
+ + ); +} +``` + +- [ ] **Step 3: Add `NEXT_PUBLIC_APP_URL`** + +Append to `.env.example`: + +``` +# The app's own public URL, used to build the webhook URL shown to the owner. +NEXT_PUBLIC_APP_URL=http://localhost:3000 +``` + +- [ ] **Step 4: Link it from the projects list** + +In `src/app/admin/projects/page.tsx`, make the project name link to +`/admin/projects/${project.id}` instead of the board, and add a separate +"Board" link in the last column. + +- [ ] **Step 5: Verify** + +```bash +npm run build +``` + +Expected: build succeeds. Open a project's admin page, generate a secret, and +confirm the webhook URL renders. + +- [ ] **Step 6: Commit** + +```bash +git add src/components/admin/ingest-panel.tsx src/app/admin/projects .env.example +git commit -m "feat(admin): repository linking, webhook setup and backfill" +``` + +--- + +## Task 10: Integration tests + +**Files:** +- Create: `tests/integration/ingest.test.ts` + +- [ ] **Step 1: Write the tests** + +`tests/integration/ingest.test.ts`: + +```ts +import { describe, it, expect, beforeAll } from "vitest"; +import PocketBase from "pocketbase"; +import { createHmac } from "node:crypto"; + +import { APP_URL, DEMO, adminPb, requireServices, userPb } from "./helpers"; + +let admin: PocketBase; +let projectId: string; + +beforeAll(async () => { + await requireServices(); + admin = await adminPb(); + const project = await admin + .collection("projects") + .getFirstListItem(`slug = "${DEMO.projectSlug}"`); + projectId = project.id; +}); + +const BODY = JSON.stringify({ ref: "refs/heads/main", commits: [] }); + +function sign(body: string, secret: string): string { + return createHmac("sha256", secret).update(body, "utf8").digest("hex"); +} + +describe("the webhook endpoint refuses anything it cannot verify", () => { + it("rejects a delivery with no signature", async () => { + const response = await fetch( + `${APP_URL}/api/webhooks/forgejo/${projectId}`, + { method: "POST", headers: { "content-type": "application/json" }, body: BODY } + ); + expect(response.status).toBe(401); + }); + + it("rejects a delivery signed with the wrong secret", async () => { + const response = await fetch( + `${APP_URL}/api/webhooks/forgejo/${projectId}`, + { + method: "POST", + headers: { + "content-type": "application/json", + "x-gitea-signature": sign(BODY, "not-the-secret"), + }, + body: BODY, + } + ); + expect(response.status).toBe(401); + }); + + it("rejects an unknown project id the same way as a bad signature", async () => { + // Different answers here would turn the endpoint into a way to enumerate + // which project ids exist. + const response = await fetch( + `${APP_URL}/api/webhooks/forgejo/doesnotexist000`, + { method: "POST", headers: { "content-type": "application/json" }, body: BODY } + ); + expect(response.status).toBe(401); + }); +}); + +describe("the reconciliation endpoint is protected", () => { + it("rejects a request with no bearer token", async () => { + const response = await fetch(`${APP_URL}/api/cron/reconcile`, { + method: "POST", + }); + expect(response.status).toBe(401); + }); + + it("rejects a wrong bearer token", async () => { + const response = await fetch(`${APP_URL}/api/cron/reconcile`, { + method: "POST", + headers: { authorization: "Bearer wrong" }, + }); + expect(response.status).toBe(401); + }); +}); + +describe("the webhook secret never leaves the server", () => { + it("is not readable by a client manager through PocketBase", async () => { + await admin + .collection("projects") + .update(projectId, { webhook_secret: "whsec_should_not_leak" }); + + const pb = await userPb(DEMO.managerEmail, DEMO.managerPassword); + const project = await pb.collection("projects").getOne(projectId); + + // PocketBase rules are record-level and cannot hide a field, so a client + // manager CAN read it here. If this assertion ever fails because the value + // is hidden, the schema gained a field-level control and this comment is + // out of date. Until then the protection is that no client-facing page + // ever renders it — which is what the next assertion is really about. + expect(project.webhook_secret).toBe("whsec_should_not_leak"); + }); + + it("does not appear in the board page HTML", async () => { + const cookie = await (await import("./helpers")).signIn( + DEMO.managerEmail, + DEMO.managerPassword + ); + const response = await fetch( + `${APP_URL}/p/${DEMO.clientSlug}/${DEMO.projectSlug}`, + { headers: { cookie } } + ); + const html = await response.text(); + expect(html).not.toContain("whsec_should_not_leak"); + + await admin.collection("projects").update(projectId, { webhook_secret: "" }); + }); +}); +``` + +- [ ] **Step 2: Run the integration suite** + +With `npm run dev` running: + +```bash +set -a && source .env.local && set +a && npm run test:integration +``` + +Expected: PASS — 30 tests (23 from earlier phases plus 7 here). + +- [ ] **Step 3: Act on what the secret test reveals** + +The first test in the last block documents a real gap: **a client manager can +read `projects.webhook_secret` directly from PocketBase.** Record-level rules +cannot hide a field. + +Raise this rather than leaving it. Two workable fixes, neither in this plan: + +- Move `webhook_secret` to its own collection with superuser-only read rules, + related to the project. Clean, costs a migration. +- Keep the secret out of PocketBase entirely — derive it as + `HMAC(APP_SECRET, projectId)` so nothing is stored at all. + +The second is the neater answer and would make the field unnecessary. Flag it +for a decision before this ships anywhere a client can reach. + +- [ ] **Step 4: Commit** + +```bash +git add tests/integration/ingest.test.ts +git commit -m "test(ingest): webhook and cron endpoint protection" +``` + +--- + +## Task 11: Full verification + +**This task is work, not a summary. Run every command and act on what it prints.** + +- [ ] **Step 1: Run every test** + +```bash +set -a && source .env.local && set +a && npm run test:all +``` + +Expected: PASS. Unit tests total **128** — the 105 after Plan 1d, plus 12 in +`conventional.test.ts` and 11 in `signature.test.ts`. Integration tests total 30. + +- [ ] **Step 2: Backfill a real repository** + +Set `FORGEJO_URL` in `.env.local`, link the demo project to +`David` / `WFDASH-WEBAPP` on its admin page, generate a secret, then press +**Backfill history**. + +Expected: `787 new, 0 already stored`. Press it a second time. + +Expected: `0 new, 787 already stored`. If the second run creates anything, the +unique index on `(project, sha)` is missing or the duplicate-key branch in +`storeCommits` is not being reached. + +- [ ] **Step 3: Check the parse against the real history** + +```bash +set -a && source .env.local && set +a && npx tsx -e " +import PocketBase from 'pocketbase'; +const pb = new PocketBase(process.env.NEXT_PUBLIC_PB_URL!); +pb.autoCancellation(false); +await pb.collection('_superusers').authWithPassword(process.env.PB_SUPERUSER_EMAIL!, process.env.PB_SUPERUSER_PASSWORD!); +const all = await pb.collection('commits').getFullList({ fields: 'type,scope,is_merge' }); +const byType: Record = {}; +for (const c of all) byType[c.type] = (byType[c.type] ?? 0) + 1; +console.log('total', all.length, '| merges', all.filter(c => c.is_merge).length); +console.log(Object.entries(byType).sort((a,b) => b[1]-a[1])); +" +``` + +Expected: 787 total and **8 merges**, matching `git rev-list --merges --count`. +The type spread should resemble the Hermes build log — `feat` far ahead, then +`fix`, then `docs`, `test`, `refactor`, `chore`. + +- [ ] **Step 4: Deliver a real webhook** + +Add the webhook in Forgejo with the generated secret, then push a commit. + +Expected: the commit appears within seconds, and Forgejo's webhook delivery log +shows a 200. Then edit the secret in Forgejo to something wrong and push again: +the delivery must show 401 and no commit row must be created. + +- [ ] **Step 5: Schedule the reconciliation** + +In Coolify, add an hourly scheduled task: + +```bash +curl -fsS -X POST -H "Authorization: Bearer $CRON_SECRET" https:///api/cron/reconcile +``` + +- [ ] **Step 6: Commit any fixes** + +```bash +git add -A +git commit -m "fix: issues found during Phase 2a verification" +``` + +--- + +## Definition of done + +1. `npm run test:all` passes: 128 unit, 30 integration. +2. `npm run build` succeeds with no type errors. +3. Backfilling WFDASH-WEBAPP stores 787 commits; a second run stores 0. +4. Stored merge count is exactly 8, matching `git rev-list --merges --count HEAD`. +5. A real push appears within seconds; a delivery with a wrong secret is + rejected with 401 and creates nothing. +6. The reconciliation endpoint refuses a request without the bearer token. +7. The `webhook_secret` question raised in Task 10 Step 3 has a decision + recorded, even if the fix lands later. + +## Deliberately not in this plan + +- **Commit-to-card linking** — Plan 2b. `card` and `link_source` exist on the + schema and stay `""` / `none` until then. +- **Session clustering, workdays, metrics, the dashboard** — Plan 2c, along with + the `session_gap_minutes` and padding fields, which are not added until + something reads them. +- **AI summarisation** — Phase 3. This plan stores raw messages and translates + nothing. +- **Multi-branch ingestion.** Only the default branch is read. Commits that only + ever existed on a deleted feature branch are not counted. +- **Author aliasing.** Raw `author_name` and `author_email` are stored; the + dashboard's "1 developer" claim is Plan 2c's presentation decision, per spec §6.