David/MEGAINFRA-WEBAPP
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(dn-batches): one batch per tick by default

Browse source 
A tick is fully sequential: one vision call per page, then each note's
checks and promotion in turn. Claiming three batches of a 13-page scan
can approach the route's 300s ceiling, and a tick that times out leaves
its batch to be reclaimed and re-read, paying for those pages twice.

One batch per tick also makes n8n's execution log directly readable —
one run, one document, one outcome. DN_TICK_BATCH_SIZE still overrides
it once the page-count distribution is known.

34 batches at a 1-minute schedule is ~34 minutes rather than ~12.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
DEls 2026-09-17 22:54:32 +02:00
parent d4ac4bd6e4
commit d02feee57f
4 changed files with 21 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.env.local.example
View file

@ -48,7 +48,7 @@ MEGA_CUSTOMER_VAT_NO=4280307796
# How many dn_batches rows the /api/dn-batches/tick route claims per
# cron hit. Tune for your batch size / model latency.
DN_TICK_BATCH_SIZE=3
DN_TICK_BATCH_SIZE=1
# Public URL of the n8n instance (used by your local Codex MCP config).
N8N_MCP_URL=https://wfdash.n8n.apphub.co.za/mcp-server/http

2
docs/superpowers/specs/2026-09-17-delivery-note-inbox-design.md
View file

@ -376,7 +376,7 @@ and the Coolify environment:
| Variable | Value | Used by |
| --- | --- | --- |
| `MEGA_CUSTOMER_VAT_NO` | `4280307796` | `checks.ts` → `customer_identity` |
| `DN_TICK_BATCH_SIZE` | `3` | `/api/dn-batches/tick` claim limit |
| `DN_TICK_BATCH_SIZE` | `1` | `/api/dn-batches/tick` claim limit. One batch per tick: a tick is fully sequential, so claiming several risks the 300s ceiling on a long document and pays for re-read pages. |
`MINIMAX_API_KEY`, `MINIMAX_BASE_URL` and `MINIMAX_MODEL` already exist and
are reused by `extract.ts`.

9
src/app/api/dn-batches/tick/route.test.ts
View file

@ -320,7 +320,10 @@ describe("POST /api/dn-batches/tick — happy path", () => {
expect(body.results.map((r: { id: string }) => r.id)).toEqual(["a1", "a2"]);
});
it("defaults DN_TICK_BATCH_SIZE to 3 when env is unset", async () => {
// One batch per tick by default. A tick is fully sequential — one
// vision call per page — so claiming several risks the route's 300s
// ceiling on a long document, and a timeout re-reads those pages.
it("defaults DN_TICK_BATCH_SIZE to 1 when env is unset", async () => {
vi.stubEnv("DN_TICK_BATCH_SIZE", "");
dnBatchesGetFullList.mockImplementation(
async (opts: { filter?: string } = {}) => {
@ -338,8 +341,8 @@ describe("POST /api/dn-batches/tick — happy path", () => {
const res = await POST(postWithValidAuth());
const body = await res.json();
expect(body.claimed).toBe(3);
expect(runBatchMock).toHaveBeenCalledTimes(3);
expect(body.claimed).toBe(1);
expect(runBatchMock).toHaveBeenCalledTimes(1);
});
it("does nothing when there are no inbox candidates", async () => {

14
src/app/api/dn-batches/tick/route.ts
View file

@ -28,7 +28,19 @@ export async function POST(request: Request) {
}
const pb = await pbAdmin();
const limit = Number(process.env.DN_TICK_BATCH_SIZE || 3);
// One batch per tick by default.
//
// Everything inside a tick is sequential — one vision call per page,
// then each note's checks and promotion in turn — so a tick's runtime
// scales with the pages it claims. Three batches of a 13-page scan can
// approach the 300s ceiling above, and a tick that times out mid-way
// leaves its batch to be reclaimed and re-read, paying for those pages
// twice.
//
// One batch per tick also makes n8n's execution log directly readable:
// one run, one document, one outcome. Raise DN_TICK_BATCH_SIZE once
// the page-count distribution is known and the ceiling is not a risk.
const limit = Number(process.env.DN_TICK_BATCH_SIZE || 1);
const results: Record<string, unknown>[] = [];
await reclaimStale(pb);