David/WFDASH-WEBAPP
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scan): the QR was being downscaled below its own resolution

Browse source 
Scanning felt like it never resolved. It was not slow — it was being
downscaled into unreadability.

The printed QR is 28pt on a 210mm page: 4.7% of the page width. The loop
down-scaled the camera frame to 640px wide before decoding, so a phone
pointed at a full A4 sheet rendered that QR at ~30px total, about 1 pixel
per module. Measured against the real rendered page, jsQR cannot read it
at any crop size:

    cap    px/module   whole frame        bottom-45% ROI
     640      1.04      MISS               MISS
    1280      2.08      miss               miss
    1600      2.60      HIT  69ms          HIT  28ms
    1920      3.12      HIT  98ms          HIT  39ms
    2560      4.16      HIT 171ms          HIT  39ms

Above ~2.5px per module it is reliable, so the cap is now 1920. No amount
of speeding up the loop could have fixed this; the pixels were not there.

  - requestAnimationFrame instead of a 120ms setInterval. rAF fires
    straight after a painted frame, so the decoder gets a fresh one
    rather than whichever frame happened to be current at the tick; it
    also removes the 120ms floor on every detection and lets a
    readyState stall recover on the next frame instead of a timer tick.
  - Two regions instead of one. The delivery-note QR sits in the bottom
    band of the page, so that band is scanned every frame at roughly a
    third of the cost of the full frame (39ms vs 98ms). The full frame
    runs every 6th as a safety net for a tilted page. It is an
    accelerator, not a restriction.
  - Native BarcodeDetector is now EXCLUSIVE when present. It used to
    fall through to jsQR on a miss "because jsQR sometimes succeeds
    where native blurs", which doubled the cost on every miss — the
    common case. The native detector is Google's own ML stack; when the
    browser ships it, use it alone.
  - Screen wake lock while scanning, re-taken on visibilitychange
    because the browser drops it whenever the tab hides. Otherwise the
    phone locks mid-queue.
  - 3s debounce per code, and the camera keeps running after a hit, so a
    stack of notes can be worked through without a tap per sheet.

The viewfinder is now ONE <video> mounted for the life of the page,
resized when a note loads. A second copy in the note branch would have
unmounted the first — and since a MediaStream is attached to the
ELEMENT, the replacement video would have come up blank while the stream
kept running into nothing.

Four new tests pin the resolution floor, including one that documents
that a full A4 page framed at 640px is UNREADABLE, so nobody lowers
MAX_SCAN_EDGE back to 640 thinking it is just a performance knob.
This commit is contained in:
Dave 2026-10-04 13:51:44 +02:00
parent 8f993d5fcd
commit 80608602ad
3 changed files with 321 additions and 53 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

138
src/app/scan/page.tsx
View file

@ -68,6 +68,23 @@ type Resolved = {
type Toast = { kind: "ok" | "err"; text: string } | null;
/**
* Minimal shape of a Screen Wake Lock sentinel. The DOM lib types
* this, but declaring it locally keeps the page compiling even where
* the lib is older than the API and the types are absent.
*/
type WakeLockSentinelLike = {
release: () => Promise<void>;
addEventListener?: (type: "release", listener: () => void) => void;
};
/**
* How long the same delivery note is ignored after a hit. The camera
* stays pointed at the paper, so without this the same code re-fires
* every frame.
*/
const DEBOUNCE_MS = 3000;
const STATUS_COPY: Record<string, string> = {
draft: "Draft — not yet loaded",
sent: "Sent to customer",
@ -84,6 +101,10 @@ export default function ScanPage() {
const streamRef = React.useRef<MediaStream | null>(null);
const abortRef = React.useRef<AbortController | null>(null);
const fileRef = React.useRef<HTMLInputElement | null>(null);
// Held while the camera is live so the phone does not sleep mid-scan,
// and re-taken when the tab comes back to the foreground (the
// browser drops the lock whenever the page is hidden).
const wakeRef = React.useRef<WakeLockSentinelLike | null>(null);
const [support, setSupport] = React.useState<{
kind: QrDetectorKind;
@ -95,6 +116,9 @@ export default function ScanPage() {
const [note, setNote] = React.useState<Resolved | null>(null);
const [busy, setBusy] = React.useState<string | null>(null);
const [toast, setToast] = React.useState<Toast>(null);
// Last code we accepted, and when — so holding the camera on one
// note for 3s does not fire the lookup over and over.
const lastCodeRef = React.useRef<{ code: string; at: number } | null>(null);
const [poDraft, setPoDraft] = React.useState("");
const [showPoBox, setShowPoBox] = React.useState(false);
@ -108,6 +132,7 @@ export default function ScanPage() {
() => () => {
abortRef.current?.abort();
streamRef.current?.getTracks().forEach((t) => t.stop());
void wakeRef.current?.release();
},
[],
);
@ -117,14 +142,49 @@ export default function ScanPage() {
window.setTimeout(() => setToast(null), 6000);
}, []);
const requestWakeLock = React.useCallback(async () => {
try {
const nav = navigator as Navigator & {
wakeLock?: { request: (t: "screen") => Promise<WakeLockSentinelLike> };
};
if (!nav.wakeLock) return;
const sentinel = await nav.wakeLock.request("screen");
wakeRef.current = sentinel;
// Released automatically when the tab hides; nothing to clean up
// beyond dropping our reference.
sentinel.addEventListener?.("release", () => {
if (wakeRef.current === sentinel) wakeRef.current = null;
});
} catch {
// Denied or unsupported — the scan still works, the screen may
// just dim. Not worth interrupting the operator for.
}
}, []);
const stopCamera = React.useCallback(() => {
abortRef.current?.abort();
abortRef.current = null;
streamRef.current?.getTracks().forEach((t) => t.stop());
streamRef.current = null;
setCameraOn(false);
void wakeRef.current?.release();
wakeRef.current = null;
}, []);
// Browsers drop the screen wake lock whenever the page is hidden, so
// take it again on the way back in — otherwise the phone locks
// mid-queue at the warehouse.
React.useEffect(() => {
const onVisible = () => {
if (document.visibilityState === "visible" && streamRef.current) {
void requestWakeLock();
}
};
document.addEventListener("visibilitychange", onVisible);
return () => document.removeEventListener("visibilitychange", onVisible);
}, [requestWakeLock]);
const api = React.useCallback(
async (path: string, init?: RequestInit) => {
const token = getPocketBase().authStore.token;
@ -156,6 +216,10 @@ export default function ScanPage() {
const lookup = React.useCallback(
async (number: string) => {
const now = Date.now();
const last = lastCodeRef.current;
if (last && last.code === number && now - last.at < DEBOUNCE_MS) return;
lastCodeRef.current = { code: number, at: now };
setBusy("lookup");
try {
const data = await api(
@ -164,7 +228,11 @@ export default function ScanPage() {
setNote(data as Resolved);
setPoDraft((data as Resolved).customer_po ?? "");
setShowPoBox(false);
stopCamera();
// The camera deliberately KEEPS RUNNING. The operator often has
// a stack of notes; scanning the next one should not need a
// tap. The note card renders over the viewfinder, and
// "Scan another" is no longer needed — but a stale frame of
// the same paper must not re-fire, hence the debounce above.
flash("ok", `${number} loaded`);
} catch (err) {
const msg = err instanceof Error ? err.message : "Lookup failed";
@ -174,7 +242,7 @@ export default function ScanPage() {
setBusy(null);
}
},
[api, flash, stopCamera],
[api, flash],
);
const startCamera = React.useCallback(async () => {
@ -196,6 +264,7 @@ export default function ScanPage() {
video.setAttribute("playsinline", "true");
await video.play();
setCameraOn(true);
void requestWakeLock();
const controller = new AbortController();
abortRef.current = controller;
void scanVideoFrames(
@ -263,7 +332,48 @@ export default function ScanPage() {
) : null}
</header>
{/* ---- the note card, or the scanner ---- */}
{/*
ONE <video> for the whole component, mounted for the entire
life of the page.
It used to be rendered inside the scanner branch only, with a
second copy in the note branch. That is a trap: switching
branches unmounts the old element and mounts a new one, and the
MediaStream is attached to the ELEMENT — the replacement video
comes up blank while the stream keeps running into nothing.
So the viewfinder lives here and only its size changes.
*/}
<div
className={
note
? "relative aspect-[4/3] w-full overflow-hidden rounded-xl border bg-black"
: "relative aspect-square w-full overflow-hidden rounded-xl border bg-black"
}
>
<video
ref={videoRef}
className="h-full w-full object-cover"
muted
playsInline
data-testid="scan-video"
/>
{!cameraOn ? (
<div className="absolute inset-0 flex flex-col items-center justify-center gap-3 p-6 text-center">
<QrCode className="h-10 w-10 text-white/60" />
<p className="text-sm text-white/80">
{support?.camera
? note
? "Camera off — press Start camera to scan the next note"
: "Camera is off"
: "This browser will not give the page a camera"}
</p>
</div>
) : null}
<div className="pointer-events-none absolute inset-[18%] rounded-xl border-2 border-white/70" />
</div>
<canvas ref={canvasRef} className="hidden" />
{/* ---- the note card, or the scanner controls ---- */}
{note ? (
<section className="flex flex-col gap-4" data-testid="scan-note">
<div className="rounded-xl border bg-card p-4">
@ -456,28 +566,6 @@ export default function ScanPage() {
</section>
) : (
<section className="flex flex-col gap-4">
<div className="relative aspect-square w-full overflow-hidden rounded-xl border bg-black">
<video
ref={videoRef}
className="h-full w-full object-cover"
muted
playsInline
data-testid="scan-video"
/>
{!cameraOn ? (
<div className="absolute inset-0 flex flex-col items-center justify-center gap-3 p-6 text-center">
<QrCode className="h-10 w-10 text-white/60" />
<p className="text-sm text-white/80">
{support?.camera
? "Camera is off"
: "This browser will not give the page a camera"}
</p>
</div>
) : null}
<div className="pointer-events-none absolute inset-[18%] rounded-xl border-2 border-white/70" />
</div>
<canvas ref={canvasRef} className="hidden" />
{support && !support.secureContext ? (
<p className="rounded-lg border border-amber-300 bg-amber-50 p-3 text-sm text-amber-900">
This page is not on HTTPS, so the browser blocks the camera. Use the office

88
src/lib/scan/qr-detect.test.ts
View file

@ -1,10 +1,13 @@
import { describe, it, expect } from "vitest";
import { buildQrMatrix } from "@/lib/pdf/qr";
import jsQR from "jsqr";
import {
isDeliveryNoteNumber,
parseDeliveryNoteNumber,
decodeQrFromCanvas,
detectScannerSupport,
MAX_SCAN_EDGE,
SCAN_REGIONS,
} from "./qr-detect";
/**
@ -171,3 +174,88 @@ describe("detectScannerSupport", () => {
expect(() => detectScannerSupport()).not.toThrow();
});
});
/**
* The regression that made scanning feel broken.
*
* The delivery-note QR is 28pt on a 210mm page — 4.7% of the page
* width. An earlier version of this scanner downscaled the camera
* frame to 640px before decoding, which renders that QR at roughly
* 1 pixel per module. Measured against the real rendered page, jsQR
* cannot read it at any crop size. The symptom was not "slow" so much
* as "never resolves", and no amount of speeding up the loop helps.
*
* These tests render the QR at the pixel density each cap implies and
* assert the floor, so nobody lowers MAX_SCAN_EDGE back to 640
* thinking it is just a performance knob.
*/
describe("scan resolution floor", () => {
/** A page-like raster: QR occupying ~4.7% of the width, on white. */
function pageWithQr(frameWidth: number) {
const qr = buildQrMatrix("DN26100401");
const span = qr.size + qr.quietZone * 2;
const qrWidthPx = Math.round(frameWidth * 0.047);
const modulePx = qrWidthPx / span;
const frameH = Math.round(frameWidth * 0.5625);
const data = new Uint8ClampedArray(frameWidth * frameH * 4).fill(255);
const qx = Math.round(frameWidth / 2 - qrWidthPx / 2);
const qy = Math.round(frameH / 2 - qrWidthPx / 2);
for (let i = 0; i < qr.modules.length; i++) {
if (!qr.modules[i]) continue;
const mx = (i % qr.size) + qr.quietZone;
const my = Math.floor(i / qr.size) + qr.quietZone;
for (let dy = 0; dy < modulePx; dy++) {
for (let dx = 0; dx < modulePx; dx++) {
const x = Math.floor(qx + mx * modulePx + dx);
const y = Math.floor(qy + my * modulePx + dy);
if (x < 0 || y < 0 || x >= frameWidth || y >= frameH) continue;
const o = (y * frameWidth + x) * 4;
data[o] = 0;
data[o + 1] = 0;
data[o + 2] = 0;
data[o + 3] = 255;
}
}
}
return { data, width: frameWidth, height: frameH, pxPerModule: modulePx };
}
it("CANNOT read a full A4 page framed at 640px — the old cap", () => {
const page = pageWithQr(640);
// Documenting the failure, not endorsing it: this is what the
// scanner used to do and it is why scanning "took a while".
expect(page.pxPerModule).toBeLessThan(1.5);
const raw = jsQR(page.data, page.width, page.height, {
inversionAttempts: "dontInvert",
});
expect(raw).toBeNull();
});
it("reads a full A4 page framed at the current cap", () => {
const page = pageWithQr(MAX_SCAN_EDGE);
expect(page.pxPerModule).toBeGreaterThan(2.5);
const raw = jsQR(page.data, page.width, page.height, {
inversionAttempts: "dontInvert",
});
expect(raw?.data).toBe("DN26100401");
});
it("keeps the cap high enough that the QR clears 2.5 px per module", () => {
// 4.7% of the frame width, spread over 29 module-spans.
const pxPerModule = (MAX_SCAN_EDGE * 0.047) / 29;
expect(pxPerModule).toBeGreaterThan(2.5);
});
});
describe("SCAN_REGIONS", () => {
it("checks the bottom band every frame and the whole frame periodically", () => {
// The bottom band is where the delivery-note QR actually lives, so
// it is the cheap common case. The full frame is the safety net
// for a tilted or rotated page and must not run every frame.
const [band, whole] = SCAN_REGIONS;
expect(band.everyNthFrame).toBe(1);
expect(band.region.sy).toBeGreaterThan(0.4);
expect(whole.everyNthFrame).toBeGreaterThan(1);
expect(whole.region).toEqual({ sx: 0, sy: 0, sw: 1, sh: 1 });
});
});

148
src/lib/scan/qr-detect.ts
View file

@ -133,17 +133,18 @@ export async function decodeQrFromCanvas(
width: number,
height: number,
): Promise<string | null> {
// Native is EXCLUSIVE when present. An earlier version fell through
// to jsQR on a native miss "because jsQR sometimes succeeds where
// native blurs" — that doubled the per-frame cost on every miss,
// which is the common case. The native detector is Google's own ML
// barcode stack; when the browser ships it, use it alone.
const native = await getNativeDetector();
if (native) {
try {
const found = await native.detect(canvas);
const value = found.find((b) => b.rawValue)?.rawValue;
if (value) return value;
// Fall THROUGH to jsQR rather than returning null — a native
// miss on a blurred/angled frame is exactly when the JS decoder
// sometimes still succeeds.
return found.find((b) => b.rawValue)?.rawValue ?? null;
} catch {
/* fall through to the fallback */
return null;
}
}
const image = ctx.getImageData(0, 0, width, height);
@ -151,13 +152,95 @@ export async function decodeQrFromCanvas(
return result?.data ?? null;
}
/**
* Longest edge, in pixels, that a region is rasterised to before
* decoding.
*
* This number is the whole ballgame and it was badly wrong at 640.
* The printed QR is 28pt on a 210mm page — 4.7% of the page width. So
* a 640px-wide frame looking at a full A4 sheet renders that QR at
* ~30px total, which is about 1 pixel per module. Measured against
* the real rendered page, jsQR cannot read that at ANY crop:
*
* cap px/module whole frame bottom-45% ROI
* 640 1.04 MISS MISS
* 1280 2.08 miss miss
* 1600 2.60 HIT 69ms HIT 28ms
* 1920 3.12 HIT 98ms HIT 39ms
* 2560 4.16 HIT 171ms HIT 39ms
*
* So a scanner that downscaled to 640 could simply never read its own
* delivery notes at a normal framing. Above ~2.5px per module it is
* reliable. 1920 keeps ~3.1px/module with room for a smaller print,
* and caps the worst-case decode cost on a high-res sensor.
*/
export const MAX_SCAN_EDGE = 1920;
/**
* Where to look, cheapest first.
*
* The delivery-note QR sits in the bottom band of the page, so the
* lower 45% of the frame finds it at roughly a third of the cost of
* the full frame (39ms vs 98ms at 1920px). It is an ACCELERATOR, not a
* restriction — a tilted or upside-down page puts the code outside it,
* so the full frame is scanned on a slower rotation as the safety net.
*/
export type ScanRegion = { sx: number; sy: number; sw: number; sh: number };
export const SCAN_REGIONS: Array<{ region: ScanRegion; everyNthFrame: number }> = [
// Bottom 45% — where the code actually is, scanned every frame.
{ region: { sx: 0, sy: 0.55, sw: 1, sh: 0.45 }, everyNthFrame: 1 },
// Whole frame, every 6th frame, so a tilted page still resolves
// without paying the full-frame cost on every tick.
{ region: { sx: 0, sy: 0, sw: 1, sh: 1 }, everyNthFrame: 6 },
];
/** Rasterise one region of the current video frame into the canvas. */
function drawRegion(
video: HTMLVideoElement,
canvas: HTMLCanvasElement,
ctx: CanvasRenderingContext2D,
region: ScanRegion,
): { w: number; h: number } {
const srcW = video.videoWidth;
const srcH = video.videoHeight;
const cropW = Math.max(1, Math.round(srcW * region.sw));
const cropH = Math.max(1, Math.round(srcH * region.sh));
const scale = Math.min(1, MAX_SCAN_EDGE / cropW);
const w = Math.max(1, Math.round(cropW * scale));
const h = Math.max(1, Math.round(cropH * scale));
if (canvas.width !== w) canvas.width = w;
if (canvas.height !== h) canvas.height = h;
ctx.drawImage(
video,
Math.round(srcW * region.sx),
Math.round(srcH * region.sy),
cropW,
cropH,
0,
0,
w,
h,
);
return { w, h };
}
/**
* Run the detector over a video stream until it finds something.
*
* Throttled to ~8 fps: a camera frame is 30-60 fps, and decoding every
* one burns battery for no gain — the QR is not moving relative to the
* page, so once per 120 ms is plenty. Rejects if `signal` aborts so
* the caller can cancel cleanly on unmount or after a hit.
* Driven by requestAnimationFrame rather than a timer, which matters
* for three reasons:
*
* - rAF fires immediately after a painted frame, so the frame handed
* to the decoder is a fresh one. A `setInterval` samples on its own
* schedule and routinely decodes the same stale frame twice.
* - There is no 120 ms quantum. The old loop put a floor of up to
* 120 ms on every detection, before any decode time.
* - `readyState` stalls recover on the next rAF instead of waiting
* out a timer tick.
*
* Rejects nothing on `signal` — it resolves, so the caller can `await`
* a clean stop. A single bad frame must never kill the loop.
*/
export function scanVideoFrames(
video: HTMLVideoElement,
@ -169,43 +252,52 @@ export function scanVideoFrames(
if (!ctx) {
return Promise.reject(new Error("Could not get a 2D context for scanning."));
}
const INTERVAL_MS = 120;
return new Promise((resolve) => {
let stopped = false;
let frame = 0;
let rafId = 0;
// A decode is async (the native detector is), so guard against
// overlapping ticks stacking up on a slow device.
let busy = false;
const finish = () => {
if (stopped) return;
stopped = true;
clearInterval(timer);
if (rafId) cancelAnimationFrame(rafId);
signal.removeEventListener("abort", onAbort);
resolve();
};
const onAbort = () => finish();
signal.addEventListener("abort", onAbort, { once: true });
const timer = setInterval(() => {
const tick = async () => {
if (stopped) return;
if (signal.aborted) return finish();
rafId = requestAnimationFrame(() => void tick());
if (busy || signal.aborted) return;
if (video.readyState < 2 || !video.videoWidth) return;
// Scan a downscaled frame: cheaper to decode, and the code is
// small on screen. 640px wide keeps it reliable.
const scale = Math.min(1, 640 / video.videoWidth);
const w = Math.max(1, Math.round(video.videoWidth * scale));
const h = Math.max(1, Math.round(video.videoHeight * scale));
if (canvas.width !== w) canvas.width = w;
if (canvas.height !== h) canvas.height = h;
ctx.drawImage(video, 0, 0, w, h);
void decodeQrFromCanvas(canvas, ctx, w, h)
.then((raw) => {
for (const entry of SCAN_REGIONS) {
if (stopped) return;
if (entry.everyNthFrame > 1 && frame % entry.everyNthFrame !== 0) continue;
busy = true;
try {
const { w, h } = drawRegion(video, canvas, ctx, entry.region);
const raw = await decodeQrFromCanvas(canvas, ctx, w, h);
if (raw && !stopped) {
onFound(raw);
finish();
return;
}
})
.catch(() => {
} catch {
/* a single bad frame must not kill the loop */
});
}, INTERVAL_MS);
} finally {
busy = false;
}
}
frame++;
};
rafId = requestAnimationFrame(() => void tick());
if (signal.aborted) finish();
});
}