start_drain_thread and start_session_channel_reaper checked is_alive()
and then created + started the daemon thread without holding a lock, a
check-then-act race: two concurrent callers could both see "not alive"
and each spawn a thread. The loser's thread was never stored in the
module global and ran forever, un-joinable by the matching stop_*. Both
check-then-start sequences now run under a dedicated
_THREAD_LIFECYCLE_LOCK (kept separate from the purpose-bound
SESSION_CHANNELS_LOCK / _EMIT_COALESCE_LOCK), so exactly one thread is
ever created.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
BG_TASK_COMPLETE_EVENTS_SEEN gained a session_id -> set[process_id] entry the
first time a bg task completed for a session and was never deleted anywhere, so
it grew unbounded for the server lifetime.
The entry is created in _process_one for EVERY completion, whether or not any
SSE channel/tab exists, so pruning it only when the SessionChannel is reaped
would miss the dominant headless case (task fires, tab closed or never opened —
no channel to collect). Instead the reaper now sweeps the map by DELIVERY: once
a completion is drained (its session_id removed from PENDING_BG_TASK_COMPLETIONS)
the short _move_to_finished dedup window is closed and the entry is swept, every
tick, regardless of any channel. The registry's per-process_id
_completion_consumed gate remains the primary idempotency backstop, so sweeping a
delivered session's set can never resurrect an already-delivered completion.
Session deletion also prunes the entry (new forget_bg_task_completion_dedup),
covering a session deleted while a completion is still pending (which the
delivery-gated sweep deliberately retains).
Refs #4633
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- test_api_timeout: assert the updates/check call keeps timeoutMs:60000 in its
new _checkBody form (functionally identical; still the 60s override)
- panels.js: reword two // comments to drop apostrophes — the smooth-text-fade
brace-matcher test treats ' in a comment as a string delimiter and lost
loadSettingsPanel's closing brace (node --check confirms the JS was always valid)
The session-listing path already opens the live agent state.db read-only
(file:...?mode=ro) so a write-capable handle doesn't add checkpoint/lock
surface while the agent streams into the same WAL DB (#5455). Three pure-read
projections were missed and still opened a read-WRITE connection:
- read_session_lineage_report
- read_session_lineage_metadata
- the gateway-watcher fingerprint projection (a 5s poll)
Route all three through a shared open_state_db_readonly() helper that mirrors
the listing path: read-only file: URI with a writable fallback that warns. The
self-heal write path (missing idx_messages_session) is intentionally left
writable. No behavior change beyond open mode; reads perform zero writes.
Refs #5455
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
text-overflow:ellipsis is a no-op on the inline-flex badge; display:block in the
<=768px rule makes the ellipsis actually render for a long experimental describe.
MUST-FIX: /api/updates/check now accepts an explicit 'channel' in the POST body;
the Settings dropdown passes the just-picked value to checkUpdatesNow(channel) so
the immediate re-check can't race the debounced autosave PUT and answer for the
previous channel (verified live: body channel wins over stale saved setting).
SHOULD-FIX:
- copy: 'soaked' -> 'soak-tested'; append 'Applies to WebUI updates only.' to the
channel helper text (all 15 locales, native translations)
- mobile: .settings-version-badge gets max-width/overflow/ellipsis in the <=768px
query so the '· Experimental' suffix can't overflow a 320px viewport
(#5 badge-suffix i18n key deferred — non-blocking fast-follow per Fable, matches
the existing English-only version badge on master.)
Codex round-2 gate: apply_clear_lock re-entered _apply_update_inner(target)
without the channel, so an experimental WebUI lock-recovery retry silently fell
back to stable. Now passes _read_update_channel() through. + regression test.
Codex gate CORE fix: check_for_updates passed the user's WebUI channel into the
Agent repo resolution, so on 'experimental' the Agent ignored its v* tags and
fell back to origin/master. The Agent is a separate project (plain v* tags,
tracks master past tags) and must ALWAYS use DEFAULT_UPDATE_CHANNEL. Forced in
all 3 sites: check_for_updates agent leg + _apply_update_inner + apply_force_update
(target=='agent' -> channel=DEFAULT_UPDATE_CHANNEL). Added a real-git regression
proving Agent release/apply resolution is identical under both WebUI channels.
- test_update_stash_recovery: add merge-base handler for the force-update
rewind guard probe (origin/master is a descendant of HEAD -> not a rewind)
- test_security_review_fixes: fake_check accepts channel kwarg
- i18n.js: native translations of the 4 update-channel keys for all 14
non-English locales (locale-coverage tests enforce key presence)
Add an update_channel setting (stable|experimental) that selects which git-tag
stream the self-updater tracks on the single linear master line:
stable -> 'v*' promoted, soaked releases (default; unchanged glob)
experimental -> 'exp-v*' every release batch (opt-in testers)
A channel is only a tag glob — no branches, no divergence — so every ff-only
guarantee (#2653/#2846/#3140) is preserved. Channel governs the WebUI repo only;
the Agent repo keeps its historical branch fall-through.
Correctness (advisor-reviewed, Codex + Fable):
- describe uses --match <glob> so a commit tagged both v* and exp-v* resolves
to the channel-correct tag
- stable never falls through to origin/master when HEAD contains the latest
stable tag (the firehose-suppression that makes channels work)
- apply_force_update refuses to reset --hard onto an ancestor ref (rewind guard)
- update cache + in-progress guard keyed by (channel, include_agent)
- channel display badge is a SEPARATE field; WEBUI_VERSION stays channel-neutral
(asset cache-busting / SW cache / stale-client skew all do exact-string equality)
Settings UX: channel dropdown with risk-forward copy + the ff-only asymmetry
helper ('switching back to Stable keeps your current version until Stable catches
up'), channel chip on the version badge, i18n keys, re-check on switch.
Tests: new tests/test_update_channels.py (12 real-git-fixture tests) + existing
update suite updated for the channel-aware signatures.
When transcript virtualization is disabled (the #4325 opt-out,
_virtualizeTranscript===false), renderMessages() renders every row with no
windowing and never runs the virtualized measure pass
(_updateMessageVirtualMeasurements early-returns when !virtualized). Under
@media (pointer: coarse), .msg-row[data-role="user"] carries
content-visibility: auto; contain-intrinsic-size: auto 96px. Every rebuild does
inner.innerHTML='' then recreates rows as fresh elements, so a fresh off-screen
tall user row (a long paste measuring thousands of px) reserves only the flat
estimate instead of its real height. scrollHeight shrinks by (realHeight -
estimate), the browser force-clamps scrollTop, and the viewport jumps backward
(a browser clamp, JS=none, so scrollTop-write compensation cannot catch it).
#5638 fixed this for the virtualized wipe-and-rebuild path but left the
non-virtualized full-rebuild path uncovered.
Fix, three coordinated parts:
- _estimateUserRowIntrinsicHeight weights CJK / full-width characters as ~2
columns (they wrap at ~24 chars/line, not 48), so a CJK paste reserves close
to its real height even before it is ever measured.
- _applyUserRowIntrinsicHeight reserves max(remembered, estimate): a remembered
height can be a partial paint (a row taller than the viewport only paints its
intersecting slice under content-visibility:auto), so the estimate floors it.
- _rememberRenderedUserRowIntrinsicHeights, called pre-wipe inside
renderMessages, reads the still-laid-out rows' real heights and persists them
keyed by session-relative index, only for rows within the viewport (a fully
off-screen never-painted row reports its collapsed reserve and must not poison
the map), floored at the estimate.
Desktop rests at content-visibility:visible so intrinsic-size is inert there;
verified no behavior change with pointer:fine.
Adds tests/test_issue5744_nonvirtual_userrow_collapse_jumpback.py (7
mutation-checked node-harness tests). Existing #5637/#5638 suites and the
render/virtualization suites pass.
When the anchor row is gone and a virtual top-spacer is present but the snapshot
carries no captured topPadBefore (null), Number(null) is 0, so the topPad-delta
branch would add the ENTIRE current spacer height to scrollTop and fling the reader
far from their content. Guard on an actual (non-null) captured topPadBefore; else
keep the raw fallback target. Mutation-checked (drop the null guard -> the reader is
flung 1300px and the test fails).
Round-2 gate-cert proved the round-1 'snapshot.top + delta' + nearest-to-current
arbiter is mathematically unwinnable: a row's rect offset is scroll-relative
(rect.top - containerRect.top = rowContentPos - scrollTop), so a staged absolute
target only equals the true content-hold when scrollTop == snapshot.top — and in
that exact case the arbiter measures |raw - cur| = 0 and picks raw, writing the
stale value (the reported up-jump, unfixed).
Rework to the app's own realign idiom (mirrors _restoreMessageViewportAnchor and
_compensateScrollForMeasurementDelta): when a measurable anchor row exists, write
el.scrollTop += (currentOffset - capturedOffset) from the LIVE scrollTop — a no-op
when already aligned, a heal when not, no arbiter. Restore the per-tier lookup
guard (key -> sessionIdx, never the rawIdx degradation that maps to a different
message after a virtualization re-window). Genuinely-gone anchor: mirror the
topPad-delta idiom the snapshot already carries (topPadBefore), else keep raw.
Gate-cert tests now use scroll-dependent rect mocks (rect.top = rowContentPos -
scrollTop) so certified hold numbers are physically realizable; mutation-checked
(revert to raw -> the hold tests fail; drop topPad-delta -> that test fails).
_steerClearCurrentOwnerDeadRun cleared busy/activeStreamId/inflight but left
the status line, elapsed timer, and optimistic streaming badge lingering after
a recovered dead run. Add the rest of the app's stale-busy cleanup idiom
(setStatus/setComposerStatus/_clearActivityElapsedTimer/clearOptimisticSessionStreaming),
still bypassing setBusy(false) so its queue-drain can't clobber the restored draft.
vv.scale != 1 means the visual-viewport shrinkage reflects zoom, not the
keyboard; on Chromium 'force enable zoom' that produced a large spurious inset
that jittered on pan. Bail out of the inset when |vv.scale-1|>0.05. + test.
On touch-primary devices (iPad Safari) the on-screen keyboard shrinks the
visual viewport but not the layout viewport, so the bottom-pinned composer
sits behind the keyboard. Compute a --keyboard-bottom-inset from visualViewport
and pad the composer up by it; clear it when the keyboard closes / on
non-touch / no-visualViewport. Reuses the existing visualViewport + pageshow
hooks; no-op on desktop/fine-pointer.
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Strict providers (DeepSeek v4, newer OpenAI) reject an assistant message
carrying tool_calls: [] with HTTP 400. _sanitize_messages_for_api already
strips orphaned tool results, but not an assistant message that literally
stores an empty tool_calls list.
Drop the empty key on ALL THREE sanitized-dict sites that must agree:
_sanitize_messages_for_api, _api_safe_message_positions, and the
_safe_projection inside _restore_reasoning_metadata (so a tool_calls: [] row
projects identically on both sides of the metadata-carry-forward alignment and
doesn't lose its reasoning/id/timestamp every turn — Fable gate follow-up,
folded in).
Added mutation-checked regression tests: empty tool_calls dropped + populated
chain preserved (both sanitizer paths), and metadata carry-forward survives a
tool_calls: [] row.
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
test_tls_startup_failure_fallback_to_http read only the first 2000 bytes of
server stdout to assert 'TLS setup failed'. When the test interpreter lacks
optional deps (requests/websockets), startup emits a burst of plugin/tool
import warnings + the startup-config banner BEFORE the TLS line, pushing the
marker past the 2000-byte window -> spurious failure. Drain all
currently-available stdout (5s deadline, short-circuit on the marker) instead
of a single fixed-size read. Verified failing->passing in a fresh venv without
the optional deps. Test-only; distinct from the agent-package-isolation flake
cause already fixed in the same test.
Frontend fix for the model-picker provider-contamination vector of #5567 (the
backend HERMES_HOME vector shipped in #5577). Credit @b3nw for the root-cause trace.
_modelStateForSelect(sel, modelId) resolved the provider from sel.selectedOptions[0]
(the dropdown's CURRENTLY-selected option) instead of the option whose value matches
modelId. During a profile/tab switch or a model-list rebuild the dropdown transiently
still holds the PREVIOUS profile's default option (e.g. an ollama model), so the wrong
provider was stamped onto a model it doesn't own, persisted into the session's
model_provider, and (since _modelProviderForSend reads the stored value first) re-sent
on every turn -> 'Provider ollama...no API key' for a provider the session never used.
Fix: resolve the provider from the option whose VALUE === modelId; trust the
currently-selected option only when its value matches (preserving the user's exact
pick in the same-value/different-provider collision case). A genuinely-absent model
yields a null provider so the backend re-infers.
This is the WRITE-site fix — it stops all NEW contamination (new-session + send paths
route through _modelStateForSelect). Both Codex and Fable reviewed it SAFE.
Repairing sessions ALREADY poisoned before this shipped is deliberately deferred to a
separate follow-up: a 3-round Codex+Fable gate proved a frontend load-time repair that
reads the shared #modelSelect DOM is a cross-session race surface (it re-manufactured
the reverse-contamination it aimed to fix), and it can't reliably heal live-only-model
sessions anyway. The robust place for that repair is the backend chat-start boundary,
where session + profile identity is unambiguous. Already-poisoned sessions self-heal on
their next healthy send / manual re-pick in the meantime.
Tests: node-behavioral resolver scenarios (tab-switch race, same-value collision,
missing-model null, match-by-value, default->null) + the unrelated-selected-option
case from #5727. Partially addresses #5567 (write-vector); follow-up tracks the
already-poisoned-session repair.
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
A valid session:// deep link to a session owned by a DIFFERENT Hermes
profile used to look identical to a deleted session (404 -> frontend
self-heals away). Now GET /api/session returns a structured 409
session_profile_mismatch envelope (error/code/session_id/profile ONLY,
no transcript) ONLY when the owning profile is KNOWN, and loadSession()
catches it, switches to the owning profile, and retries once. Truly
missing/deleted or legacy None-profile sessions keep the 404 self-heal.
Gate fixes applied (Codex + Fable):
- Codex CORE: added a post-await stale-load guard after
_switchProfileForSessionLoad so a navigation during the switch can't
hijack the UI back to the old session.
- Fable Finding 1: only emit 409 when _session_profile is truthy;
a None-profile (missing/legacy) session under a non-default active
profile now keeps 404 instead of a useless profile=null 409 (which
skipped self-heal + spun the SSE reconnect against a dead sid). Both
detail branches. + 2 regression tests.
- Fable Finding 2: _switchProfileForSessionLoad now clears the sidebar
skeleton + re-renders from cache on switch-POST failure (mirrors the
#4671 canonical-switch catch), then rethrows, so a failed switch can't
strand the sidebar on the skeleton.
Reconciled tests/test_issue1611_session_profile_filtering.py (4 tests) to
the 409 contract while preserving the no-leak boundary assertion.
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
