#4771 added an explicit relay-failure 409 (code: gateway_run_unavailable)
to /api/approval/respond so a gateway approval whose run is gone keeps its
card actionable instead of silently failing. But the 409 was emitted
regardless of backend.
On the DEFAULT local in-process backend, every guarded command parks an
entry in tools.approval._gateway_queues (via _await_gateway_decision), and
the local streaming notify callback mirrors it into _pending tagged with
_GATEWAY_MIRROR_FLAG. There is no gateway run and no _STREAM_RUN_IDS entry
by design. _gateway_pending_approval_without_run_id() therefore returned
True for that purely local approval, so the handler 409'd
("active run unavailable") and refused to resolve an approval that resolves
perfectly well locally. Local-backend users hit this on every approval
click (reported by b3nw on Discord; independently filed as #4948 by
claw-io, which also notes the local-runtime trigger).
Fix: gate the 409 on the WebUI actually running the gateway chat backend
(webui_gateway_chat_enabled). Local approvals fall through to the local
resolution path; the legitimate #4771 gateway behaviour is unchanged.
Tests:
- New tests/test_issue4771_local_approval_regression.py drives the real
_handle_approval_respond handler and asserts: local backend -> 200 +
agent thread unblocked; gateway backend -> 409 preserved. These fail
before the fix (409) and pass after.
- The two existing #4771 tests that asserted the 409 are updated to pin
HERMES_WEBUI_CHAT_BACKEND=gateway at respond time, since the 409 is now
gateway-deployment behaviour.
Reported-by: b3nw
Reported-by: claw-io (#4948)
Opus SHIP-WITH-FIXES (no code change required): the comment/docstring overstated
that the session-list-change listener covers ALL structural mutations. In-app
mutations fire it, but externally-driven changes (scheduled cron completion,
external CLI writes) don't — for those the 30s streaming TTL is the load-bearing
backstop (≤30s bounded, self-healing lag). Comment-only accuracy fix; logic
unchanged (both gates: perf fix correct, freeze effective, staleness bounded).
/api/sessions re-ran the expensive CLI/cron session projection on every
poll while a turn was streaming — a major cause of the multi-second sidebar
latency + 100% CPU on cron-heavy installs (continuing #4672/#4808/#4889).
Root cause: the CLI/cron projection is gated by _CLI_SESSIONS_CACHE, whose
key folds in _sqlite_file_stat_cache_key -> _sqlite_content_fingerprint
(MAX(rowid) FROM messages). During a live turn the gateway writes a message
row per streamed delta, so that fingerprint advances on essentially every
~5s poll — busting the cache and re-running the full candidate-join +
projection (and the lineage-metadata pass) on every poll, contending for the
same SQLite/global lock the streaming worker holds.
The route-level session-list cache already froze its key during streaming
(#4808 _session_list_cache_streaming_freeze_marker), but that freeze never
reached this inner CLI-sessions cache, so the heavy CLI/cron query still
re-ran whenever the outer cache validated.
Fix: while any stream is active, the CLI-sessions cache key folds in the same
stable streaming-freeze marker (keyed only on the set of active stream ids)
instead of the volatile content fingerprint, and the cache TTL widens to a
streaming window. The projection is reused across polls mid-stream and
rebuilt at most once per window. The instant a stream starts/stops the marker
changes, so freshly-finished rows surface promptly. Structural mutations
(cron completion, new/renamed/archived sessions, attention) now also clear
the CLI cache via the existing session-list-changed listener — those signals
never fire per streamed token, which is what makes the freeze safe.
Idle behavior is byte-identical (freeze only engages with active streams).
Applies to both the active-profile and all-profiles cache paths.
Adds tests/test_issue4842_cli_sessions_streaming_freeze.py (6 tests): marker
idle/stable/changes semantics, the core 'key stable across message writes
while streaming' guarantee, idle key still advances with the fingerprint,
streaming TTL > idle TTL, and the structural-mutation listener clears the
CLI cache.
Settings -> Extensions offered an Install button even when extensions were
unconfigured, then dead-ended on 'Extensions not configured' because
install_extension required HERMES_WEBUI_EXTENSION_DIR to be set to an existing
directory. Default to a WebUI-managed root under STATE_DIR/extensions and
bootstrap it on first install so install just works with zero config; the env
var remains an optional override (never auto-created). Gallery-installed
extensions already auto-load via the runtime manifest, so the extension loads
on the next app-shell render with no further setup.
- api/extensions.py: _default_extension_root + _writable_extension_root;
_extension_root/_extension_root_status fall back to the managed default;
install_extension bootstraps via _writable_extension_root; dir-unavailable
warning only fires for an explicitly-set-but-broken env path.
- tests: update disabled-by-default expectations (configured=True, valid=False
pre-install) + new regression proving install bootstraps + auto-loads with
no env.
- docs/EXTENSIONS.md: document zero-config one-click install + env now optional.
- CHANGELOG [Unreleased].
Tool-call args were truncated to 120 chars at three sites (_truncate_tool_args,
the live _tool_args_snapshot in api/streaming.py, and _toolArgsSnapshot in
static/ui.js). For incidental args that's fine, but content/diff-bearing keys
(command/cmd/script/code/patch/diff/old_string/new_string/content/path/
file_path) feed the rendered tool card AND the diff reconstruction in
recovery-rebuilt sessions, so the cap silently corrupted long commands/paths and
produced broken/empty diffs. Exempt those keys with a much larger cap
(_TOOL_ARG_CONTENT_CAP = _TOOL_RESULT_SNIPPET_MAX = 4000) at every site, keyed by
key name; incidental args keep the 120 cap; storage stays bounded. Part of #4925.
format_wakeup_prompt() in api/background_process.py returned None for any event
type other than 'completion'/watch_*, so async_delegation events (produced when
a background delegate_task subagent finishes) were silently dropped — the
subagent showed as perpetually 'running' and its consolidated result never
re-entered the parent conversation.
Add an async_delegation branch that delegates to the agent-side
tools.process_registry.format_process_notification (which already knows the
async_delegation shape), so the subagent result is rendered into the wakeup
prompt and re-enters chat. Scoped to async_delegation specifically (not all
unknown types) so genuinely-unrecognized events are still skipped — preserving
the existing skip-unknown-event contract.
Reported + root-caused by @mydelren.
Co-authored-by: mydelren <mydelren@users.noreply.github.com>
On Windows a reserved-device-name file (nul/con/prn/aux/com1-9/lpt1-9) can land
in the working tree (e.g. a shell command redirecting to `> nul` under Git Bash,
which treats it as a literal filename). git can't delete it via the normal Win32
path, so `git clean -fd` exits non-zero — and force_apply_update treated that as
fatal, aborting the whole force update with 'Failed to remove untracked files
before force reset' and leaving the user stuck.
The clean is now best-effort: a failure is logged (stderr captured for
diagnostics) but non-fatal, because the subsequent `git reset --hard` overwrites
any tracked-file collisions regardless, and residual untracked files git can't
delete are harmless. Updates the prior test that asserted the (buggy) fatal
behavior to the corrected non-fatal contract, plus a new regression test.
Reported + root-caused by @rodboev.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Opus gate finding (BLOCKING): the two #4385 archive-reappears regression tests
mocked Session.load_metadata_only to return an archived stub WITHOUT writing a
real sidecar file, and never set SESSION_DIR. The new stat-gate in
_state_projection_sidecar_metadata short-circuits on SESSION_DIR/{sid}.json
before consulting metadata, so those mocks were never reached -> archived=False.
This is NOT a production regression (in prod, load_metadata_only reads exactly
SESSION_DIR/{sid}.json and returns None when absent, so archived metadata only
exists when the file does — in which case the stat() succeeds and behavior is
identical). The divergence existed only at the tests' mock boundary.
Fix: both tests now write a real archived sidecar JSON under a monkeypatched
SESSION_DIR, exercising the actual production read path (strictly better
coverage than mocking load_metadata_only). Also documented the
SESSION_DIR-as-sole-source assumption on the cache function per Opus's
forward-looking note.
GET /api/sessions pinned CPU to 100% and took multiple seconds on profiles with
many cron sessions. The state.db -> sidebar projection did three pieces of
redundant per-row file I/O, in BOTH the visible pass and the higher-capped
(CRON_PROJECT_CHIP_LIMIT=200) cron-only second pass:
1. _state_projection_sidecar_metadata(sid): uncached open() + 64KB prefix read +
JSON-key scan, once per row.
2. get_last_workspace(): up to two file reads + an is_dir()/remote probe,
returning the SAME workspace for every row.
3. cron/jobs.json: full read + parse per untitled cron row.
On a cron-heavy install that is hundreds of file reads per build, and because
the enclosing _CLI_SESSIONS_CACHE is keyed on a state.db content fingerprint
that advances on every streamed message row, the whole scan was re-paid on
essentially every 5s poll during a live turn (the recurring 100% CPU / slow
/api/sessions from #4672 -> #4808 -> #4842).
Fix:
- Memoize _state_projection_sidecar_metadata by the sidecar's
(path, mtime_ns, size, ctime_ns) stat signature in a bounded module cache, so
a warm build re-stats instead of re-reads. A rename/archive/edit bumps the
signature and invalidates just that one entry (no staleness). This survives
the 5s CLI-cache busting during streaming, which is the actual trigger.
Mirrors the Claude Code parse cache (#4718). Folded into
clear_cli_sessions_cache() for explicit-reset/test parity.
- Build-scoped memos for cron jobs.json (parse once -> job_id->name map) and
get_last_workspace() (resolve once), matching the existing _cron_pid() pattern.
Output is byte-identical. Adds tests/test_issue4842_cron_projection_perf.py
(7 tests: warm-rebuild no re-read, stat-change invalidation, dict-copy isolation,
missing-sidecar default, jobs.json once, workspace once, lifecycle clear).
Dual-gate (Codex + Opus) findings on the Claude Code transcript parse cache:
1. Cache-key staleness hardening: add st_ctime_ns to the (path, mtime_ns, size,
max_messages) key so a rare same-size, same-mtime in-place edit can't serve a
stale parse. Size is already the strong discriminator for append-only JSONL;
ctime_ns is cheap defense-in-depth (a spurious bump costs one harmless re-parse,
never a stale read).
2. Timestamp-fallback bug-for-bug fix: the row-builder guard used identity
(first_ts is None and last_ts is None) but the field assignment uses truthiness
(first_ts or last_ts or _mtime). They disagreed on a falsy 0.0 timestamp
(epoch-0/1970 transcripts), leaving created_at/updated_at/last_message_at = None
instead of the file mtime the pre-cache inline code fell back to. Guard now uses
the same 'not first_ts and not last_ts' test.
Adds 3 regression tests: epoch-0 mtime fallback, the shared-dict read-only
contract (pins the load-bearing invariant), and same-size+mtime ctime invalidation.
The /api/sessions cold build (which a profile switch triggers) was
dominated by get_claude_code_sessions() re-parsing every Claude Code
JSONL transcript line-by-line on each call — 650-1000ms on a 200-file
/ ~130MB tree, repeated on every switch, every 5s CLI-cache expiry, and
every sidebar poll, because the directory is global but the higher
session cache is keyed per active profile.
Memoize the per-file parse by (path, mtime_ns, size) so a warm build
re-stats files instead of re-parsing them; any genuine edit/append
changes the stat signature and invalidates just that file. Also hoist
the loop-invariant get_last_workspace() out of the Claude Code row loop
(it stats config + probes terminal cwd, ~200 redundant calls per build).
Measured /api/sessions cold-rebuild on a 250-session profile: ~670-880ms
-> ~100-200ms (verified in-browser). Output byte-identical.
Credit @rodboev for the browser-level profiling that isolated the real
bottleneck (the prior pre-warm hypothesis could not move it).
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Codex reproduced a SILENT regression: the PR's 4096-BYTE read could miss the
continuation marker the old read_text()[:4096] (4096 CHARS) found. Session.save()
writes compression_anchor_summary (ensure_ascii=False) BEFORE parent_session_id,
so a multi-byte emoji summary puts the marker at char offset ~1230 but byte
offset ~4900 — within the old 4096-char window, past a 4096-byte one → continuation
child no longer detected. Fix: read 16384 bytes (4096 chars worst-case for 4-byte
UTF-8) then slice [:4096] chars, keeping the same needle + bounded I/O. Multibyte
regression test added + proven non-vacuous; updated the read-size assertion to <=16384.
Co-authored-by: starship-s <starship-s@users.noreply.github.com>
Codex round-4 found my followlinks=True change introduced a perf regression: the
probe os.walk lacked the .git/.venv/node_modules/site-packages + skill-support-dir
pruning that iter_skill_index_files applies, so a skill vendoring a dependency
tree made this EVERY-CALL probe walk thousands of irrelevant files — defeating the
cache's whole purpose. The probe now prunes dirnames[:] with the same
EXCLUDED_SKILL_DIRS + SKILL_SUPPORT_DIRS sets as the compute path. Regression test
(change deep inside node_modules/ must not move the probe) added + non-vacuous.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Codex round-3 reproduced: _skill_tree_max_mtime_ns used os.walk(followlinks=False)
while the compute path iter_skill_index_files uses followlinks=True, so an edit to
a symlinked skill's target SKILL.md was invisible to the probe (stale up to TTL).
Added followlinks=True to the probe walk so it descends symlinked skill dirs and
sees target mtime changes. Regression test added + proven non-vacuous.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Codex re-gate: my prior 'TTL-expired + unchanged-mtime -> refresh TTL, no recompute'
branch could leave counts stale INDEFINITELY for an mtime-PRESERVING out-of-band
change (e.g. a git checkout restoring the old mtime) — worse than master's 8s
recompute. Codex proved it (cached (2,2) stayed (2,2) while a fresh compute returned
(1,2)). Now: within TTL + unchanged mtime serves cached (perf win kept); mtime
changed OR TTL expired both force a full recompute (the TTL is a genuine safety
net). Updated the row-2 test to assert the safety recompute.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Codex gate found a SILENT regression: the cache returned on a zero-I/O fast path
while inside the 300s TTL, so _skill_tree_max_mtime_ns never ran during the TTL
window and an out-of-band (CLI/git) skill create/edit/delete was invisible for up
to 5 minutes — WORSE than master's ~8s stale window. Restructured so the cheap
stat-only mtime probe runs on EVERY call (prompt change detection); the TTL now
only bounds the expensive SKILL.md recompute. Unchanged mtime still serves cached
+ refreshes TTL with no recompute (the perf win is preserved). Updated 2 tests
that encoded the flawed zero-I/O design + added a within-TTL-change regression
(proven non-vacuous).
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Two gate-found CORE/SILENT edges fixed on-branch:
- api/config.py: switching the main default to a different custom:* provider kept
the OLD provider's base_url, routing New Chat to the wrong endpoint. The
base_url-clear on provider-change now covers custom:* too (was exempted).
- static/sessions.js: newSession() skipped the configured default when S.session
was null but the picker held stale state (e.g. after deleting the last session),
inheriting the stale model. Default now wins over legacy picker regardless of
loaded-session state (explicit empty-composer override still takes priority).
- Updated 2 stale source-grep precedence assertions + widened a brittle 6000-char
newSession() slice window to 7000 (assignment block grew; intent unchanged).
- 2 new non-vacuous regression tests (custom base_url clear; precedence).
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Codex gate: the apply path now surfaces sanitized non-network fetch stderr, but
_QUERY_SECRET_RE only redacted access_token/token/password/auth/key — leaving
client_secret, private_token, oauth_token, api_key in a remote URL exposed in
the user-facing error. Broadened the pattern to cover those + app_secret/secret
+ api[_-]?key. Regression test added + proven non-vacuous.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
