Adds _session_message_dedup_key (full-precision timestamp) so true duplicates
(same role + content + EXACT timestamp) fold, while legitimately-repeated
identical turns with sub-second-distinct timestamps survive — avoiding the
#3268 data-loss class. Wired into both the no-sidecar path and the merge loop's
seen_dedup_keys guard. Closes#3346.
Co-authored-by: thanhtoantnt <thanhtoantnt@users.noreply.github.com>
Codex regression-gate finding: since Session.save() no longer auto-clears the
truncation_watermark, the unconditional 'timestamp > watermark' skip in
merge_session_messages_append_only became a permanent ceiling — a genuine future
state.db-only row (recovery/compaction, missed by the sidecar) would be silently
dropped from /api/session and model-context reconstruction forever. Only apply the
above-watermark skip while the sidecar has NOT advanced past the watermark. Preserves
the #2914 deleted-tail filtering (revert-verified). Adds 2 regression tests.
When state.db has many non-cron sessions, the normal sidebar query caps
at CLI_VISIBLE_SESSION_LIMIT (20) rows ordered by latest activity. Older
cron runs get squeezed out before _include_project_hidden_background_sidebar_sessions
can rescue them, making them invisible under their project chip.
Add a second-pass cron-only query with a higher cap (CRON_PROJECT_CHIP_LIMIT=200)
that merges into the CLI session list. The project-chip rescue layer then
marks them default_hidden so they stay addressable without polluting the
default sidebar window.
Verification: regression test seeds 25+ newer non-cron sessions and asserts
the older messageful cron session still appears with project_id set.
PR #3023 only updated Session.load() and Session.load_metadata_only(), leaving
three sibling validators (Session-internal _repair_stale_pending and the
/api/session/worktree/remove + /api/session/delete route handlers) still
gated on the old lowercase-only character set. That would have shipped a
confusing UX where api-* and reachy-voice-* sessions could be loaded into
the sidebar but rejected with HTTP 400 on delete or worktree removal.
This commit factors the validation into a single is_safe_session_id helper
in api.models and updates all five call sites to use it. Adds regression
coverage in tests/test_issue3023_safe_session_id_validators.py for both
the helper itself and a repo-wide guarantee that no narrow lowercase-only
magic string survives.
Closes the follow-up flagged by the parallel reviewer agent on #3023.
nesquena APPROVED 2026-05-22. Cherry-picked onto post-v0.51.127
master via 3-way apply. Resolved api/routes.py conflict: master had
the inline correctness fix from the deep-review iteration; PR
refactors it into _metadata_only_message_summary() helper. Took the
helper AND added profile= threading (post-#2827 master adds
profile-aware state.db reads). Kept master's pre-existing
test_api_session_reload_drops_stale_cached_user_tail_after_saved_assistant
alongside the PR's new test_metadata_fast_path_matches_reconciliation_for_restamped_replays.
Co-authored-by: dobby-d-elf <dobby.the.agent@gmail.com>
Two bugs combined to cause historical messages to vanish from the WebUI
after a session was continued in a later conversation.
**Bug 1 — missing `id` in state.db SELECT (models.py)**
`get_state_db_session_messages()` did not include the `id` column in its
SELECT, so every row got a `("legacy", ...)` merge key instead of
`("message_id", ...)`. The timestamp gate in
`merge_session_messages_append_only()` explicitly exempts `message_id`-keyed
rows from its "skip if older than newest sidecar message" rule, but
legacy-keyed rows are unconditionally dropped. With a session that has any
new sidecar messages (max_sidecar_timestamp == today), all older state.db
rows were silently discarded.
Fix: include `id` when the column is present so rows get proper
`("message_id", ...)` keys and survive the timestamp filter.
**Bug 2 — always reads active profile's state.db, not the session's (models.py + routes.py)**
`get_state_db_session_messages()` always called `_active_state_db_path()`,
which returns the currently-active profile's database. Sessions belonging to
a different profile (e.g. `jump`) were read from the wrong state.db, returning
either no rows or unrelated ones.
Fix: add an optional `profile` parameter; when supplied, resolve the path via
`_get_profile_home(profile)` with a fallback to the active path if the
profile-specific db does not exist. The call-site in `routes.py` now reads
`session.profile` and passes it through.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When display.personality is set in config.yaml (e.g. personality: taleb),
new sessions now inherit it automatically instead of starting with
personality=None and requiring an explicit /personality command.
This makes the selected personality sticky across new conversations rather
than requiring per-session activation.
Behavior:
- display.personality values 'none', 'default', 'neutral', '' are treated
as no personality (personality=None), matching TUI gateway semantics.
- Config read is wrapped in try/except — if it fails, personality falls
back to None (no crash, no regression).
- Case-insensitive: 'Taleb' normalizes to 'taleb'.
The /personality slash command still works for per-session overrides as
before; this change only affects the initial default.
Two functions on the /api/session/handoff-summary hot path were opening
sqlite3.connect(...) inside a bare `with` statement, which commits the
transaction at scope exit but does NOT close the connection. Per-turn
invocations accumulated state.db / state.db-wal file descriptors and
CPython heap pages on long-lived worker threads, surfacing as the
multi-GB VmRSS and 6x duplicated state.db fds observed on the live
instance (D0 pre-restart baseline: VmRSS 1,334,248 kB, 55 fds; cold
baseline after restart: VmRSS 136,668 kB, 10 fds).
Wrap both call sites with contextlib.closing(...) (already imported and
used at seven other sites in the same files) so the connection is
closed deterministically:
- api/models.py :: count_conversation_rounds
- api/routes.py :: _persist_handoff_summary_to_state_db
Regression test:
tests/test_issue2233_sqlite_connection_leak.py loops both functions
20 times against a tmp state.db and asserts /proc/<pid>/fd count
does not grow more than 2. Linux-only via sys.platform skip.
D1 live soak against a freshly-built worktree server (port 8799,
isolated HERMES_HOME / HERMES_WEBUI_STATE_DIR) hitting
/api/session/handoff-summary 20 times:
fd_before = 5
fd_after = 5 (growth 0, threshold < 5)
vmrss_before = 52636 kB
vmrss_after = 52636 kB (growth 0 kB, threshold < 30 MB)
The patched fix curve trends below the leak curve.
Rollback: single git revert <this-sha> reverts both file edits.
Refs #2233.
The full rebuild path scans SESSION_DIR via glob('*.json') and appends every loaded session to a plain list without deduplicating by session_id. When old-format session_*.json files coexist alongside WebUI-format xxx.json files (both sharing session_id), the index gets duplicate entries, causing frontend Vue key crashes.
Fix: use dict[session_id -> compact_entry] to naturally deduplicate.
The full rebuild path of _write_session_index scans SESSION_DIR via
glob('*.json') and appends every loaded session to a plain list without
deduplicating by session_id. When old-format session_*.json files coexist
alongside WebUI-format xxx.json files (both sharing the same session_id),
the same session appears multiple times in the index, causing frontend
Vue key collisions and a blank page.
Fix: use dict[session_id -> compact_entry] to naturally deduplicate.
Prefer the entry with the larger message_count when conflicts arise.
