Codex gate: the apply path now surfaces sanitized non-network fetch stderr, but
_QUERY_SECRET_RE only redacted access_token/token/password/auth/key — leaving
client_secret, private_token, oauth_token, api_key in a remote URL exposed in
the user-facing error. Broadened the pattern to cover those + app_secret/secret
+ api[_-]?key. Regression test added + proven non-vacuous.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Codex final-gate found the boundary<watermark advanced signal made
sidecar_advanced_past_watermark globally true, so a stale ts>watermark suffix
row appearing in state.db BEFORE the edited checkpoint bypassed the skip and
resurrected. Gate the bypass on state_replay_idx >= len(sidecar_visible_sequence)
(checkpoint consumed); apply the same gate to the same-second CORE-B exception.
Regression test added + proven non-vacuous. All legit cases (state-only reply
after checkpoint, CORE-B same-second) still pass.
Co-authored-by: AlexeyDsov <alexeydsov@gmail.com>
Final-gate Codex repro found a non-empty-sidecar drop: when the post-edit USER
turn is checkpointed into the sidecar (ts == advanced watermark) but its
ASSISTANT reply exists only in state.db (ts > watermark), the
sidecar_advanced_past_watermark guard was False (max_sidecar_timestamp ==
watermark, not >), so the 'timestamp > watermark' skip dropped the legitimate
reply. GET /api/session returned ['first','first reply','edited prompt'] and
omitted the assistant.
Fix: treat truncation_boundary < watermark (genuinely advanced) as advanced too,
so post-watermark state-only rows merge through even when the sidecar tail only
EQUALS the watermark. Conservative skip preserved for boundary None / == watermark.
Hoisted the loop-invariant boundary check out of the per-message loop. Regression
test added + proven non-vacuous.
Co-authored-by: AlexeyDsov <alexeydsov@gmail.com>
Opus deep-gate found a latent data-loss the Codex harnesses missed: the
empty-sidecar branch kept at_or_after (ts>=watermark) UNCONDITIONALLY, which is
only valid once the watermark was ADVANCED strictly past the original cutoff
(a new turn committed -> boundary < watermark). In the not-advanced states the
content above the watermark is the deleted suffix, so it got resurrected — the
same data-loss class this fix exists to kill. Reachable via crash/cold-load
metadata-vs-sidecar divergence.
- Keep at_or_after ONLY when boundary < watermark (genuinely advanced).
- boundary == watermark (just truncated) OR boundary is None (legacy: a positive
watermark always meant 'frozen at cutoff' pre-#4767) -> conservative ts<=watermark
filter; never resurrects.
- Replaced the backward-scan heuristic with the provable boundary guard.
- Removed the now-dead 'or (not sidecar_messages and watermark>0)' clause in the
non-empty-sidecar path (unreachable; empty sidecar returns earlier).
- Added 2 regression tests (boundary==watermark, legacy None-boundary), both proven
non-vacuous; updated 2 contributor tests that assumed advanced-watermark-without-boundary
(an unreachable state) to pass the boundary the real persistence path always writes.
Co-authored-by: AlexeyDsov <alexeydsov@gmail.com>
The contributor's empty-sidecar recovery uses the persisted truncation_boundary
to distinguish a legitimate prefix from a deleted suffix, but two production
call sites still passed only truncation_watermark — the default /api/session
full reload (routes.py ~9504) and _merged_session_messages_for_display
(routes.py ~6098) — so a normal full reload fell back to the old 'drop one
turn pair' heuristic and resurrected deleted suffix turns when an OLDER message
was edited leaving >=2 later turns. Thread truncation_boundary through both.
Co-authored-by: AlexeyDsov <alexeydsov@gmail.com>
ModuleNotFoundError fires both when cron.jobs is absent AND when an existing
cron/jobs.py fails to import its own dependency. Catch as exc and only return the
graceful {jobs:[],cron_unavailable:true} when exc.name is 'cron'/'cron.jobs';
re-raise otherwise so a real cron bug isn't silently swallowed.
In split-container / minimal Docker deployments the WebUI image may not ship the
agent's cron package on its import path, so the Tasks tab's GET /api/crons did
'from cron.jobs import list_jobs' and 500'd with ModuleNotFoundError — breaking the
whole tab. Now the import is guarded and returns {jobs: [], cron_unavailable: true}
instead, so the Tasks tab loads (empty) instead of erroring. Happy path unchanged.
Co-authored-by: reporter
Maintainer-completed @thanhtantran's #4241 (Nathan approved bringing it in):
- vi locale block in static/i18n.js (contributor's 1615-line translation) rebased onto master.
- Added vi to _LOGIN_LOCALE in api/routes.py (login-page localization, issue #1442 parity).
- Bumped the per-locale hardcoded-count assertions 13->14 across the locale-parity
tests (rtl, quota_chip, hide_suggestions, virtualize_transcript, no_git, help_tab,
verdigris/zeus skins, auxiliary_models) and added 'vi' to the composer-voice-buttons
LOCALES tuples. vi already carries all current English keys (verified 14x per key).
All 497 locale/parity tests pass. Co-authored-by: thanhtantran
The frontend polls /api/sessions every 5s while a turn streams
(static/sessions.js _streamingPollMs=5000), but the cache TTL was 2.5s
(_SESSIONS_CACHE_TTL_SECONDS) — so every streaming poll landed in a fresh window
and forced a full all_sessions() rebuild on the hot path under the global store
LOCK, pinning CPU to 100% and starving token rendering on large stores
(recurrence of #4672). The #4680 freeze-marker only de-dupes rebuilds WITHIN one
TTL window; with polls spaced wider than the TTL it never helped.
Fix: while _session_list_cache_streaming_freeze_marker() indicates an active
stream, use _SESSIONS_CACHE_STREAMING_TTL_SECONDS=10.0 (> the 5s poll) so the
sidebar holds steady across streaming polls. Live runtime state (active stream,
sort order, pending flags) is overlaid on every response regardless of cache, and
structural/settings changes still evict immediately via the source stamp — only a
streaming session's own persisted title/message_count can lag up to 10s, which the
#4672 design already deems acceptable. Idle path byte-for-byte unchanged.
Diagnosis credit: investigation reproduced 6 streaming polls -> 6 rebuilds before,
1 after. Co-authored-by: batkuip <reporter>
Took master's i18n.js as base (it has all 13 locale blocks + had reorganized the
kanban keys, so the stale-branch conflict block would have duplicated them in
zh-Hant), then inserted #2814's 5 kanban_view_* keys into every locale block after
kanban_new_task — real translations for it/ja/zh/zh-Hant/fr (from the PR), English
fallback elsewhere. All 13 locales now have exactly 1 of each key; locale-parity +
kanban tests green (291 passed).
The JSON/YAML tree viewer (#484) hardcoded its default: valid json/yaml
fenced blocks opened in Tree view at 10+ lines (`const showTree=lineCount>=10;`).
Make that default user-configurable while preserving current behavior.
- New settings (persisted via /api/settings -> settings.json):
- structured_code_default_view: auto | on | off (default auto)
- structured_code_auto_tree_lines: int 1..1000 (default 10)
- ui.js: replace the hardcode with a sanitized, unit-testable decision
helper _structuredCodeShowTree(mode, threshold, lineCount); Tree/Raw
toggle, Raw <pre><code>, parse-failure fallback, and YAML lazy-load
are unchanged.
- Appearance settings: 'JSON/YAML code blocks' select + 'Auto threshold
lines' input (disabled outside auto mode); wired through the existing
appearance-autosave and explicit-save paths; live re-render on change.
- boot.js seeds window globals; i18n labels added to all 13 locales.
- Default (auto + 10) reproduces the original >=10 behavior exactly.
The `/`, `/index.html`, and `/session/<id>` routes are the hottest
navigations. Each request re-read the ~190 KB static/index.html from disk
and re-applied two process-constant token substitutions
(__WEBUI_VERSION__, __MAX_UPLOAD_BYTES__), then ran inject_extension_tags()
(which itself reads the extension state + manifest from disk).
WEBUI_VERSION and MAX_UPLOAD_BYTES are fixed for the process lifetime, so
this caches the partially rendered template in an in-process cache keyed by
(size, mtime_ns) -- mirroring the existing _STATIC_CACHE pattern in this
file -- so a redeploy is still picked up without a restart. The per-session
CSRF token and runtime extension tags are still applied per request, so
output is byte-identical to before.
Verified: byte-identical to the old inline render, CSRF still varies per
request, cache invalidates on mtime change, ~5x faster shell render in a
local microbenchmark. Adds tests/test_index_shell_template_cache.py.
Two greptile findings on #4741:
- api/routes.py: removed the redundant 'from api.config import load_settings'
inside _session_list_payload_to_response — load_settings is already imported at
module scope, and this function only runs at request time (after module load).
- api/config.py: _load_yaml_config_file_raw now deep-copies the cached raw dict by
default so a caller can never mutate the shared _yaml_file_cache entry. The
internal _load_yaml_config_file passes _copy=False (it immediately runs
_expand_env_vars, which returns a fresh structure and never mutates its input)
so the /api/reasoning hot path keeps its cheap no-copy fast path; reload_config
uses the safe default copy.
The 'and loaded' guard skipped the _cfg_mtime stamp for a config that parses to
an empty {} dict, so get_config()'s current_mtime != _cfg_mtime stale check fired
on every call and re-entered reload_config() under _cfg_lock forever — a hot-path
spin for a freshly created/reset profile's {} config. Split the guard: the no-op
_cfg_cache.update is still skipped for {}, but _cfg_mtime is always stamped when
the file parsed to a dict, matching master's pre-#4662 behavior. Incidentally
fixes the pre-existing empty/None-config spin. Adds a regression test.
Phase 2 of #4662. reload_config() did a raw yaml.safe_load of config.yaml on
every call (~125ms+ on a large file), including on the profile-switch hot path
(~446ms load_settings stage in /api/sessions diagnostics). It now parses through
the mtime+size-keyed cache added in #4652, via a shared _load_yaml_config_file_raw
core (DRY: _load_yaml_config_file = raw + env-expand). reload_config still runs
its OWN env expansion pinned to the unscoped process-env view (block_process_env_fallback
reset + _thread_ctx.env={}) so the #798 TLS profile-isolation invariant is byte-for-byte
preserved; only the disk parse is now cached. An unchanged config.yaml is no longer
reparsed; an mtime change still busts the cache.
Phase 3 of #4662. _session_list_payload_to_response read settings.json once
per row via _redact_text's per-call load_settings() fallback (the multi-second
response_write stage in /api/sessions diagnostics on large lists). The
_sidebar_session_response_item -> _redact_text(_enabled=...) threading already
existed; this wires the caller to read the api_redact_enabled flag once and
thread it through every row, matching the read-once optimization
redact_session_data() already uses. Behavior-preserving; redaction still
applies (or not) exactly as before.
