* fix(streaming): normalize inline thinking extraction across live and persisted turns (#3599)
# Conflicts:
# api/streaming.py
# static/messages.js
# static/ui.js
* fix(streaming): code-aware inline-thinking extraction + position-aware unclosed handling
Codex deep-review caught two regressions in the leading-only -> full-scan
rewrite (both silent data-mangling on the persist/reload path):
1. Code-span unawareness: the scanner only protected triple fences, so a
literal <think> in an inline single-backtick code span or an indented
(>=4-space/tab) code block got silently extracted into reasoning. Added
_inline_thinking_indented_code_at + inline-backtick tracking (Python +
the JS twin _thinkingIndentedCodeAt), so all three code contexts now keep
thinking tags visible.
2. Unclosed-tag truncation: any unmatched open tag moved the trailing prose
into reasoning. Now position-aware — a LEADING unclosed block (cut off
mid-thought) is still reasoning (#3455 intent), but an unclosed tag AFTER
visible content stays visible so literal typed tags don't truncate prose.
Gated partial handling on the previously-unused options.streaming param
(live streaming keeps 'still thinking' behavior; persist/reload does not).
Updated 2 tests that pinned the buggy behavior + added 4 regression tests
(inline-backtick, indented-code, mid-body-unclosed-visible, leading-unclosed-
extracted). Updated the node driver harness to include the new helper.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): recognize fenced code blocks indented 1-3 spaces
Codex round-3: a fence indented 1-3 spaces is valid Markdown but the fence
detector only matched at column 0, so a literal think tag inside such a fence
(not 4+-space indented code either) was still extracted. Both detectors
(_inline_thinking_fence_marker_at / _thinkingFenceMarkerAt) now walk back over
up to 3 leading spaces to a line start. Added backtick + tilde indented-fence
regression tests.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): O(n) inline-thinking scan + merge separate reasoning on reload
Round-4 Codex deep-review caught two real issues in my own fixes:
1. PERF (O(n^2)): the indented-code check (_inline_thinking_indented_code_at /
_thinkingIndentedCodeAt) scanned to line boundaries at EVERY character index,
plus the leading check sliced+stripped the whole prefix per unclosed tag. On
long no-newline content this was quadratic (~8.4s @ 200k, called repeatedly
on the streaming path). Replaced with incremental O(1)-per-iteration line
state (_line_is_indented_code / _lineIsIndentedCode evaluated only at line
starts) + a seen_nonspace flag. 200k now extracts in ~55-140ms.
2. RELOAD reasoning-drop: renderMessages() seeded the shared extractor with ''
so a message with BOTH an inline <think> block AND a separate m.reasoning
payload showed only the inline part — the separate payload was dropped
because the !thinkingText worklog resolution was then skipped. Now seeds with
the message's direct reasoning (m.reasoning_content||m.reasoning||...) so the
two MERGE (deduped); separate-only reasoning is preserved without promoting
it into visible prose.
Python + JS twins kept line-for-line parity. Added merge + perf + reload
regression tests; updated the reload structure test and the node driver harness
for the renamed helper.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): revert reload reasoning-seed; keep O(n) perf fix
Codex round-4 finding #2 (seed renderMessages' inline extractor with
m.reasoning so a separate payload merges) turned out to VIOLATE a deliberate
architectural invariant pinned by test_issue2565 +
test_sprint42: the reload content-extraction path must NOT touch
m.reasoning/m.reasoning_content — reasoning metadata is owned exclusively by
the Worklog Thinking Card path (_worklogReasoningTextFromMessage /
_assistantReasoningPayloadText), never conflated with inline-content
extraction (which would risk promoting provider reasoning into final-answer
prose). Reverted the ui.js seed to the PR's original `thinkingText` arg.
The inline+separate merge is still a genuine extractor capability (exercised
by the live streaming path via liveReasoningText) and is covered by a unit
test, just not invoked from the reload render path by design.
The O(n) perf fix (finding #1) and the code-awareness + position-aware
unclosed handling (rounds 1-3) are all retained.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): only lstrip extracted content when a leading block was removed
Codex round-5 catch: the extractor unconditionally lstripped the final content
(.lstrip() / .replace(/^\s+/,'')) even when NO thinking block was extracted, so
an assistant reply that legitimately starts with an indented code block or blank
lines lost its leading whitespace on live display, reload, and persistence. This
was a real regression vs master (master returned non-thinking content unchanged).
Now track leading_removed (set only when a LEADING thinking block/prefix is
actually extracted) and lstrip only in that case. Mid-body / no-thinking content
keeps its exact leading whitespace. Python + JS twins kept in parity; added
backend regression tests (indented-first preserved, leading-blank preserved,
leading-think still strips).
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): reconnect restore prefers raw inflight accumulator
Codex round-6 CORE catch: on reconnect, the single-live-message restore used
(_liveInflightAssistant.content || ''). Because the PR now splits a leading
unclosed <think> into empty content, restoring from the split content dropped
the open tag — so a later </think> token leaked into the visible reply and
corrupted the live accumulator. Restore from
(_fullInflightAssistant || _liveInflightAssistant.content || '') so the raw
open tag survives reconnect and the accumulator stays correct. Added a
reconnect-restore regression test.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* Release v0.51.335 — Release KY (normalize inline thinking extraction, #3633)
Unify inline-thinking (<think>/<|channel>/<|turn|>) extraction across live,
reload, and persisted turns (#3599/#3633, @rodboev). Deep-reviewed: Opus +
6 Codex rounds; maintainer fixes resolved every Codex finding — code-awareness
(inline-backtick/indented/1-3-space fences keep literal tags visible),
position-aware unclosed handling, O(n) line scanning (was O(n^2) on long
content), conditional lstrip (preserve leading whitespace when no leading block
removed), and a reconnect-restore CORE fix (raw accumulator preferred so an open
<think> tag survives reconnect). Python + JS twins in parity. Full suite 8330,
Opus SHIP-SAFE, Codex SAFE-TO-SHIP, ESLint/scope-undef/ruff clean.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
---------
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: Hermes Agent <hermes-agent@nesquena-hermes.local>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* Harden interrupted recovery control filtering
* Redesign live-to-final assistant replies
* Fix live activity anchor test fixture
* Fix CI lint issues for live reply tests
* Strengthen live progress prompt contract
* Recover PR #3401 refresh on origin/master
* Repair live-to-final refresh regressions
* Fix live worklog refresh regressions
* Show live footer timer on initial stream start
* Restore live stream shell after reload
* Preserve per-frame live SSE replay cursors
* Preserve reasoning as Worklog Thinking cards
* Quiet Worklog Thinking card styling
* Align Worklog Thinking card styling
* Scope live Worklog Thinking cards by segment
* Suppress exact duplicate settled Thinking
* Close#3401 merge review test gaps
* fix(#3401): resolve 4 deep-review regressions (inline-think, reconnect-dup, neon skin, busy-gate worklog)
Deep review (Codex diff-vs-master + live-browser drive) of the live-to-final refactor
surfaced 4 regressions vs master that the rewritten suite no longer guarded:
1. Inline <think>…</think>answer reasoning vanished — _assistantReasoningPayloadText
used $-anchored regexes so a leading think block + visible answer extracted nothing
and the Thinking card never rendered. Removed the 3 $ anchors to match the
(non-anchored) display stripper. Live: inline-think thinking-only turn now renders.
2. (CORE) reconnect/reload duplicated the live reply — _rememberRunJournalCursor advanced
a closure-local seq but never wrote INFLIGHT[activeSid].lastRunJournalSeq, so a reload
replayed the journal from after_seq=0 over restored lastAssistantText. Now mirrors the
cursor onto INFLIGHT + schedules a throttled persist.
3. Neon skin silently broke — PR deleted the :root[data-skin="neon"] CSS but left Neon in
the picker. Restored the neon CSS block from master.
4. Settled tool-worklog rebuild gated purely on !S.busy — dropped every prior settled
turn's worklog when renderMessages re-ran during an active stream (switch-back to an
in-progress session). Restored master's !S.busy || (S.toolCalls && S.toolCalls.length).
Live: busy re-render now preserves tool cards (4→4, was 4→0).
Live-verified all 4 + confirmed #3709/#3592 invariants still hold (1 thinking card, none
below footer; distinct siblings preserved). + tests/test_issue3401_deep_review_fixes.py (7).
* test(#3401): realign 3 stale source-shape assertions to the deep-review fixes
Fix commit changed two source literals that existing stage tests scanned for:
- test_live_activity_timeline.py (x2): split anchor 'if(!S.busy){' → the restored
'if(!S.busy || (S.toolCalls&&S.toolCalls.length)){' guard (fix 4).
- test_run_journal_frontend_static.py: 'after_seq=0' not in source — fix 2's comment
contained that literal; rephrased the comment to 'the zero floor (after_seq of 0)'.
Intent of all three assertions unchanged; only the matched string updated. No code
behavior change.
* docs(changelog): v0.51.294 — Release JJ (stage-3401, #3401 live-to-final redesign)
---------
Co-authored-by: Frank Song <franksong2702@gmail.com>
Co-authored-by: Nathan-Hermes <nesquena-hermes@users.noreply.github.com>
Co-authored-by: nesquena-hermes <[email protected]>
nesquena-hermes