34 test files read static/*.js via read_text()/open() without an encoding,
so Python's cp1252 default on Windows choked on non-Latin1 bytes and errored
at collection (Ubuntu CI defaults to utf-8, so it never saw this). Add
explicit encoding="utf-8" to those reads — purely mechanical, no assertion
changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(streaming): normalize inline thinking extraction across live and persisted turns (#3599)
# Conflicts:
# api/streaming.py
# static/messages.js
# static/ui.js
* fix(streaming): code-aware inline-thinking extraction + position-aware unclosed handling
Codex deep-review caught two regressions in the leading-only -> full-scan
rewrite (both silent data-mangling on the persist/reload path):
1. Code-span unawareness: the scanner only protected triple fences, so a
literal <think> in an inline single-backtick code span or an indented
(>=4-space/tab) code block got silently extracted into reasoning. Added
_inline_thinking_indented_code_at + inline-backtick tracking (Python +
the JS twin _thinkingIndentedCodeAt), so all three code contexts now keep
thinking tags visible.
2. Unclosed-tag truncation: any unmatched open tag moved the trailing prose
into reasoning. Now position-aware — a LEADING unclosed block (cut off
mid-thought) is still reasoning (#3455 intent), but an unclosed tag AFTER
visible content stays visible so literal typed tags don't truncate prose.
Gated partial handling on the previously-unused options.streaming param
(live streaming keeps 'still thinking' behavior; persist/reload does not).
Updated 2 tests that pinned the buggy behavior + added 4 regression tests
(inline-backtick, indented-code, mid-body-unclosed-visible, leading-unclosed-
extracted). Updated the node driver harness to include the new helper.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): recognize fenced code blocks indented 1-3 spaces
Codex round-3: a fence indented 1-3 spaces is valid Markdown but the fence
detector only matched at column 0, so a literal think tag inside such a fence
(not 4+-space indented code either) was still extracted. Both detectors
(_inline_thinking_fence_marker_at / _thinkingFenceMarkerAt) now walk back over
up to 3 leading spaces to a line start. Added backtick + tilde indented-fence
regression tests.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): O(n) inline-thinking scan + merge separate reasoning on reload
Round-4 Codex deep-review caught two real issues in my own fixes:
1. PERF (O(n^2)): the indented-code check (_inline_thinking_indented_code_at /
_thinkingIndentedCodeAt) scanned to line boundaries at EVERY character index,
plus the leading check sliced+stripped the whole prefix per unclosed tag. On
long no-newline content this was quadratic (~8.4s @ 200k, called repeatedly
on the streaming path). Replaced with incremental O(1)-per-iteration line
state (_line_is_indented_code / _lineIsIndentedCode evaluated only at line
starts) + a seen_nonspace flag. 200k now extracts in ~55-140ms.
2. RELOAD reasoning-drop: renderMessages() seeded the shared extractor with ''
so a message with BOTH an inline <think> block AND a separate m.reasoning
payload showed only the inline part — the separate payload was dropped
because the !thinkingText worklog resolution was then skipped. Now seeds with
the message's direct reasoning (m.reasoning_content||m.reasoning||...) so the
two MERGE (deduped); separate-only reasoning is preserved without promoting
it into visible prose.
Python + JS twins kept line-for-line parity. Added merge + perf + reload
regression tests; updated the reload structure test and the node driver harness
for the renamed helper.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): revert reload reasoning-seed; keep O(n) perf fix
Codex round-4 finding #2 (seed renderMessages' inline extractor with
m.reasoning so a separate payload merges) turned out to VIOLATE a deliberate
architectural invariant pinned by test_issue2565 +
test_sprint42: the reload content-extraction path must NOT touch
m.reasoning/m.reasoning_content — reasoning metadata is owned exclusively by
the Worklog Thinking Card path (_worklogReasoningTextFromMessage /
_assistantReasoningPayloadText), never conflated with inline-content
extraction (which would risk promoting provider reasoning into final-answer
prose). Reverted the ui.js seed to the PR's original `thinkingText` arg.
The inline+separate merge is still a genuine extractor capability (exercised
by the live streaming path via liveReasoningText) and is covered by a unit
test, just not invoked from the reload render path by design.
The O(n) perf fix (finding #1) and the code-awareness + position-aware
unclosed handling (rounds 1-3) are all retained.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): only lstrip extracted content when a leading block was removed
Codex round-5 catch: the extractor unconditionally lstripped the final content
(.lstrip() / .replace(/^\s+/,'')) even when NO thinking block was extracted, so
an assistant reply that legitimately starts with an indented code block or blank
lines lost its leading whitespace on live display, reload, and persistence. This
was a real regression vs master (master returned non-thinking content unchanged).
Now track leading_removed (set only when a LEADING thinking block/prefix is
actually extracted) and lstrip only in that case. Mid-body / no-thinking content
keeps its exact leading whitespace. Python + JS twins kept in parity; added
backend regression tests (indented-first preserved, leading-blank preserved,
leading-think still strips).
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* fix(streaming): reconnect restore prefers raw inflight accumulator
Codex round-6 CORE catch: on reconnect, the single-live-message restore used
(_liveInflightAssistant.content || ''). Because the PR now splits a leading
unclosed <think> into empty content, restoring from the split content dropped
the open tag — so a later </think> token leaked into the visible reply and
corrupted the live accumulator. Restore from
(_fullInflightAssistant || _liveInflightAssistant.content || '') so the raw
open tag survives reconnect and the accumulator stays correct. Added a
reconnect-restore regression test.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* Release v0.51.335 — Release KY (normalize inline thinking extraction, #3633)
Unify inline-thinking (<think>/<|channel>/<|turn|>) extraction across live,
reload, and persisted turns (#3599/#3633, @rodboev). Deep-reviewed: Opus +
6 Codex rounds; maintainer fixes resolved every Codex finding — code-awareness
(inline-backtick/indented/1-3-space fences keep literal tags visible),
position-aware unclosed handling, O(n) line scanning (was O(n^2) on long
content), conditional lstrip (preserve leading whitespace when no leading block
removed), and a reconnect-restore CORE fix (raw accumulator preferred so an open
<think> tag survives reconnect). Python + JS twins in parity. Full suite 8330,
Opus SHIP-SAFE, Codex SAFE-TO-SHIP, ESLint/scope-undef/ruff clean.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
---------
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: Hermes Agent <hermes-agent@nesquena-hermes.local>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Fixes <|turn|>thinking delimiter (was wrong as <|turn>thinking) in api/streaming.py, static/messages.js, and static/ui.js. Adds 13 regression tests. Independent review by @nesquena.
Remove ^ anchor from think/Gemma regexes in ui.js; trimStart() before startsWith checks in messages.js streaming path. Fixes MiniMax M2.7 and any model emitting leading newlines before <think>. 10 new tests, 768 total.
