HERMES-GIELIE/tests/test_issue1257_llm_wiki_status.py

245 lines
8.4 KiB
Python
Raw Normal View History

2026-05-04 23:54:08 +00:00
from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
from urllib.parse import urlparse
from unittest.mock import patch
REPO = Path(__file__).resolve().parents[1]
def _write(path: Path, text: str = "# Synthetic\n") -> Path:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
return path
def test_llm_wiki_status_reads_synthetic_fixture_without_exposing_content(tmp_path, monkeypatch):
"""The wiki status API should summarize counts/mtime without leaking page text."""
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "SCHEMA.md", "# Schema\n")
_write(wiki / "index.md", "# Index\n")
_write(wiki / "log.md", "# Log\n## [2026-05-04] update | Secret project name\n- Details stay private\n")
_write(
wiki / "entities" / "private-agent.md",
"---\ntitle: Private Agent\nupdated: 2026-05-04\n---\nSensitive body text must not ship.\n",
)
_write(wiki / "concepts" / "safe-summary.md", "---\ntitle: Safe Summary\n---\nMore private text\n")
_write(wiki / "raw" / "articles" / "source.md", "Raw source body should not count as wiki page\n")
monkeypatch.setenv("WIKI_PATH", str(wiki))
status = routes._build_llm_wiki_status()
assert status["available"] is True
assert status["enabled"] is True
assert status["entry_count"] == 2
assert status["page_count"] == 2
assert status["raw_source_count"] == 1
assert status["last_updated"] is not None
Release v0.51.230 — Release GX (stage-p14 — extract <think> to m.reasoning #3455 + LLM Wiki last-writer #1257) (#3466) * Release v0.51.230 (stage-p14): extract <think> to m.reasoning #3455 + LLM Wiki last-writer (#1257) Salvage of #3455 (@gsurenull): dropped the stale api/config.py bits (MiniMax-M3 + SCHEMA_VERSION 3->4 — both already on master via #3374). Kept the two genuine fixes: (1) _splitThinkFromContent persist-path extraction of inline <think> blocks into m.reasoning (fixes 30-50% session bloat for reasoning-only providers like MiniMax-M3); (2) LLM Wiki status Last-writer 3-tier fallback (was always 'Not available' since #1257). Added 9 Node-driven think-split regression tests (data-loss guards: content-before/after preserved, unclosed blocks intact, lookalike tags not extracted). * fix(#3455): renderer-matching think extraction + wiki symlink/bounded-read guards (Codex review) Codex review of stage-p14 found 3 SILENT bugs, all fixed: (1) DATA-LOSS: _splitThinkFromContent's Pass-2 whole-body scan extracted a CLOSED literal <think>...</think> from visible prose/code (e.g. inside a fenced code block) into m.reasoning, emptying it — more aggressive than the renderer (which only strips LEADING blocks). Removed Pass 2; extraction now matches _streamDisplay semantics (leading-only, loop captures consecutive leading blocks). +fenced-code regression test. (2) PRIVACY: _llm_wiki_last_writer followed symlinked .md pages resolving OUTSIDE the wiki (is_file follows symlinks), leaking external frontmatter. Now requires resolved path under wiki_root. +symlink-containment regression test. (3) CONTRACT/PERF: replaced full read_text() with bounded line-by-line reads (frontmatter block only / capped log-heading scan), never page bodies. * fix(#3455): think-split is leading-single (renderer-matching) + fix 2 stale source-match tests Codex re-review finding #2: looping consecutive leading blocks diverged from the renderer (_streamDisplay/_parseStreamState strip ONE leading block). Now extracts exactly one leading block. Also updated 2 tests that asserted pre-split implementation strings: test_live_stream_tokens_persist (content:assistantText -> content:split.content, invariant preserved) and the consecutive-blocks test. NOTE: Codex finding #1 (client-only split doesn't persist server-side) is a separate architectural decision pending Nathan. * feat(#3455): split inline <think> server-side before s.save() so persisted file is compacted (Codex #1) Codex finding #1: the think-split was client-only, so the SAVED session file still carried inline <think> blocks (bloat) — the fix only compacted the browser copy. Added _split_thinking_from_content (api/streaming.py), a server-side twin of the JS helper with identical leading-only/single-block semantics, applied to the final assistant message before s.save() (extended the existing reasoning-persist block). Merges with on_reasoning-stream reasoning. +8 backend-parity regression tests covering the mid-body-code-block data-loss guard, unclosed-intact, single-leading, none-content. * test: update 3 save-path source-assertion tests for #3455 server-side think-split The backend think-split (api/streaming.py reasoning-persist block) changed the literal code shape + grew the pre-save block, breaking 8 source-assertion tests that anchor on it: - test_sprint42: assert _rm['reasoning']=_reasoning_text -> now _merged_reasoning/_existing_reasoning + _split_thinking_from_content present (intent preserved: reasoning persisted before save). - test_pr1318 (6) + test_pr1341: re-anchored the locator from the changed 'if _reasoning_text and s.messages:' line to the stable 'Persist reasoning trace in the session' comment marker; bumped the 1341 byte-distance limit 15000->16000 (the test self-documents bumping on legit pre-save growth). All behavioral invariants (reasoning persisted + context fields before save) unchanged. --------- Co-authored-by: nesquena-hermes <[email protected]>
2026-06-03 05:48:35 +00:00
# log.md in the fixture has a "## [2026-05-04] update | ..." heading,
# so the new last-writer reader must surface that action verb.
assert status["last_writer"] == "ai-agent (update)"
2026-05-04 23:54:08 +00:00
assert status["toggle_available"] is False
assert status["docs_url"].endswith("/research-llm-wiki")
serialized = repr(status)
assert "Sensitive body text" not in serialized
assert "Secret project name" not in serialized
assert str(wiki) not in serialized
def test_llm_wiki_status_reports_unavailable_when_path_missing(tmp_path, monkeypatch):
import api.routes as routes
missing = tmp_path / "does-not-exist"
monkeypatch.setenv("WIKI_PATH", str(missing))
status = routes._build_llm_wiki_status()
assert status["available"] is False
assert status["enabled"] is False
assert status["entry_count"] == 0
assert status["page_count"] == 0
assert status["raw_source_count"] == 0
assert status["last_updated"] is None
assert status["status"] == "missing"
def test_api_wiki_status_route_is_registered(monkeypatch, tmp_path):
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "entities" / "one.md")
monkeypatch.setenv("WIKI_PATH", str(wiki))
captured = {}
def fake_j(handler, payload, status=200, extra_headers=None):
captured["status"] = status
captured["payload"] = payload
with patch("api.routes.j", side_effect=fake_j):
handled = routes.handle_get(SimpleNamespace(), urlparse("/api/wiki/status"))
assert handled is True
assert captured["status"] == 200
assert captured["payload"]["entry_count"] == 1
def test_insights_panel_fetches_and_renders_llm_wiki_status_card():
panels_src = (REPO / "static" / "panels.js").read_text(encoding="utf-8")
index_src = (REPO / "static" / "index.html").read_text(encoding="utf-8")
style_src = (REPO / "static" / "style.css").read_text(encoding="utf-8")
assert "api('/api/wiki/status')" in panels_src
assert "function _renderLlmWikiStatus" in panels_src
assert "llmWikiStatusCard" in index_src
assert "wiki-status-card" in style_src
assert "raw/" in panels_src
assert "recent_entries" not in panels_src
Release v0.51.230 — Release GX (stage-p14 — extract <think> to m.reasoning #3455 + LLM Wiki last-writer #1257) (#3466) * Release v0.51.230 (stage-p14): extract <think> to m.reasoning #3455 + LLM Wiki last-writer (#1257) Salvage of #3455 (@gsurenull): dropped the stale api/config.py bits (MiniMax-M3 + SCHEMA_VERSION 3->4 — both already on master via #3374). Kept the two genuine fixes: (1) _splitThinkFromContent persist-path extraction of inline <think> blocks into m.reasoning (fixes 30-50% session bloat for reasoning-only providers like MiniMax-M3); (2) LLM Wiki status Last-writer 3-tier fallback (was always 'Not available' since #1257). Added 9 Node-driven think-split regression tests (data-loss guards: content-before/after preserved, unclosed blocks intact, lookalike tags not extracted). * fix(#3455): renderer-matching think extraction + wiki symlink/bounded-read guards (Codex review) Codex review of stage-p14 found 3 SILENT bugs, all fixed: (1) DATA-LOSS: _splitThinkFromContent's Pass-2 whole-body scan extracted a CLOSED literal <think>...</think> from visible prose/code (e.g. inside a fenced code block) into m.reasoning, emptying it — more aggressive than the renderer (which only strips LEADING blocks). Removed Pass 2; extraction now matches _streamDisplay semantics (leading-only, loop captures consecutive leading blocks). +fenced-code regression test. (2) PRIVACY: _llm_wiki_last_writer followed symlinked .md pages resolving OUTSIDE the wiki (is_file follows symlinks), leaking external frontmatter. Now requires resolved path under wiki_root. +symlink-containment regression test. (3) CONTRACT/PERF: replaced full read_text() with bounded line-by-line reads (frontmatter block only / capped log-heading scan), never page bodies. * fix(#3455): think-split is leading-single (renderer-matching) + fix 2 stale source-match tests Codex re-review finding #2: looping consecutive leading blocks diverged from the renderer (_streamDisplay/_parseStreamState strip ONE leading block). Now extracts exactly one leading block. Also updated 2 tests that asserted pre-split implementation strings: test_live_stream_tokens_persist (content:assistantText -> content:split.content, invariant preserved) and the consecutive-blocks test. NOTE: Codex finding #1 (client-only split doesn't persist server-side) is a separate architectural decision pending Nathan. * feat(#3455): split inline <think> server-side before s.save() so persisted file is compacted (Codex #1) Codex finding #1: the think-split was client-only, so the SAVED session file still carried inline <think> blocks (bloat) — the fix only compacted the browser copy. Added _split_thinking_from_content (api/streaming.py), a server-side twin of the JS helper with identical leading-only/single-block semantics, applied to the final assistant message before s.save() (extended the existing reasoning-persist block). Merges with on_reasoning-stream reasoning. +8 backend-parity regression tests covering the mid-body-code-block data-loss guard, unclosed-intact, single-leading, none-content. * test: update 3 save-path source-assertion tests for #3455 server-side think-split The backend think-split (api/streaming.py reasoning-persist block) changed the literal code shape + grew the pre-save block, breaking 8 source-assertion tests that anchor on it: - test_sprint42: assert _rm['reasoning']=_reasoning_text -> now _merged_reasoning/_existing_reasoning + _split_thinking_from_content present (intent preserved: reasoning persisted before save). - test_pr1318 (6) + test_pr1341: re-anchored the locator from the changed 'if _reasoning_text and s.messages:' line to the stable 'Persist reasoning trace in the session' comment marker; bumped the 1341 byte-distance limit 15000->16000 (the test self-documents bumping on legit pre-save growth). All behavioral invariants (reasoning persisted + context fields before save) unchanged. --------- Co-authored-by: nesquena-hermes <[email protected]>
2026-06-03 05:48:35 +00:00
def test_last_writer_reads_frontmatter(tmp_path):
"""#3455 part 2: the Last writer field reads page frontmatter updated_by/writer/author."""
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "entities" / "a.md", "---\ntitle: A\nupdated_by: alice\n---\nbody\n")
pages = routes._llm_wiki_page_files(wiki)
assert routes._llm_wiki_last_writer(wiki, pages) == "alice"
def test_llm_wiki_status_rechecks_cached_page_targets(monkeypatch, tmp_path):
import os as _os
import api.routes as routes
wiki = tmp_path / "wiki"
page = _write(wiki / "concepts" / "sub" / "real.md", "---\nauthor: public\n---\nbody\n")
_write(wiki / ".env", "---\nauthor: hidden-author\n---\nPRIVATE=1\n")
routes._llm_wiki_clear_page_files_cache()
monkeypatch.setenv("WIKI_PATH", str(wiki))
monkeypatch.setattr(routes, "_WIKI_ALLOWLIST_TTL", 60.0)
assert routes._llm_wiki_page_files(wiki) == [page]
page.unlink()
try:
page.symlink_to(_os.path.join("..", "..", ".env"))
except (OSError, NotImplementedError):
import pytest
pytest.skip("symlinks not supported on this platform")
status = routes._build_llm_wiki_status()
assert status["page_count"] == 0
assert status["last_writer"] == "ai-agent"
assert "hidden-author" not in repr(status)
def test_llm_wiki_status_drops_cached_entry_replaced_by_directory(monkeypatch, tmp_path):
import api.routes as routes
wiki = tmp_path / "wiki"
page = _write(wiki / "concepts" / "sub" / "real.md", "---\nauthor: public\n---\nbody\n")
routes._llm_wiki_clear_page_files_cache()
monkeypatch.setenv("WIKI_PATH", str(wiki))
monkeypatch.setattr(routes, "_WIKI_ALLOWLIST_TTL", 60.0)
assert routes._llm_wiki_page_files(wiki) == [page]
page.unlink()
page.mkdir()
status = routes._build_llm_wiki_status()
assert status["page_count"] == 0
assert status["last_writer"] == "ai-agent"
def test_llm_wiki_status_last_writer_rechecks_identity(monkeypatch, tmp_path):
import os as _os
import api.routes as routes
wiki = tmp_path / "wiki"
page = _write(wiki / "concepts" / "real.md", "---\nauthor: public\n---\nbody\n")
_write(wiki / ".env", "---\nauthor: hidden-author\n---\nPRIVATE=1\n")
routes._llm_wiki_clear_page_files_cache()
monkeypatch.setenv("WIKI_PATH", str(wiki))
original = routes._llm_wiki_allowlisted_entries
def swapped(root):
entries = original(root)
page.unlink()
page.symlink_to(_os.path.join("..", ".env"))
return entries
monkeypatch.setattr(routes, "_llm_wiki_allowlisted_entries", swapped)
status = routes._build_llm_wiki_status()
assert status["page_count"] == 0
assert status["last_updated"] is None
assert status["last_writer"] == "ai-agent"
assert "hidden-author" not in repr(status)
def test_llm_wiki_status_log_heading_rechecks_identity(monkeypatch, tmp_path):
import os as _os
import api.routes as routes
wiki = tmp_path / "wiki"
_write(wiki / "log.md", "## [2026-06-19] update | safe\n",)
_write(wiki / ".env", "## [2026-06-19] leak | hidden\n")
monkeypatch.setenv("WIKI_PATH", str(wiki))
original_open = routes.os.open
swapped = {"done": False}
def fake_open(path, flags, mode=0o777):
if not swapped["done"] and Path(path) == wiki / "log.md":
swapped["done"] = True
(wiki / "log.md").unlink()
(wiki / "log.md").symlink_to(_os.path.join(".env"))
return original_open(path, flags, mode)
monkeypatch.setattr(routes.os, "open", fake_open)
status = routes._build_llm_wiki_status()
assert status["last_writer"] == "ai-agent"
Release v0.51.230 — Release GX (stage-p14 — extract <think> to m.reasoning #3455 + LLM Wiki last-writer #1257) (#3466) * Release v0.51.230 (stage-p14): extract <think> to m.reasoning #3455 + LLM Wiki last-writer (#1257) Salvage of #3455 (@gsurenull): dropped the stale api/config.py bits (MiniMax-M3 + SCHEMA_VERSION 3->4 — both already on master via #3374). Kept the two genuine fixes: (1) _splitThinkFromContent persist-path extraction of inline <think> blocks into m.reasoning (fixes 30-50% session bloat for reasoning-only providers like MiniMax-M3); (2) LLM Wiki status Last-writer 3-tier fallback (was always 'Not available' since #1257). Added 9 Node-driven think-split regression tests (data-loss guards: content-before/after preserved, unclosed blocks intact, lookalike tags not extracted). * fix(#3455): renderer-matching think extraction + wiki symlink/bounded-read guards (Codex review) Codex review of stage-p14 found 3 SILENT bugs, all fixed: (1) DATA-LOSS: _splitThinkFromContent's Pass-2 whole-body scan extracted a CLOSED literal <think>...</think> from visible prose/code (e.g. inside a fenced code block) into m.reasoning, emptying it — more aggressive than the renderer (which only strips LEADING blocks). Removed Pass 2; extraction now matches _streamDisplay semantics (leading-only, loop captures consecutive leading blocks). +fenced-code regression test. (2) PRIVACY: _llm_wiki_last_writer followed symlinked .md pages resolving OUTSIDE the wiki (is_file follows symlinks), leaking external frontmatter. Now requires resolved path under wiki_root. +symlink-containment regression test. (3) CONTRACT/PERF: replaced full read_text() with bounded line-by-line reads (frontmatter block only / capped log-heading scan), never page bodies. * fix(#3455): think-split is leading-single (renderer-matching) + fix 2 stale source-match tests Codex re-review finding #2: looping consecutive leading blocks diverged from the renderer (_streamDisplay/_parseStreamState strip ONE leading block). Now extracts exactly one leading block. Also updated 2 tests that asserted pre-split implementation strings: test_live_stream_tokens_persist (content:assistantText -> content:split.content, invariant preserved) and the consecutive-blocks test. NOTE: Codex finding #1 (client-only split doesn't persist server-side) is a separate architectural decision pending Nathan. * feat(#3455): split inline <think> server-side before s.save() so persisted file is compacted (Codex #1) Codex finding #1: the think-split was client-only, so the SAVED session file still carried inline <think> blocks (bloat) — the fix only compacted the browser copy. Added _split_thinking_from_content (api/streaming.py), a server-side twin of the JS helper with identical leading-only/single-block semantics, applied to the final assistant message before s.save() (extended the existing reasoning-persist block). Merges with on_reasoning-stream reasoning. +8 backend-parity regression tests covering the mid-body-code-block data-loss guard, unclosed-intact, single-leading, none-content. * test: update 3 save-path source-assertion tests for #3455 server-side think-split The backend think-split (api/streaming.py reasoning-persist block) changed the literal code shape + grew the pre-save block, breaking 8 source-assertion tests that anchor on it: - test_sprint42: assert _rm['reasoning']=_reasoning_text -> now _merged_reasoning/_existing_reasoning + _split_thinking_from_content present (intent preserved: reasoning persisted before save). - test_pr1318 (6) + test_pr1341: re-anchored the locator from the changed 'if _reasoning_text and s.messages:' line to the stable 'Persist reasoning trace in the session' comment marker; bumped the 1341 byte-distance limit 15000->16000 (the test self-documents bumping on legit pre-save growth). All behavioral invariants (reasoning persisted + context fields before save) unchanged. --------- Co-authored-by: nesquena-hermes <[email protected]>
2026-06-03 05:48:35 +00:00
def test_last_writer_rejects_symlink_outside_wiki(tmp_path):
"""#3455 review (Codex): a symlinked .md page resolving OUTSIDE the wiki must
not be read — its frontmatter must never leak into the status card."""
import api.routes as routes
outside = tmp_path / "outside"
outside.mkdir(parents=True, exist_ok=True)
secret = outside / "private.md"
secret.write_text("---\nauthor: outside-secret\n---\nprivate body\n", encoding="utf-8")
wiki = tmp_path / "wiki" / "entities"
wiki.mkdir(parents=True, exist_ok=True)
link = wiki / "linked.md"
try:
link.symlink_to(secret)
except (OSError, NotImplementedError):
import pytest
pytest.skip("symlinks not supported on this platform")
wiki_root = tmp_path / "wiki"
pages = routes._llm_wiki_page_files(wiki_root)
writer = routes._llm_wiki_last_writer(wiki_root, pages)
# The external symlink's frontmatter must NOT surface; falls back to ai-agent.
assert writer != "outside-secret"
assert "outside-secret" not in writer