Commit graph

522 commits

Author SHA1 Message Date
nesquena-hermes
cbfc544f50 fix(renderer): YAML/JSON/diff code blocks lose newlines (#1618 / #1463)
Closes #1618 (reported by @Zixim) and corrects #1463's previous fix.

Bug: YAML, JSON, and diff/patch fenced code blocks render flattened to a
single line. Reporter noted the bug persisted v0.50.279 -> v0.50.291 ->
v0.50.292 despite PR #1516's CSS-only "fix".

Root cause: PR #484 (v0.50.237) added a JSON/YAML tree-viewer that routes
those languages through <div class="code-tree-wrap">...<pre class="tree-raw-view">
instead of bare <pre>. Same release added the diff/patch coloring path
that emits <pre class="diff-block">. The _pre_stash regex at
static/ui.js:1914 matched only literal <pre> with no attributes:

    <pre>[\s\S]*?<\/pre>

Both new shapes failed to match, fell through to the paragraph-wrap pass,
and \n characters inside the code blocks got replaced with <br> tags
inside <code>. By the time Prism ran, no newlines remained for the CSS
rule (PR #1516, language-yaml .token { white-space: pre !important }) to
preserve.

Fix: relax the regex to accept any attribute on <pre>:

    <pre>[\s\S]*?<\/pre>  ->  <pre[^>]*>[\s\S]*?<\/pre>

One regex character. Pulls JSON, YAML, and diff/patch blocks into the
stash so paragraph-wrap can't mangle them. Bash, Python, Go, etc. were
never affected because they emit bare <pre>.

Tests: 9 new (2 source-string invariants + 7 behavioural via node-driver
against the actual static/ui.js renderMd()). 6 of the 7 behavioural tests
fail on master and pass with the fix; the 3 sanity checks (yml-alias,
bash, mermaid) pass on both.

Plus widened source-scan window in 3 pre-existing test_745 assertions
from 400 to 1500 chars. The new comment block above the fixed regex
pushed it past the previous scan window. Pure window-narrowness bug,
not a behavior regression.

4245 -> 4254 passing.
2026-05-04 18:11:58 +00:00
test
6bbf913e22 Stage 294: PR #1631 — streaming stability trio (closes #1623, #1624, #1625) by @nesquena-hermes — APPROVED 2026-05-04 17:13:08 +00:00
nesquena-hermes
66b925f59d fix(cache): stamp /api/models disk cache with WebUI version + schema version (#1633)
Closes #1633. STATE_DIR/models_cache.json was persisted across server
restarts without any version stamp, so a Docker container update from
version A to B read the cache file written by version A — users saw
stale picker contents (missing models, phantom provider groups) for
up to 24 hours until either the TTL expired, an unrelated provider
edit triggered invalidate_models_cache(), or they manually deleted
the file.

Reporter Deor (Discord) updated to v0.50.292 — which contained fixes
for #1538, #1539, and #1568 — did a hard refresh and cleared site
data, and still saw byte-for-byte identical picker contents because
the server kept reading the v0.50.281 cache file off the host-mounted
state volume.

Fix:
  * _save_models_cache_to_disk() stamps payloads with _webui_version
    (resolved lazily from api.updates.WEBUI_VERSION via sys.modules
    lookup to avoid the api.config <-> api.updates circular import)
    and _schema_version = 2.
  * New _is_loadable_disk_cache() validator checks both stamps in
    addition to shape. Mismatch on either field rejects the load.
  * _load_models_cache_from_disk() calls the new validator and
    strips the disk-only metadata before returning, so the rest of
    the code sees the same shape it always did.
  * _is_valid_models_cache() kept loose (shape-only) so in-memory
    cache writes that never touch disk don't fail validation.

Schema version is independent of the WebUI version stamp so future
cache-shape changes can invalidate older releases without relying
on a tag bump alone.

Early-init edge case (api.updates not yet loaded) skips the version
check rather than wedging the boot — at worst an unstamped file is
written once and rejected on the next call.

Updated existing tests/test_model_cache_metadata.py to use subset/
round-trip semantics rather than byte-for-byte equality, since the
disk payload now has additional stamps. The four response-shape
fields still round-trip verbatim; the load result is unchanged
(stamps stripped). 19 new regression tests.

4180 -> 4199 tests pass.
2026-05-04 17:03:02 +00:00
nesquena-hermes
040cb8af70 Apply Opus pre-release SHOULD-FIX + NITs (in-PR per release policy)
SHOULD-FIX: rate-limit _repair_stale_pending repair-firing telemetry. Switch
from unconditional logger.warning to age-keyed: WARNING when pending_age <
5min (the diagnostically valuable race window — actual leak-path candidates
that slipped past the grace guard) and DEBUG for the long-tail (orphaned
sidecars from prior process lifetimes). Prevents reconnect loops on stuck
sessions from flooding the log while preserving the diagnostic signal we
want for tuning _REPAIR_STALE_PENDING_GRACE_SECONDS empirically.

NIT: _LOCAL_SERVER_PROVIDERS expanded with lm-studio (hyphenated alias used
in some custom_providers configs and already recognized at api/config.py:2189
for SSRF host trust) and localai (LocalAI project). Test parametrize expanded
from 7 to 11 names, also covering pre-existing koboldcpp and textgen for
symmetry. +4 regression tests.

NIT (docs): CHANGELOG callout for the RFC1918 behavior change. Internal-
network OpenAI-compatible proxies now preserve the model prefix on private-IP
base_urls. Documented the migration path: configure as a custom_providers
entry to bypass the local-server detection.

NIT (deferred, optional): narrowing the heuristic to is_loopback only is
left as future work; the broader scope was an explicit goal in the bug
body and Opus flagged it as SHOULD-DISCUSS-but-not-block.

4184 -> 4188 passing. 0 regressions. ~10 LOC absorbed total.
2026-05-04 16:50:22 +00:00
nesquena-hermes
bea57beba9 fix(streaming): SSE heartbeat alignment, repair grace period, local-server model id preservation (#1623, #1624, #1625)
Closes #1623 — Lower SSE app heartbeat from 30s to 5s at every long-lived
handler (main agent, terminal, gateway-watcher, approval-poller, clarify-poller).
Kernel TCP keepalive declares peer dead at 25s worst-case (10s KEEPIDLE +
5s KEEPINTVL * 3 KEEPCNT, added v0.50.289 #1581). 30s app heartbeat let the
kernel tear sockets down on flaky networks before the app sent its first
keepalive byte — drops at ~10s during long thinking phases. New named
constant _SSE_HEARTBEAT_INTERVAL_SECONDS=5; regression test pins the
inequality (app_heartbeat * 2 <= kernel_window) so future tuning can't
re-introduce the misalignment.

Closes #1624 — Add 30s grace period to _repair_stale_pending() trigger.
Without it, any narrow race between the streaming thread clearing
pending_user_message and STREAMS.pop(stream_id) produces a false-positive
'Previous turn did not complete.' marker on a turn that finished correctly
(reproducible after every command-approval turn). Defense-in-depth, not
the root-cause fix — the actual streaming-thread leak path is tracked
separately. Falsy pending_started_at (legacy sidecars) treated as
'old enough' so legitimate legacy-data recovery still works. Plus
logger.warning telemetry on every legitimate repair so the next batch of
user reports tells us whether the underlying race still fires.

Closes #1625 — Local model servers (LM Studio, Ollama, llama.cpp, vLLM,
TabbyAPI, koboldcpp, textgen-webui) now keep the full HuggingFace-style
model id (e.g. 'qwen/qwen3.6-27b' instead of stripped 'qwen3.6-27b'). New
_LOCAL_SERVER_PROVIDERS set + _base_url_points_at_local_server() loopback/
RFC1918 heuristic — either signal triggers no-strip. Backward compat
preserved for OpenAI-compatible proxies on public hosts (LiteLLM at
litellm.example.com still strips openai/gpt-5.4 -> gpt-5.4). Updated the
existing #230/#433 test to reflect that #1625 supersedes the strip-on-custom
rule for loopback hosts (see api/config.py and test_model_resolver.py
docstring update). Reported by @akarichan8231 in Discord on 2026-05-04.

42 regression tests across:
  tests/test_issue1623_sse_heartbeat_alignment.py (3)
  tests/test_issue1624_repair_stale_pending_grace.py (9)
  tests/test_issue1625_local_server_model_id_preservation.py (30)

4142 -> 4184 passing. 0 regressions.
2026-05-04 16:49:43 +00:00
test
838645fd50 Stage 293: PR #1629 — profile isolation trio (closes #1611, #1612, #1614) by @nesquena-hermes — APPROVED 2026-05-04 16:21:29 +00:00
test
341b4c7abd Stage 293: PR #1627 — show Hermes Agent version in Settings (closes #1606) by @franksong2702 2026-05-04 16:20:39 +00:00
nesquena-hermes
6bc0f9c4d5 Apply Opus pre-release SHOULD-FIX + NITs (in-PR per release policy)
SHOULD-FIX #1 (renamed-root client cross-alias): drop strict-equality client
filter at static/sessions.js:1853. Server-side _profiles_match cross-aliases
'default'-tagged rows to a renamed root 'kinni'; the strict-equality client
would reject them, dropping every legacy session for renamed-root users. The
server is now solely authoritative for profile scoping.

SHOULD-FIX #2 (messaging-source dedupe ordering): _keep_latest_messaging_session_per_source
now runs AFTER the profile filter at api/routes.py:2078. Before, it ran on
the merged-cross-profile list with profile-blind keys, discarding the older
profile's row across profiles before the scope filter — leaving zero rows for
any messaging identity the active profile shared with another profile.

NIT #3: _projects_migrated flag now set only AFTER successful save_projects.
NIT #4: cleaned dead test code in test_is_root_profile_invalidation_drops_stale.
NIT #5: _create_profile_fallback's clone_from=='default' literal now routes
through _is_root_profile() for parity with the 5 other callsites.

+2 regression tests pin the SHOULD-FIX shapes:
- test_keep_latest_messaging_runs_after_profile_filter (source-string ordering)
- test_static_sessions_js_trusts_server_profile_scoping (no client re-filter)

4173 -> 4175 tests pass. 0 regressions.
2026-05-04 16:17:26 +00:00
Michael Lam
b6c695e1ab test: cover provider config cleanup path 2026-05-04 09:04:07 -07:00
nesquena-hermes
e8862632ed fix(profiles): scope sessions, projects, and root-profile resolution to active profile (#1611, #1612, #1614)
Closes #1611 — /api/sessions filters by active profile by default; ?all_profiles=1
opt-in for aggregate views; new _profiles_match() helper honours renamed-root
cross-aliasing; static/sessions.js drops the s.is_cli_session bypass; toggle-on
re-fetches with all_profiles=1 instead of slicing client-cached rows.

Closes #1612 — new _is_root_profile() central helper consults list_profiles_api()
for is_default=True matches alongside the legacy 'default' alias. Replaces five
literal-default callsites in api/profiles.py. Memoized with explicit invalidation
hooks at create + delete. Sticky active_profile file write now stores '' for
renamed root, consistent with the legacy empty==root contract.

Closes #1614 — projects carry a profile field stamped at create-time;
/api/projects filters by active profile; /api/projects/{create,rename,delete}
and /api/session/move reject ops on cross-profile projects with 404; new
_PROJECTS_MIGRATION migration in load_projects() back-tags untagged projects
from any session that uses them, fall back to 'default'; ensure_cron_project
keys lookup by (name, profile) so each profile gets its own Cron Jobs project.

31 regression tests (9+11+11) pin the renamed-root resolution, server-side
profile scoping shape, helper invariants, cross-alias matching, migration
behavior, and active-profile guards on every project mutation endpoint.
4148 tests pass.

Reporter: @stefanpieter

Co-authored-by: stefanpieter <noreply@github.com>
2026-05-04 16:03:05 +00:00
Frank Song
59efb42dcd Show Hermes Agent version in settings 2026-05-04 23:57:56 +08:00
test
06a71563de Stage 292: PR #1621 — polish forked session indicator by @franksong2702 2026-05-04 15:34:21 +00:00
test
21eb8a89bf Stage 292: PR #1598 — broadcast SSE stream events to multiple tabs (closes #1584) by @Michaelyklam 2026-05-04 15:34:17 +00:00
test
8a10532d29 Stage 292: PR #1601 — keep frontend routes under subpath mounts by @Michaelyklam 2026-05-04 15:34:08 +00:00
test
6f8424e5b7 Stage 292: PR #1622 — don't attach image on paste when clipboard has text (closes #1620) by @s905060 2026-05-04 15:33:32 +00:00
test
b6702fbeae Stage 292: PR #1602 — keep cross-source continuations separate in sidebar by @ai-ag2026 2026-05-04 15:33:32 +00:00
test
51848fb67d Stage 292: PR #1603 — preserve git remote names in update links by @ai-ag2026 2026-05-04 15:33:32 +00:00
test
165356e744 Stage 292: PR #1608 — tighten worker-side broad-except in _run_cron_tracked (closes #1578) by @franksong2702 2026-05-04 15:33:32 +00:00
test
ead91878ef Stage 292: PR #1605 — show update branches in banner labels by @ai-ag2026 2026-05-04 15:33:32 +00:00
test
e5a5720e00 Stage 292: PR #1600 — render streaming markdown on subpath mounts by @Michaelyklam 2026-05-04 15:33:32 +00:00
test
5b4ab72452 Stage 292: PR #1597 — isolate pytest Hermes config path by @Michaelyklam 2026-05-04 15:33:32 +00:00
Jash Lee
1ad0ab42e5 Fix #1620: don't attach image on paste when clipboard also has text
When the clipboard carries both text and an image (rich-text sources like
Notes, Word, Slack, browser selection attach a rendered preview alongside
the plain text), the paste handler in static/boot.js unconditionally
called e.preventDefault() and routed the image into addFiles(), silently
discarding the text payload.

Fix:
  - Detect text in the clipboard via items[].kind === 'string' &&
    (type === 'text/plain' || type === 'text/html'). When present, return
    early so the browser's default text-paste runs.
  - Tighten the image filter to kind === 'file' && type.startsWith('image/')
    so string items advertising an image MIME (e.g. text/html with an
    embedded data URI) are not misclassified as a true screenshot paste.

Pure-screenshot paste (image-only clipboard, e.g. Cmd+Shift+Ctrl+4 on macOS)
is unchanged.

Adds tests/test_1620_paste_text_with_image.py with 6 static-analysis checks
on the handler shape, matching the pattern of test_issue1095_pasted_images.py.
2026-05-04 10:48:36 -04:00
Frank Song
3f56ed7283 Polish forked session indicator 2026-05-04 21:50:40 +08:00
Frank Song
cdcd6021cc fix(cron): tighten worker-side broad-except in _run_cron_tracked (closes #1578)
Remove the try/except Exception wrapper around
cron_profile_context_for_home(...).__enter__() in _run_cron_tracked.
A silent fallback to ctx=None would leave the worker thread unpinned
against process-global HERMES_HOME, silently corrupting cross-profile
state — the same class of bug as #1573.

Add regression test to catch any future re-introduction.
2026-05-04 16:28:33 +08:00
Manfred
0b7f60a714 fix: show update branches in banner labels 2026-05-04 09:46:45 +02:00
Manfred
3c93d5a702 fix: keep cross-source continuations separate in sidebar 2026-05-04 09:30:47 +02:00
Manfred
93251e5bcb fix: preserve git remote names in update links 2026-05-04 09:30:47 +02:00
Michael Lam
e9d7d5e427 fix: keep frontend routes under subpath mounts 2026-05-04 00:06:58 -07:00
Michael Lam
032b680e26 fix: render streaming markdown on subpath mounts 2026-05-03 23:55:45 -07:00
Sanjay Santhanam
14fac05dc9 fix(streaming): use truthy-check for _pending_started_at fallback
Switch the per-turn duration fallback from `is not None` to a truthy check so
None, missing-attr, and an explicit 0 all uniformly fall back to time.time().

Without this, a 0 timestamp (e.g. via a buggy migration or manual file edit)
would yield `time.time() - 0` ≈ wall-clock-since-epoch, displaying nonsense
like 'Done in 56 years 4 months ...'. In practice pending_started_at is always
set via int(time.time()) so this is a hardening fix, not a live-bug fix.

Also drop the brittle source-string assertion in the regression test that
pinned the literal expression. The behavioural test
test_done_handler_persists_duration_on_last_assistant_message already proves
the duration field is set; pinning the source line broke twice during the
v0.50.290 release pipeline alone (Opus tightening + maintainer revert).

Fixes #1595

Signed-off-by: Sanjay Santhanam <51058514+Sanjays2402@users.noreply.github.com>
2026-05-03 23:21:19 -07:00
Michael Lam
ad46d82060 fix: isolate pytest Hermes config path 2026-05-03 22:47:55 -07:00
Michael Lam
6c5bc95b3b fix: broadcast SSE events to all tabs 2026-05-03 22:43:11 -07:00
nesquena-hermes
3369a08f37 fix(updates): use merge-base for compare URL so 'What's new?' link resolves
Closes #1579.

api/updates.py was building the GitHub compare URL from local HEAD short SHA:

    repoUrl + '/compare/' + curSha + '...' + newSha
    where curSha = `git rev-parse --short HEAD`

Whenever local HEAD diverges from upstream — unpushed work, dirty stage
branches, forks, in-flight rebases, release-time merge commits whose SHA
only lives in the maintainer's local history — the compare URL points at
a SHA github.com has never seen and returns the standard 404 page.

Reporter (@ai-ag2026) observed:
  c660c7f...86cb22e
  → 404 because c660c7f was an unpushed local commit.

The right base is `git merge-base HEAD <compare_ref>` — the most recent
commit local and upstream share. Since `git fetch` succeeded just before,
the merge-base is guaranteed to exist on the upstream GitHub repo.

Behavior matrix:
  Pure-behind clone (no local commits): merge-base == HEAD; URL unchanged.
  Behind + local-only commits (#1579):  merge-base != HEAD; URL points at
                                        public ancestor instead of local HEAD.
  merge-base failure (shallow clone):   current_sha=None; JS link guard
                                        suppresses link rather than emitting
                                        a known-broken URL.

Also hardens static/ui.js: reset the link's href and display:none on every
banner render, so a stale link from a prior render can't survive a re-render
where the new payload has current_sha=null.

Tests:
  - test_current_sha_is_merge_base_not_local_HEAD — reporter's scenario
  - test_current_sha_equals_HEAD_when_no_local_commits — backward compat
  - test_current_sha_falls_back_to_None_when_merge_base_fails — defensive
  - test_whats_new_link_resets_display_and_href_on_every_render
  - test_whats_new_link_suppressed_when_curSha_falsy
  - test_reporter_url_shape_no_longer_produces_invalid_compare_url

4094 → 4100 passing. 0 regressions.
2026-05-04 05:26:19 +00:00
Hermes Release Agent
1636ab9ef9 release: stamp v0.50.290 — 5-PR batch (#1586+#1590+#1591+#1592+#1464) — 4094→4111 tests
- #1586 (Michaelyklam): login asset SW cache exemption
- #1590 (Michaelyklam): hot-apply compact tool activity setting
- #1591 (Michaelyklam): first-turn sidebar visibility (optimistic upserts)
- #1592 (Michaelyklam): turn duration display (Done in 1m 12s) + Opus follow-up (truthy-check on _pending_started_at)
- #1464 (JKJameson, maintainer-augmented): workspace dropdown sort+search+chip-sync (rebased + ternary fix + regression test)

Maintainer-side test fixes in stage:
- tests/test_465_session_branching.py: widen compact() search window 1500→3000
- tests/test_regressions.py: anchor on api('/api/chat/start' instead of comment line

Browser API sanity: 11/11 passed. Live UX verification: vision-confirmed dropdown sort+search+empty-state on test server. Opus advisor: SHIP AS-IS.
2026-05-04 05:10:29 +00:00
Hermes Bot
47d1a29ead Stage 290: PR #1464 — workspace dropdown sort+search+chip-sync by @JKJameson (maintainer-augmented: ternary fix + regression test) 2026-05-04 04:51:43 +00:00
Hermes Bot
d15b0a2929 Stage 290: PR #1592 — turn duration display 'Done in 1m 12s' by @Michaelyklam 2026-05-04 04:51:43 +00:00
Hermes Bot
38a9878821 Stage 290: PR #1591 — first-turn sidebar visibility (optimistic upsert) by @Michaelyklam 2026-05-04 04:51:43 +00:00
Hermes Bot
84429b2298 Stage 290: PR #1590 — hot-apply compact tool activity setting by @Michaelyklam 2026-05-04 04:51:43 +00:00
Josh
4174a7a860 fix: immediate syncTopbar on chat switch + sortable searchable workspace dropdown
Co-authored-by: Josh Jameson <josh@jjameson.com>

Maintainer-augmented:
- Flip noResults ternary (visible?'none':'' instead of visible?'':'none') —
  the contributor's first-push bug rendered 'No workspaces found' alongside
  valid filtered results. Verified on contributor's own screenshot in PR.
- Add tests/test_issue1464_workspace_dropdown_filter.py to lock the
  visibility relationship (mirror-image opt/noResults ternaries) so future
  edits cannot silently re-invert.
- Rebased onto master (was 124 commits behind v0.50.275).
2026-05-04 04:51:30 +00:00
Michael Lam
3afa23ecb7 fix: clear first-turn sidebar spinner on start failure 2026-05-03 21:14:21 -07:00
Michael Lam
0eddb0580e fix: document turn duration fallback 2026-05-03 21:12:07 -07:00
Michael Lam
f3fa106cd7 feat: show agent turn duration 2026-05-03 20:20:17 -07:00
Michael Lam
9ed0639319 fix: show first-turn chats in sidebar immediately 2026-05-03 20:10:05 -07:00
Michael Lam
c9c985933f fix: hot-apply compact tool activity setting 2026-05-03 20:00:10 -07:00
Michael Lam
f0e6a9b788 fix: keep login assets out of service worker cache 2026-05-03 18:18:27 -07:00
Hermes Bot
c07999f0ce Stage 288: PR #1572 — collapse duplicate provider groups (closes #1568) by @nesquena-hermes — APPROVED 2026-05-03 22:37:43 +00:00
Hermes Bot
421f40c2cf Stage 288: PR #1571 — cron profile isolation (closes #1573) by @kowenhaoai — APPROVED + reviewer fix + post-review tightening 2026-05-03 22:37:43 +00:00
Nathan Esquenazi
556f2390d4 test(cron-profile): auto-skip cron.jobs-dependent tests when agent unavailable
Two of the three new tests in test_scheduled_jobs_profile_isolation.py
import cron.jobs (from hermes-agent) and fail with ModuleNotFoundError
in environments where hermes-agent isn't installed at ~/hermes-agent.

The contributor's path-injection trick at module load
(`AGENT_ROOT = Path(os.environ.get("HERMES_AGENT_ROOT", Path.home() / "hermes-agent"))`)
assumes the agent lives at ~/hermes-agent, which isn't always true on
maintainer/reviewer machines or in some CI configurations. The repo's
existing convention for this is conftest.py's `_AGENT_DEPENDENT_TESTS`
auto-skip, but that requires test names to be explicitly listed.

Cleaner fix: gate the two cron.jobs-importing tests with
`pytest.importorskip("cron.jobs")` so they self-skip cleanly when the
module isn't available, while leaving the third test
(`test_cron_profile_context_serializes_concurrent_access`) untouched —
it doesn't actually need cron.jobs and provides useful coverage even
without hermes-agent installed.

Verified: full suite goes from `2 failed, 4001 passed` to `4001 passed,
57 skipped` with no regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 22:36:26 +00:00
nesquena-hermes
df03055def Address review feedback: tighten profile-resolution error handling
Three small follow-ups from the review:

1. Remove the over-broad except Exception around get_active_hermes_home()
   in _handle_cron_run. The function is in-memory dict reads + one
   Path.is_dir() stat — if it raises from inside a request handler,
   api.profiles is in a state we shouldn't be making cron decisions in.
   A silent fallback to _profile_home=None re-introduces the exact
   bug #1573 fixes (worker thread runs unpinned against process-global
   HERMES_HOME). Better to 500 the request than risk silent cross-
   profile state corruption.

2. Add a thread-safety note on os.environ mutation in api/profiles.py
   explaining why _cron_env_lock is sufficient — CPython env-var
   assignment is GIL-protected at the bytecode level but the multi-step
   read-modify-write pattern (snapshot prev → assign new → restore on
   exit) is not atomic without explicit serialization. The lock makes
   the entire context-manager body run-to-completion serially, including
   any subprocess.Popen() calls inside run_job() that inherit the env.

3. New regression test (test_cron_run_does_not_silently_swallow_profile_resolution_errors)
   pinning the no-silent-fallback contract via source-level assertion.
   Catches future re-introduction of the over-broad except clause.

Co-authored-by: kowenhaoai <kowenhaoai@users.noreply.github.com>
2026-05-03 22:29:57 +00:00
nesquena-hermes
458cf38ac9 fix(picker): collapse duplicate provider groups + guard provider-id-as-model.default (closes #1568)
Reporter (Deor, Discord #report-bugs, May 03 2026 14:19 PT, relayed by
@AvidFuturist) saw the Settings → Default Model dropdown rendering the
OpenCode Go provider as TWO separate optgroups: "OpenCode Go" (the
canonical one with all 14 catalog models) and "Opencode_Go" (a phantom
group containing one self-referential entry).

Three structural causes, all in api/config.py:_build_available_models_uncached:

1. **Detection-path id leakage.** The detection block at line ~1980
   reads cfg["providers"] keys verbatim. If the user's config has
   ``providers.opencode_go.api_key`` (underscore variant) AND another
   path adds the canonical ``opencode-go`` (e.g. via active_provider),
   both end up in detected_providers and the build loop creates two
   distinct provider groups with the second labelled via the
   ``pid.title()`` fallback as ``"Opencode_Go"``.

2. **Injection-block rogue model.** The default-model injection block
   at line ~2598 puts ANY ``model.default`` string into the picker as
   a fake option. A stray ``model.default: opencode_go`` (provider id
   mistakenly used as a model id) surfaces as a phantom model
   labelled ``"Opencode GO"``.

3. **Empty-group bleed.** When a non-canonical provider id makes it
   into detected_providers but has no entry in _PROVIDER_MODELS, the
   build loop creates an optgroup with zero models — pure UI noise.

This PR addresses all three:

- **New `_canonicalise_provider_id()` helper** that folds underscores
  to hyphens, lowercases, and applies alias resolution only when the
  alias target is itself a canonical id in `_PROVIDER_DISPLAY`. The
  last constraint avoids round-tripping ``x-ai`` (canonical) through
  the alias table to ``xai`` (which the WebUI doesn't index by).

- **Detection-path canonicalisation.** The cfg["providers"] scan
  applies the helper before adding to detected_providers. Same
  treatment in the only_show_configured intersection so that mode
  doesn't accidentally exclude the canonical id when configured_providers
  only contains the underscore-variant key.

- **Post-collection dedup pass** that re-canonicalises every entry in
  detected_providers — belt-and-braces against future regressions in
  any of the ~25 ``detected_providers.add(...)`` callsites without
  auditing each one. Idempotent for already-canonical ids.

- **Provider-id guard on the model.default injection block.** When
  the injected value matches a known provider display name or alias
  (after underscore/case normalisation), skip the injection and emit
  a `logger.warning` instead. Real unknown model ids (newly released
  models, custom endpoints) still get injected — only provider-shaped
  values are rejected.

- **Empty-group filter at end of build.** Drop optgroups with zero
  models. Custom: groups (`provider_id` starts with `custom:`) are
  exempt — users may want an empty card visible as a reminder.

Tests
-----

`tests/test_issue1568_duplicate_provider_groups.py` (17 tests):

- TestCanonicaliseProviderId (8): unit tests pinning helper behaviour —
  canonical preserved, underscore folded, case folded, aliases
  resolved, x-ai not round-tripped, empty input, unknown ids
  normalised, idempotence
- TestProviderGroupDedup (4): end-to-end picker behaviour —
  underscored providers-key produces ONE group not two (Deor's case),
  uppercase providers-key collapsed, aliased keys (z-ai → zai)
  collapsed, happy path unchanged
- TestDefaultModelProviderIdGuard (3): provider id as model.default
  doesn't inject phantom + WARNING logged; alias as model.default also
  caught; legitimate unknown model IDs (forward-compat) still injected
- TestEmptyGroupFilter (2): empty optgroups dropped from picker;
  custom: providers exempted from filter

Plus one structural test fix in
`tests/test_issue604_all_providers_model_picker.py:test_cfg_providers_only_adds_known`
— widened the regex window from 500 to 1500 chars so the new
documentation comment block doesn't push `_PROVIDER_MODELS` past the
substring slice. Pre-existing brittle window pattern, not a new issue.

Verification
------------

Live on port 8789 with Deor's exact reproduction config
(`providers.opencode_go.api_key` + `model.provider: opencode-go`):

  /api/models groups: 1 (was 2)
  Browser <select> optgroups: 1 (was 2)
  Total options under "OpenCode Go": 14 (was 14 in real group + 0 in phantom group)

Five-scenario sweep all collapse to ONE provider group:

| Config shape | Pre-fix | Post-fix |
|---|---|---|
| Hyphenated provider + underscored providers-key (Deor's case) | 2 groups | 1 group ✅ |
| Hyphenated provider + UPPERCASE providers-key | 2 groups | 1 group ✅ |
| Aliased providers-key (z-ai resolved to zai) | 2 groups | 1 group ✅ |
| model.default = provider-id (orig #1568 scenario) | 15 models with phantom | 14 models, no phantom ✅ |
| Happy path (canonical-only) | 1 group | 1 group ✅ |

4070 pytest passed (was 4053 → 4070, +17 from this PR).
3 CI runs to follow on push.
QA harness 11/11 passed.
JS unaffected — pure backend fix.

Reporter: Deor (Discord #report-bugs, May 03 2026 14:19 PT)
Relayed by: @AvidFuturist
2026-05-03 22:04:58 +00:00