The suite failed 6 tests (test_sprint12 default-model, test_issue1538 Nous
catalog, test_issue1567 picker capacity/symmetry) whenever it ran on a box that
exports HERMES_MODEL (e.g. inside a live Hermes agent session where HERMES_MODEL
is the agent's runtime model). Root cause: get_effective_default_model() in
api/config.py treats HERMES_MODEL / OPENAI_MODEL / LLM_MODEL as the
highest-priority default-model source — above the isolated test config — so a
runner-exported value overrode the sandbox config and polluted the model
picker/catalog (e.g. injecting "opus-4.8" and breaking the @nous: prefix
invariant / default-model readback).
Fix: scrub the three model-selection env vars at conftest module level (before
any product module imports, covering in-process tests) and again in the
out-of-process test_server subprocess env, and add them to the per-test
_scrub_provider_env helpers in the two Nous test modules. The isolated config /
HERMES_WEBUI_DEFAULT_MODEL is now authoritative regardless of the runner env.
Verified: with HERMES_MODEL=opus-4.8 set, the full suite goes from 6 failed to
0 failed (13550 passed); non-vacuity confirmed (disabling the scrub reproduces
the failures).
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Both fail ONLY in the full suite, not in isolation — they've been noise on every
release gate. Root-caused and fixed at the source (no retries/skips):
1. test_issue1567 nous_picker (test_hermes_cli_unavailable_falls_back_to_static_4):
NOT actually a flake — a real assertion that silently broke as the curated list
grew. It asserted every curated @nous: id (now 28) appears in the picker group's
visible 'models', but when the list exceeds _NOUS_FEATURED_THRESHOLD (25) the
dropdown intentionally shows only the featured subset and stashes the rest under
'extra_models'. Fixed to assert the invariant that matters ('no curated entry is
DROPPED') against the visible+extra_models union, so it tracks the real behavior
instead of drifting to failure every time the catalog grows.
2. test_issue4536 service_tier (test_main_service_tier_roundtrip_via_auxiliary_endpoint):
Classic test-isolation leak. get_auxiliary_models() reads the module-global
config.cfg after reload_config(); if an EARLIER full-suite test rebinds
config.cfg via monkeypatch and leaves 'cfg is not _cfg_cache',
_cfg_has_in_memory_overrides() stays True and get_auxiliary_models() reads that
stale dict instead of the freshly-reloaded temp config. Added an autouse fixture
that re-aliases cfg to _cfg_cache and clears the override fingerprint before AND
after each test in the class. PROVEN: reproduced the failure deterministically
with a leak-probe test, confirmed the fixture neutralizes it (fixture off = FAIL,
fixture on = PASS).
Converged both bounce items: enrich-existing-only (continue on webui_list-is-None,
no new providers -> routing side-effect gone, routes.py no longer touched) + respects
per-provider ID prefix convention (@nous:). Co-authored-by: kaishi00 <kaishi00@users.noreply.github.com>
* stage-3712: model picker show-all overflow + fix per-row count leak (#3691)
Self-rebased rodboev's #3712 onto master. FIX (Nathan-caught): the per-row
provider chip stripped the '(N of M)' overflow count — that count belongs on the
group heading only, and was reading as nonsense post-expand (row 30 showing
'15 of 30'). Updated test to assert heading-carries-count + rows-do-not.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
* stage-3712: + dedupe overflow expand (Codex SILENT) — reuse existing injected option
Codex caught: select hidden overflow model from search → reopen → Show all could
duplicate the row (_appendOverflowOptionsToGroup blindly appended every extra
model without checking the select for an already-injected option). Now moves/reuses
an existing same-value option instead of re-creating it.
* stage-3712: + fix expanded-heading double-count (Opus) — strip decoration, count rendered rows
Opus caught: post-expand the heading showed 'Nous (2 of 4) (4)' (appended its own
count onto the already-decorated backend label) and used modelCount (incl hoisted-
configured) vs rendered rows. Now strips the '(N of M)' suffix when expanded +
counts groupRows.length. Test asserts no double count on out[expanded].
* stage-3712: + include extra_models in Settings/Cron/Profile/Aux native selectors (Codex CORE)
Codex caught: the server overflow split moves models>15 into extra_models for ALL
providers, but the 4 native <select> builders in panels.js (settingsModel:6690,
cronFormModel:1112, profileFormModel:5784, _auxProviders:8287) read g.models only
— silently dropping the overflow tail for large providers. Now concat
models+extra_models at all 4 sites. Test guard added.
* Release PG: model picker show-all overflow for large provider catalogs (#3691)
Self-rebased rodboev's #3712 onto v0.51.445; full browser-tested (desktop+mobile,
before/after) + Nathan screenshot-approved. 4 gate-caught issues fixed inline:
per-row count leak (Nathan), overflow-expand dedup (Codex), heading double-count
(Opus), and native Settings/Cron/Profile/Aux selectors dropping the overflow tail
(Codex CORE). Codex SAFE + Opus SHIP + 41 picker tests green.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
---------
Co-authored-by: nesquena-hermes <agent@nesquena-hermes>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Two related dropdown bugs in one PR — same root shape (model-picker
endpoints disagreeing about which Nous Portal models exist) plus the
preemptive UX guard against the picker becoming unusable on large-tier
Nous accounts.
#1567 — Endpoint disagreement
=============================
Reporter (Deor, Discord, May 03 2026) saw Settings → Providers card
showing "Nous Portal — 396 models · OAuth" while the in-conversation
picker dropdown listed only the four hardcoded curated entries.
Two structural causes:
1. ``api/providers.py:get_providers`` iterates ALL OAuth providers
regardless of authentication state and unconditionally live-fetches
the catalog.
2. ``api/config.py:_build_available_models_uncached`` only iterates
providers in ``detected_providers``, gated on
``hermes_cli.models.list_available_providers().authenticated``.
That flag can disagree with ``get_auth_status(<id>).logged_in`` on
some hermes_cli versions.
When the disagreement happens for Nous, the picker silently falls
through to the curated 4-entry static list while the providers card
keeps showing the live catalog — exactly the asymmetry users report.
Plus: the Nous live-fetch branch in `_build_available_models_uncached`
fell back to the same curated 4-entry list when `provider_model_ids`
returned an empty list (transient failure / OAuth refresh in flight),
which doubles down on the disagreement instead of healing it.
UX cap (the design concern Nathan flagged on triage)
====================================================
Even with the disagreement fixed, dumping a 397-model catalog into a
flat dropdown is unusable. We trim the visible picker to a curated
~15-entry featured set when the catalog exceeds 25 models, and surface
the rest under a new ``extra_models`` field so:
- ``/model`` slash autocomplete (commands.js) covers the full catalog
- ``_dynamicModelLabels`` (ui.js) hydrates from both lists, so a model
selected from outside the featured slice still gets a proper label
- The optgroup label gets ``" (15 of 397)"`` appended so the user
understands the dropdown is intentionally trimmed, not broken
- The providers card surfaces ``models_total`` separately so the
header still reads "397 models · OAuth"
- A small "+N more" disclosure pill appears at the end of the rendered
pill list (only fires for non-OAuth providers — OAuth cards never
render pills) with a tooltip pointing at the slash command
Featured selection rules
------------------------
Deterministic; same algorithm runs in both `/api/models` and
`/api/models/live` so background enrichment doesn't undo the trim:
1. Always include the user's currently-selected model (sticky — no
orphan IDs in the dropdown after a refresh)
2. Always include every entry from the curated static
``_PROVIDER_MODELS["nous"]`` list whose id maps onto a live id
3. Top up to 15 by walking ``_NOUS_VENDOR_PRIORITY`` round-robin
(one model per vendor each pass) so no vendor monopolises the slots
Changes by file
===============
api/config.py
- New `_format_nous_label` neighbour: `_NOUS_FEATURED_THRESHOLD = 25`,
`_NOUS_FEATURED_TARGET = 15`, `_NOUS_VENDOR_PRIORITY` tuple,
`_build_nous_featured_set()` helper (~80 LOC)
- `_build_available_models_uncached` Nous branch:
- Apply featured-set cap with sticky-selection signal
- Return `extra_models` alongside `models` for the catalog tail
- Decorate optgroup label with truncation count
- Drop stale-4 fallback when authenticated but live-fetch empty
(omit the group entirely; truth lives in the providers card and
the next cache rebuild will heal it)
- Keep stale-4 fallback when hermes_cli is unavailable (test envs,
package mismatches) — that's a different failure mode
- Detection symmetry: explicit `get_auth_status("nous").logged_in`
check after the existing `list_available_providers()` loop, so the
picker matches the providers card on hermes_cli versions where the
two signals disagree
api/providers.py:get_providers
- Apply same featured-set cap so card body doesn't render 397 pills
- Add `models_total` field reporting full catalog size (used by
frontend for the "N models · OAuth" header text)
api/routes.py:_handle_live_models
- Apply same featured-set cap for `/api/models/live` so background
enrichment via `_fetchLiveModels()` doesn't undo the dropdown trim
- Use sticky-selection from `cfg["model"]["model"]` matching the main
endpoint's logic
static/ui.js:populateModelDropdown
- Hydrate `_dynamicModelLabels` from `g.extra_models` so a selection
outside the visible dropdown still renders with its proper label
static/commands.js:_loadSlashModelSubArgs
- Iterate `group.extra_models` so `/model` autocomplete covers the
full catalog (not just the trimmed featured slice)
static/panels.js:_buildProviderCard
- Header count uses `p.models_total` (full catalog size) instead of
`p.models.length` (trimmed slice)
- Render trailing "+N more" disclosure pill when `models.length <
models_total` with a tooltip pointing at the slash command
static/style.css
- New `.provider-card-model-tag-more` rule (italic, dashed border,
cursor:help, no select) — visually distinct from real model pills
Tests
=====
`tests/test_issue1567_nous_picker_capacity_and_symmetry.py` (20 tests):
- TestBuildNousFeaturedSet (8): unit tests on the helper —
small-catalog no-op, large-catalog cap to target, disjoint+complete
invariants, priority-vendor round-robin guarantee, sticky selection
with and without `@nous:` prefix, curated-flagship preservation,
empty-catalog handling, determinism
- TestApiModelsLargeCatalog (2): /api/models cap behavior end-to-end
on a synthetic 397-model catalog vs a 20-model catalog
- TestNousDetectionSymmetry (2): picker includes Nous when
`get_auth_status` agrees but `list_available_providers` disagrees;
picker omits Nous when both disagree
- TestNousLiveFetchEmpty (2): authenticated + empty-fetch omits group;
hermes_cli unavailable still falls back to static-4
- TestProvidersCardPickerSymmetry (1): both endpoints agree on
exactly the same featured-set IDs + total catalog count
- TestFrontendExtrasContract (4): static-source assertions pinning
the JS contract for `extra_models`, `models_total`, and the "+N more"
disclosure
Verified live on port 8789 (30-model catalog):
- /api/models Nous group: provider="Nous Portal (15 of 30)", 15 models,
15 extra_models
- /api/models/live?provider=nous: 15 entries (matches main path)
- /api/providers Nous card: models_total=30, models=15
- Browser dropdown after backfill: 15 options, 30 entries in
_dynamicModelLabels
- Sticky selection: Claude Opus 4.7 (the active model) in the featured
slice as expected
4073 pytest passed (was 4053 → 4073, +20 from this PR).
3 CI test runs (3.11/3.12/3.13) green.
QA harness 11/11 passed.
Reporter: Deor (Discord #report-bugs, May 03 2026 14:15 PT)
Relayed by: AvidFuturist
nesquena-hermes