HERMES-GIELIE/tests
nesquena-hermes 42dd2b562d
fix: warn on provider/model mismatch, surface auth errors (#266)
* fix: warn on provider/model mismatch, surface auth errors (#266)

Fixes #266 — WebUI silently ignores provider/model selection mismatch.

The problem: selecting an OpenRouter (or Anthropic/OpenAI) model while
Hermes is configured for a different provider (e.g. local Ollama) sends
the request to the wrong endpoint, which returns a 401 Unauthorized error
with no UI indication of why.

Three-layer fix:

1. api/streaming.py — detect 401/auth errors explicitly
   Added is_auth_error detection covering '401', 'AuthenticationError',
   'authentication', 'unauthorized', 'invalid api key', and the specific
   Ollama error string 'no cookie auth credentials'. Auth errors emit
   apperror with type='auth_mismatch' and a hint pointing to 'hermes model'.

2. static/ui.js — expose active_provider and warn on selection
   - populateModelDropdown() stores data.active_provider from /api/models
     as window._activeProvider (the field was already in the response but
     the frontend never used it)
   - New _checkProviderMismatch(modelId) helper: compares the selected
     model's slash-prefix (e.g. 'openai/' from 'openai/gpt-4o') against
     the active provider. Skips the check for 'openrouter' and 'custom'
     to avoid false positives on configs that legitimately route any model.

3. static/boot.js — warn on model dropdown change
   modelSelect.onchange calls _checkProviderMismatch() and shows a toast
   when the selected model looks incompatible with the configured provider.

4. static/messages.js — distinct UI label for auth errors
   apperror handler now distinguishes type='auth_mismatch' and shows
   'Provider mismatch' as the error label instead of 'Error'.

5. static/i18n.js — provider_mismatch_warning and provider_mismatch_label
   keys added to all 5 locales (en, es, de, zh-Hans, zh-Hant).

Tests: 21 new tests in tests/test_provider_mismatch.py covering all
five change areas. 679/679 total pass (658 baseline + 21 new).

* fix: t() call args spread + use i18n label for auth mismatch

1. ui.js: _checkProviderMismatch passed [modelId, ap] as a single
   array arg to t(). Since t(key, ...args) spreads, the function
   received the array as m and undefined as p. Fixed to pass as
   separate args: t('provider_mismatch_warning', modelId, ap).

2. messages.js: 'Provider mismatch' label was hardcoded instead of
   using t('provider_mismatch_label'). Now uses the i18n key with
   fallback for when t() isn't available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Nathan Esquenazi <nesquena@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:25:18 -07:00
..
__init__.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
conftest.py feat: real-time gateway session sync (Phase 1) (#274) 2026-04-11 20:53:12 -07:00
test_approval_unblock.py fix: unskip 16 approval tests — remove stale has_pending/pop_pending imports 2026-04-10 10:10:28 -07:00
test_auth_sessions.py fix(auth): prune expired sessions on every verify to prevent memory leak (#196) 2026-04-09 18:05:23 -07:00
test_cancel_interrupt.py v0.46.0: security, Docker UID/GID, model discovery, i18n, cancel fix 2026-04-11 10:17:52 -07:00
test_gateway_sync.py feat: real-time gateway session sync (Phase 1) (#274) 2026-04-11 20:53:12 -07:00
test_mobile_layout.py v0.47.0: dialogs, session menu, /skills, mobile fixes, mobile QA suite 2026-04-11 12:19:12 -07:00
test_model_resolver.py v0.47.0: dialogs, session menu, /skills, mobile fixes, mobile QA suite 2026-04-11 12:19:12 -07:00
test_provider_mismatch.py fix: warn on provider/model mismatch, surface auth errors (#266) 2026-04-11 21:25:18 -07:00
test_regressions.py v0.47.0: dialogs, session menu, /skills, mobile fixes, mobile QA suite 2026-04-11 12:19:12 -07:00
test_security_redaction.py v0.46.0: security, Docker UID/GID, model discovery, i18n, cancel fix 2026-04-11 10:17:52 -07:00
test_spanish_locale.py feat(i18n): add Spanish locale for WebUI (#275) 2026-04-11 20:06:37 -07:00
test_sprint1.py Hermes Web UI — Sprints 11-14: multi-provider models, settings, session QoL, alerts, polish 2026-03-31 07:02:47 +00:00
test_sprint2.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint3.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint4.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint5.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint6.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint7.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint8.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint9.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint10.py Hermes WebUI v0.1.0 — initial public release 2026-03-30 20:40:19 -07:00
test_sprint11.py Hermes Web UI — Sprints 11-14: multi-provider models, settings, session QoL, alerts, polish 2026-03-31 07:02:47 +00:00
test_sprint12.py Hermes Web UI — Sprints 11-14: multi-provider models, settings, session QoL, alerts, polish 2026-03-31 07:02:47 +00:00
test_sprint13.py Hermes Web UI — Sprints 11-14: multi-provider models, settings, session QoL, alerts, polish 2026-03-31 07:02:47 +00:00
test_sprint14.py Hermes Web UI — Sprints 11-14: multi-provider models, settings, session QoL, alerts, polish 2026-03-31 07:02:47 +00:00
test_sprint15.py feat: Sprint 15 — session projects, code copy button, tool card toggle 2026-04-02 00:11:49 -07:00
test_sprint16.py v0.47.0: dialogs, session menu, /skills, mobile fixes, mobile QA suite 2026-04-11 12:19:12 -07:00
test_sprint17.py feat: Sprint 17 -- workspace breadcrumbs, slash commands, send key setting 2026-04-03 04:13:38 -07:00
test_sprint19.py fix(auth): harden password_hash handling in settings API 2026-04-03 06:21:04 -07:00
test_sprint20.py feat: pluggable i18n with English/Chinese language switcher in Settings 2026-04-08 18:57:50 -07:00
test_sprint20b.py fix: test_send_pop_in keyframe parser hit wrong @keyframes block 2026-04-03 14:23:56 +00:00
test_sprint23.py fix: test hardening for sprint 23 — handle missing cron/skills modules in test env 2026-04-03 19:16:17 -07:00
test_sprint26.py fix: settings unsaved-changes guard, add Slate theme, improve Light theme 2026-04-05 04:00:24 +00:00
test_sprint27.py fix: harden bot_name — crash guard, XSS escape, sanitization, tests 2026-04-06 15:06:16 +00:00
test_sprint28.py fix: personalities from config.yaml + ephemeral_system_prompt (#139) (#148) 2026-04-06 14:10:30 -07:00
test_sprint29.py fix: broaden session ID validator to support new hermes-agent format (#212) 2026-04-10 00:00:02 -07:00
test_sprint30.py v0.44.0: approval fix, login CSP, update diagnostics, Lucide icons 2026-04-10 10:02:28 -07:00
test_sprint31.py feat: custom endpoint fields in new profile form (fixes #170, closes #214) 2026-04-10 11:43:49 -07:00
test_sprint32.py feat: auto-install missing agent deps on startup (#215) 2026-04-10 00:42:02 -07:00
test_sprint33.py v0.47.0: dialogs, session menu, /skills, mobile fixes, mobile QA suite 2026-04-11 12:19:12 -07:00
test_tls_support.py feat: optional HTTPS/TLS support via cert and key env vars (#199) 2026-04-09 18:08:29 -07:00
test_update_checker.py v0.44.0: approval fix, login CSP, update diagnostics, Lucide icons 2026-04-10 10:02:28 -07:00