Update the tui_desc text shown for each provider in the interactive
`hermes model` / setup wizard / `/model` pickers. Pure copy refresh —
slugs, labels, PROVIDER_GROUPS folding, and all typed paths are unchanged,
so the 7 grouped families (OpenAI, xAI Grok, GitHub Copilot, Google Gemini,
Kimi / Moonshot, MiniMax, OpenCode) still fold identically.
Also aligns the auto-injected alibaba-coding-plan provider description to
the same parenthetical style.
Follow-up to the synthetic-notification DM-topic routing fix. The new
_is_telegram_dm_topic_target probed the adapter's _get_dm_topic_info via
instance-level getattr, which a MagicMock auto-creates as a truthy callable —
so any test double with a non-dm chat_type and a thread_id would be
misclassified as a DM topic lane and have the fallback routing keys injected.
Resolve the method on type(adapter) and treat only dict-shaped returns as an
operator-declared topic, mirroring the existing guard in
_rename_telegram_topic_for_session_title. Update the home-channel startup test
to declare _get_dm_topic_info on a real adapter subclass instead of patching a
MagicMock onto the instance.
Background tasks on non-local backends (SSH/Docker/Modal/Daytona/Singularity)
go through `ProcessRegistry.spawn_via_env`, which builds a hand-crafted,
shell-safe wrapper:
mkdir -p T && ( nohup bash -lc CMD > LOG 2>&1; rc=$?; ... ) & echo $! > PID && cat PID
`BaseEnvironment.execute()` unconditionally ran `_rewrite_compound_background`
on every command, including this wrapper. The rewrite (meant to defuse the
`A && B &` subshell-wait trap for user commands) turns `( ... ) & echo $!` into
`{ ( ... ) & } echo $!` — note `} echo` with no separator, which is a bash
syntax error. The wrapper then never produces a PID, the redirected output file
is never created, and the agent sees an immediate exit code -1. This breaks
*every* background launch on a non-local backend (e.g. a simple
count-and-redirect script over SSH), not just edge cases.
Fix:
- Add `rewrite_compound_background: bool = True` to `BaseEnvironment.execute()`
(and the `BaseModalExecutionEnvironment` override, which accepts and ignores
it). Default preserves existing behavior; the user foreground terminal path
still rewrites.
- `spawn_via_env` passes `rewrite_compound_background=False` so its already
shell-safe wrapper is left intact.
- Treat a wrapper that produces no PID as a failed launch (mark the session
exited with a real exit code instead of exposing a fake running session), and
don't register/checkpoint a session that never started.
Verified empirically: with the rewrite skipped, the wrapper is valid bash,
launches the process, captures the PID, and writes the log/pid/exit files; the
old rewritten form fails `bash -n` with a syntax error.
Based on #33756 by @CharZhou (extracted from a multi-feature branch; the
unrelated image_gen / docker-media changes are not included here).
Co-authored-by: CharZhou <17255546+CharZhou@users.noreply.github.com>
terminal_tool re-sent the init-time/config cwd on every command, clobbering
session-local `cd` state: the environment tracked the new directory in
`env.cwd`, but foreground/background calls forced the old cwd back. A small
`_resolve_command_cwd` resolver now applies the precedence
`workdir > live env.cwd > config/override cwd` to:
- foreground `env.execute(...)`
- background `process_registry.spawn_local(...)`
- background `process_registry.spawn_via_env(...)`
Additionally, syncing the cwd onto the live cached env when a `cwd` override is
(re-)registered. Preferring live `env.cwd` would otherwise demote the ACP
`update_cwd` override (registered via `register_task_env_overrides` on
`session/load` / `session/resume`) below an already-set `env.cwd`, silently
ignoring an editor's mid-session project-root change once any command had run.
`register_task_env_overrides` now pushes a new cwd onto the cached env so an
explicit ACP cwd change wins, while ordinary in-session `cd` tracking is
preserved.
Regression coverage:
- foreground/background commands follow live `env.cwd`
- explicit `workdir` still overrides everything
- registering a cwd override updates the live env cwd (ACP authority)
- no-op when no live env exists; non-cwd overrides leave env.cwd untouched
Based on #35510 by @Dusk1e.
Co-authored-by: Dusk1e <yusufalweshdemir@gmail.com>
In a per-user thread (thread_sessions_per_user=True), each participant
gets an isolated session key (...:{thread_id}:{user_id}). A run another
user started lives under a different key, so the caller's own /stop found
nothing and replied 'no active task to stop'.
When /stop finds no run under the caller's own key, fall back to
interrupting any running agent(s) sharing the caller's thread prefix
({chat_id}:{thread_id}), gated on _is_user_authorized. Thread-only — the
fallback returns [] for non-thread channels, and a prefix-collision guard
prevents thr1 from matching thr11.
* feat(setup): Quick Setup routes through Nous Portal (OAuth + model + messaging)
First-time quick setup now goes straight to the Nous Portal provider
instead of showing the full provider picker. Runs the device-code OAuth
login, selects a Nous model, configures the terminal backend, and offers
messaging setup — applying recommended defaults for everything else.
- Rename menu entry to 'Quick Setup (Nous Portal)'.
- _run_first_time_quick_setup now calls _model_flow_nous (handles both the
logged-out OAuth+model-select path and the logged-in curated picker),
then re-syncs config from disk to avoid the #4172 stale-overwrite.
- Terminal / defaults / messaging steps unchanged.
* feat(setup): thin out Full Setup with happy defaults
Full Setup no longer asks for every config knob — anything with an
obvious default is applied silently and stays tunable via the per-section
commands (hermes setup agent|terminal|tts, hermes auth add).
- Model section: drop the same-provider rotation pool, vision-backend
picker, and TTS provider sub-flows. Vision auto-detects from the main
provider; TTS defaults to Edge; rotation lives in hermes auth add.
- Terminal section: keep the backend picker (Local default) and any
required credentials (Modal token, SSH host/user/key, Daytona key),
but stop prompting for container image, CPU/mem/disk resources, gateway
cwd, and sudo password — all use defaults.
- Agent Settings: removed from the wizard. First installs get recommended
defaults silently; existing installs keep their tuned values.
- New defaults: max_turns 90 -> 150, session_reset both -> none.
- Tests: reconfigure tests assert agent settings are no longer prompted
on existing installs; drop 3 tests covering the deleted in-setup
rotation flow.
* fix(tui): persist gateway lifecycle breadcrumbs to crash log
A backend SIGTERM (`=== SIGTERM received ===` in tui_gateway_crash.log) is
always a parent action — `gw.kill()` (graceful-exit on a signal to Node, or an
explicit /quit) or `start()` replacing a live child. #31051 added parent-side
lifecycle breadcrumbs but left them in an in-memory CircularBuffer that dies
with the process, so SIGTERM crash reports arrive with no parent context and no
way to tell a signal-driven kill from a memory-critical `process.exit(137)`
(which closes the child's stdin → clean EOF, not SIGTERM).
Persist the death-explaining breadcrumbs (spawn / transport-exit / child-exit /
replace-live-child / kill-reason / startup-timeout) plus the graceful-exit
signal name and the memory-critical exit into the same crash log the Python
side writes, so they interleave by timestamp next to the child's panic entry —
making these recurring reports diagnosable.
Gated off under VITEST so unit tests stay hermetic.
* feat(tui): auto-recover the session when the gateway dies unexpectedly
When a still-owned gateway child dies while the TUI is alive (a crash, OOM
process.exit, or a SIGTERM/SIGHUP forwarded to it), the app currently nulls the
session and drops to an inert "gateway exited" state — the user loses a long
session and has to restart + re-run everything. That single behavior is most of
the "TUI doesn't survive heavy work" complaint, independent of what does the
killing.
The 'exit' event only reaches this handler on an *unexpected* death: a user
/quit calls process.exit before it fires, and a replaced child is identity-
skipped in GatewayClient. So on exit we now respawn the gateway and resume the
session that was live (history is persisted in SQLite) via a one-shot
recoverSidRef the next gateway.ready consults before forging a new session. The
in-flight reply is lost (it died with the process) but the session survives.
Bounded to GATEWAY_RECOVERY_LIMIT (3) attempts per GATEWAY_RECOVERY_WINDOW_MS
(60s) so a gateway that crash-loops on startup can't spawn-storm; past the
budget we fall back to the inert state.
* fix(tui): sanitize newlines + soften SIGTERM-cause claim in parentLog
Address PR review:
- recordParentLifecycle collapses embedded \r\n so a multi-line value (e.g. an
error message) stays a single breadcrumb and can't masquerade as a separate
entry or as the child's panic output sharing the crash log.
- Reword the header: a backend SIGTERM is *usually* a parent action but can come
straight from an external supervisor (s6, cgroup OOM, stray kill); the
presence/absence of a [tui-parent] line before the child's panic is precisely
what disambiguates the two.
* fix(tui): clear sid during recovery + extract/test the recovery budget
Address PR review:
- Null `sid` immediately in the gateway exit handler. While the gateway is down
(busy=false) the old sid would otherwise let sid-guarded effects (the 1.5s
session.active_list poll, queue drain) fire RPCs at a dead/respawning gateway.
recoverSidRef carries the session forward; resumeById restores sid on ready.
- Extract the respawn budget into a pure evalRecovery() (gatewayRecovery.ts) and
unit-test the bound: allows GATEWAY_RECOVERY_LIMIT within the window, blocks
past it, and prunes attempts older than the window so recovery re-arms.
* fix(tui): cap parent-log breadcrumb length (PR review)
Truncate a single persisted breadcrumb to 4096 chars (matching GatewayClient's
in-memory log-line cap) so a pathological value — e.g. a giant error string —
can't bloat the shared crash log or add noticeable blocking on the synchronous
append during a failure path. Covered by a test.
* fix(tui): keep "recovering session…" status visible during resume (PR review)
resumeById() synchronously sets status to 'resuming…' on entry, so the
recovery branch now applies its 'recovering session…' label *after* calling
resumeById — the distinct label sticks for the duration of the resume RPC
(which later flips to 'ready') instead of being immediately clobbered. Test
updated to assert the ordering.
* fix(tui): keep recovery budget alive across a startup crash-loop (PR review)
deadSid was read from getUiState().sid, which the first exit nulls — so if the
respawned gateway crash-looped before gateway.ready (resumeById never restored
sid), later exits saw null and abandoned the session after a single attempt,
defeating the bounded retry budget.
Lift the whole decision into a pure planGatewayRecovery() that falls back to the
pending recoverSidRef target when the live sid is already cleared, and unit-test
the crash-loop sequence (keeps retrying the same session up to the limit, then
falls back to inert). Supersedes evalRecovery.
* chore(tui): drop non-null assertion + clarify breadcrumb cap comment (PR review)
- Recovery branch guards on `recoverSidRef && recoverSid` so the ref write needs
no `!` assertion (avoids a future unsafe refactor).
- Reword the parentLog cap comment: it slices the value to 4096 chars and
appends a short truncation marker (so the written line is slightly longer),
rather than implying a strict 4096-byte limit.
* chore(tui): soften "absence ⇒ external signal" + "any in-flight reply" (PR review)
- parentLog header: a missing [tui-parent] line only *suggests* an external
signal (the logger is best-effort: VITEST-disabled, failed append swallowed),
not a definitive conclusion.
- Recovery notice says "any in-flight reply was lost" since the gateway can also
exit while idle.
Extended-thinking Claude models (4.6+, e.g. Opus 4.8) emit a signed `thinking`
block on assistant turns that also carry parallel `tool_use` blocks. Anthropic
signs that block against the full, original turn content.
When a parallel tool batch is interrupted before every `tool_result` returns,
`_strip_orphaned_tool_blocks` removes the unanswered `tool_use` on replay — which
mutates the turn. The latest-assistant branch of `_manage_thinking_signatures`
then replays the now-stale signed thinking block verbatim, and Anthropic rejects
the request with a non-retryable HTTP 400:
messages.N.content.M: `thinking` or `redacted_thinking` blocks in the latest
assistant message cannot be modified. These blocks must remain as they were
in the original response.
Because the poisoned turn is rebuilt from the persisted store every turn, the
gateway crash-loops with no self-recovery (a soft session reset does not clear
it). The drifting content index in the error is the changing count of stripped
`tool_use` blocks across rebuilds.
Fix: when orphan-stripping removes a `tool_use` from a turn that also holds a
thinking/redacted_thinking block, flag the turn. `_manage_thinking_signatures`
then demotes every thinking block on that latest turn to a plain text block
(preserving the reasoning text) instead of replaying a signature that can no
longer validate. An intact turn is unaffected — its signed thinking is still
replayed verbatim. The internal flag is stripped before the payload is sent.
Adds two regression tests:
- demotion when an orphaned parallel tool_use is stripped
- control: signed thinking preserved verbatim when nothing is stripped
PR #35718 added a per-slot "cumulative-resend" latch to the universal
streaming tool-call accumulator to fix DeepSeek / Baidu Qianfan (#35592).
The latch fires when a delta is a strict superset of the accumulated
buffer (len(_new) > len(_prev) and _new.startswith(_prev)) and then
REPLACES the buffer instead of appending.
That superset test is not an unambiguous cumulative signature. A normal
incremental stream can emit a single fragment that restates an already-
accumulated prefix — trivially common in large code-patch arguments with
repeated lines / indentation — which trips the latch and clobbers the
accumulated buffer, corrupting the tool call. Observed in the wild on
Anthropic Opus (the primary model) building a large patch: corrupted /
short arguments → finish_reason='length' dead-end → session killed.
A guessing heuristic that can silently clobber a tool-call buffer has no
place on the path every provider and model shares. Reverting restores the
known-good plain `+=` accumulator. The #35592 narrow provider bug should
be re-addressed provider-gated so it is structurally impossible to touch
Anthropic / OpenAI incremental streams, rather than via a heuristic on the
shared path.
Reverts ca03486b6.
The status bar read context_compressor.last_prompt_tokens directly with
an 'or 0' guard that only catches 0/None. Right after a compression the
compressor parks last_prompt_tokens at the -1 sentinel
(awaiting_real_usage_after_compression) until the next API call reports
real usage. -1 is truthy, so it sailed through and rendered as '-1/200K'
and '-1%' for that one transitional turn.
Clamp negative token/context-length values to 0 in the status-bar
snapshot so the gap reads as empty context until real usage arrives.
* feat(tools): always show Nous Tool Gateway backends, login on select
The Nous-managed Tool Gateway rows in `hermes tools` (Firecrawl, OpenAI
TTS, Browser Use, FAL image/video) were hidden unless the user was already
logged into Nous Portal with paid access. Now they are always listed.
Selecting one runs an inline Nous Portal device-code OAuth + entitlement
check — auth only, no inference-provider switch and no bulk 'enable all
tools' prompt (that stays in `hermes model`). The row only activates the
gateway once paid access is confirmed.
- _visible_providers: stop hiding managed_nous_feature rows (incl. those
also flagged requires_nous_auth); pure pre-auth UX rows still gate on login
- nous_subscription.ensure_nous_portal_access(): auth + entitlement gate
that preserves the user's active inference provider
- _configure_provider / _reconfigure_provider: run the inline gate for
managed backends; write config only when entitled
- picker marker: 'via Nous Portal (login on select)' for logged-out users
- _hidden_nous_gateway_message: now a no-op (rows are never hidden)
* docs: hermes tools is a first-class Tool Gateway entry point
The Tool Gateway docs framed `hermes setup --portal` / `hermes model` as
the activation path and only mentioned `hermes tools` for mixing in your
own keys. With the inline-login change, picking a Nous-managed backend in
`hermes tools` is a complete path on its own — it logs you into Nous
Portal on select if needed, without switching your inference provider or
prompting to enable every other tool.
- tool-gateway.md: Get started now lists three peer entry points; new
paragraph explaining login-on-select and the no-prompt fast path when
OAuth is already active
- nous-portal.md + run-hermes-with-nous-portal.md: note that managed rows
appear logged-out and trigger inline login on select
The three curses menus (curses_checklist / curses_radiolist /
curses_single_select) each hand-rolled an identical event loop: cursor
hide + color-pair init, the per-frame clear/getmaxyx/refresh cycle,
scroll-offset math, row iteration, the read_menu_key dispatch with
NAV_UP/NAV_DOWN cursor wrap, flush_stdin, and the
KeyboardInterrupt/curses-unavailable fallback. Terminal-behavior changes
(e.g. Ghostty raw-escape handling, scroll tweaks, a new key) had to be
made in three places.
Extract that boilerplate into one _run_curses_menu driver. Each public
menu now supplies small callbacks for the parts that genuinely differ:
draw_header (returns the item-list start row), draw_row (checkbox vs
radio vs bare prefix), an on_action reducer (toggle-set vs return-cursor
vs return-None + the single_select cancel-row guard), an optional
draw_footer (the checklist status bar), reserve_bottom, and the numbered
fallback. Behavior is passed as functions; the loop is the only stateful
piece — so future terminal/Ghostty work is a one-place edit.
Duplicated event-loop primitives drop 3 -> 1 (stdscr.clear, read_menu_key
dispatch, scroll math). Verified byte-identical: a render harness records
every addnstr(y, x, clamped-text, attr) call across frames plus the
return value for 6 cases (checklist, checklist+status, radiolist,
radiolist+description, single_select, single_select ESC-cancel); output
diffs clean against origin/main. Non-TTY returns the cancel value
directly (not the input()-based numbered fallback), matching the old
per-menu guard. 150 menu/setup/browse/plugins tests pass.
The setup provider->model sub-menu (and three sibling pickers) used
simple_term_menu.TerminalMenu, whose ESC and arrow-key handling was
unreliable across terminals — notably ESC failed to back out of the
model selection list on terminals that emit raw escape sequences (e.g.
Ghostty). The codebase already notes simple_term_menu 'conflicts with
/dev/tty' and causes 'ghost-duplication rendering', and a prior attempt
to migrate these (closed PR) confirmed the same root cause.
Route all four single-select pickers through the shared, already-hardened
curses_radiolist (which decodes raw CSI/SS3 escape sequences and handles
ESC consistently, fixed in #35776):
- auth.py _prompt_model_selection — model picker; the pricing column
header and the unavailable-models block are passed as the radiolist
description so they survive the curses screen clear. ESC now cancels.
- main.py _prompt_reasoning_effort_selection — reasoning-effort picker.
- main.py _model_flow_named_custom — named custom-provider model picker.
- main.py _remove_custom_provider — provider-removal picker.
simple_term_menu is no longer imported anywhere (only stale comments
referenced it; one in setup.py is corrected). The numbered-input
fallbacks are unchanged and still trigger on curses errors / non-TTY.
Tests: updated test_terminal_menu_fallbacks / test_reasoning_effort_menu
/ test_custom_provider_model_switch / test_model_provider_persistence to
drive the fallback via curses_radiolist errors instead of breaking
simple_term_menu. New test_setup_menu_curses_migration.py asserts each
picker routes through curses_radiolist, ESC cancels, and the pricing
header is preserved. Net -147/+183 (mostly the new test file; production
code shrinks by removing TerminalMenu boilerplate).
The setup wizard's provider/model pickers (curses_radiolist via
prompt_choice) bailed to the numbered "Select [1-N]" fallback the moment
a user pressed up or down. Root cause: even with keypad(True) — which
curses.wrapper sets — many terminals/terminfo entries deliver cursor keys
to getch() as raw CSI/SS3 byte sequences (e.g. 27, 91, 66 for arrow-down)
rather than the translated curses.KEY_DOWN. The menus matched only
curses.KEY_UP/KEY_DOWN and treated the leading 27 (ESC) as cancel, so
navigation dropped into the text fallback and the trailing bytes leaked
into the next input().
Add a shared read_menu_key() helper that decodes CSI/SS3 escape sequences
into normalized NAV_* actions (only a lone ESC, with no continuation byte
within a short timeout, still cancels) and consumes the tail of unhandled
sequences so stray bytes can't corrupt later input(). Route all three
curses menus (checklist, radiolist, single_select) through it.
Add regression tests covering raw CSI/SS3 arrows, translated KEY_*
constants, vim keys, lone-ESC cancel, and full consumption of unhandled
sequences (Delete/Home/End).
* feat(kanban): goal_mode cards run workers in a /goal loop
A goal_mode card wraps its dispatched worker in the Ralph-style goal
loop behind /goal: after each turn an auxiliary judge checks the
worker's response against the card title+body, and if not done the
worker keeps going in the SAME session until the judge agrees, the
worker terminates the task itself, or the turn budget runs out (which
blocks the card for human review — never a silent exit).
- kanban_db: goal_mode + goal_max_turns columns (additive migration),
Task fields, create_task params, INSERT wiring, created-event payload.
- kanban_tools: goal_mode/goal_max_turns on the kanban_create tool so
orchestrators can opt cards in when fanning out.
- kanban CLI: --goal / --goal-max-turns on 'kanban create'.
- dashboard API: goal_mode/goal_max_turns on the create endpoint
(auto-surfaced back via asdict).
- _default_spawn: sets HERMES_KANBAN_GOAL_MODE / _GOAL_MAX_TURNS only
when the card opts in.
- goals.run_kanban_goal_loop: standalone, callback-injected loop engine
(no SessionDB persistence; ephemeral worker). cli.py quiet path calls
it after the worker's first turn when the env vars are set.
- Docs: orchestrator skill + kanban feature page.
Tests: DB roundtrip + legacy migration, spawn env gating, and the loop's
continuation/completion/budget-block/finalize-nudge branches. E2E run
against a real kanban DB confirms a budget-exhausted goal worker lands
in a sticky blocked state.
* feat(kanban/dashboard): goal-mode toggle in the create form
Wires the goal_mode card setting into the dashboard UI (the plugin's
hand-written IIFE bundle, no build step):
- InlineCreate: 'goal mode' checkbox after the skills field; checking it
reveals an optional 'max turns' number input. Both reset on submit and
only post goal_mode/goal_max_turns when enabled.
- TaskDrawer: a 'Goal mode: on (max N turns)' MetaRow so a card's
goal-mode setting is visible after creation (auto-fed by asdict via the
existing _task_dict).
Live-tested through the running dashboard with a browser: created a
goal-mode card with max-turns=8, confirmed it persisted to the kanban DB
(goal_mode=1, goal_max_turns=8) and rendered back in the drawer as
'on (max 8 turns)'. No JS console errors.
Self-review follow-up on top of the salvaged perf fixes:
- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
(#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
`watchers.clear()`, but `watchers` aliased the registry's live list.
A watcher appended by a concurrent session during the `await
asyncio.sleep(0)` yield would be cleared without ever being scheduled.
Detach the batch atomically (`pending_watchers = []`) before iterating.
- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
LRU (#30523) to match feishu/codebase precedent — module-level
`_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
post-insert `move_to_end` (a fresh insert is already most-recent).
- gateway/platforms/feishu.py: drop the same redundant post-insert
`move_to_end` from the salvaged _message_text_cache LRU (#23706).
- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
authors (amathxbt #22155, ErnestHysa #32636/#32708) so the contributor
audit passes when these commits land on main.
- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
module-level limits cache between tests.
- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
_message_text_cache as a plain dict; it's now an OrderedDict, so the
fixture type had to match.
N43 — Silent plugin/bundle errors:
- Plugin command dispatch: logger.debug() -> logger.warning()
- Bundle dispatch: logger.debug() -> logger.warning()
Plugin/auth failures are no longer invisible to operators.
N42 — O(n^2) pending_watchers recovery:
- Both recovery loops (startup + per-message) used while+pop(0) which is O(n) per pop
- Replaced with enumerate() over the list + periodic asyncio.sleep(0) yield points
- Clears the list after iteration instead of per-pop
- Batch size of 100 balances throughput vs event-loop responsiveness
PAIN BEFORE:
Inside _handle_auth_error_and_retry() (a sync function that runs on the MCP
event loop thread), there was a blocking polling loop:
while time.monotonic() < deadline:
if srv.session is not None and srv._ready.is_set():
break
time.sleep(0.25) # BLOCKS THE ENTIRE EVENT LOOP
Since _handle_auth_error_and_retry is invoked from tool handlers that run ON
the MCP event loop, time.sleep(0.25) blocked ALL concurrent MCP operations
(including other tools, keepalive heartbeats, OAuth refreshes) for 250ms per
iteration. With a 15-second deadline, worst case = 60 * 250ms = 15 seconds
of fully blocked concurrency.
WHAT WAS FIXED:
Extracted the blocking poll into an async helper _await_ready() that uses
asyncio.sleep(0.25) (non-blocking), and runs it via _run_on_mcp_loop().
_run_on_mcp_loop() properly awaits the coroutine on the event loop without
blocking the caller's thread. Added exception handling around the poll so
stuck reconnects still fall through to the error path.
The sync _handle_auth_error_and_retry now:
1. Fires reconnect signal (threadsafe)
2. Calls _run_on_mcp_loop(_await_ready(), timeout=15) — non-blocking
3. Returns; the event loop handles the polling
File: tools/mcp_tool.py
Lines: _handle_auth_error_and_retry() (~1886-1920)
Found by: exhaustive multi-pass audit (10 strategies, 1901 files, 913K lines)
The _guid_cache dict grows without bound as new contacts/groups are
resolved. In a long-running gateway instance with many unique targets
this becomes a slow memory leak.
Replace the plain dict with an OrderedDict capped at 500 entries.
When the cap is exceeded the oldest (least-recently-used) entries are
evicted.
_message_text_cache was a plain dict with no size limit. Every unique
message_id whose text was fetched (for reply-context lookups) stayed in
memory permanently, causing unbounded growth in long-running deployments
with active group chats.
Replace with an OrderedDict and evict the least-recently-used entry
whenever the cache exceeds _FEISHU_MESSAGE_TEXT_CACHE_SIZE (512). Cache
hits call move_to_end() to refresh LRU order. Mirrors the identical
pattern already used by _pending_processing_reactions in the same class.
Lower the model_catalog disk-cache TTL from 24h to 1h so freshly
published model-catalog.json deploys reach the picker within an hour
instead of up to a day. The picker now refetches on the next
`hermes model` / `/model` once the cache is older than 1h; younger
than 1h still serves the cache (no network hit), and network failures
still fall back to the stale copy.
- DEFAULT_TTL_HOURS 24 -> 1 (model_catalog.py)
- DEFAULT_CONFIG model_catalog.ttl_hours 24 -> 1, _config_version 24 -> 25
- migration v24->25 rewrites a stale ttl_hours:24 to 1, preserving any
custom value the user set
E2E: verified >1h refetches / <1h skips, and migration rewrites 24->1
while preserving a custom 6.
DeepSeek / Baidu Qianfan stream tool-call arguments in cumulative mode:
each chunk resends the full arguments-so-far instead of the new fragment.
The stream accumulator blindly concatenated arg deltas with +=, turning
that into '{...}{...}{...}', which failed json.loads and got nuked to '{}'
— a silently corrupted tool call (#35592). Worse on multi-param tools
(search_files, session_search, memory replace) because longer args take
more chunks, giving more resend opportunities.
- Per-slot cumulative latch in the stream accumulator: a delta that is a
strict superset of the accumulated buffer marks the slot cumulative and
replaces (not appends); exact duplicates are dropped only after latching.
Incremental fragments are untouched (default += path).
- Backstop _collapse_repeated_json_arguments() in the repair pipeline
collapses pure identical-resend buffers (K exact repeats of a valid-JSON
unit) for providers that resend the complete object from chunk 1. Only
reached after json.loads already failed, so compliant single objects are
never touched.
Not a gateway or DeepSeek-model bug — any OpenAI-wire provider in
cumulative streaming mode is affected.
Resize vision tool-result images down to a 4 MB embed cap at load time,
not just at the 20 MB hard ceiling. A 5-20 MB image previously sailed
through the native fast path and got baked into conversation history,
where Anthropic's 5 MB per-image base64 limit rejected every subsequent
turn with a 400 — and because history is immutable, retries could never
clear it, permanently wedging the session.
Also harden the reactive shrink-recovery: it now returns False (don't
retry) when any oversized image part can't be brought under target, so
the single retry isn't burned re-sending a payload that will fail
identically. Previously it returned True after shrinking *any* part,
even when the actual oversized culprit survived.
SSH sessions hard-failed voice mode on the presence of SSH_* env vars
alone, even when a PulseAudio/PipeWire server is running on the host and
audio works (ffplay/aplay/pw-play -> pulseaudio). Probe the default
sound-server sockets (PULSE_SERVER unix path, PULSE_RUNTIME_PATH/native,
$XDG_RUNTIME_DIR/{pulse/native,pipewire-0}) and actually connect() so a
stale socket doesn't count; downgrade the SSH branch to a notice when
audio is reachable. Mirrors the existing Docker/WSL forwarding handling.
Fixes#35622
The all/* wildcard expands to every registered toolset, but a handful of
tools have an additional check_fn gate on top of toolset membership and
are intentionally NOT turned on by all/* alone:
- Capability-gated tools (browser, computer_use, code_execution, Feishu,
Home Assistant, cronjob) require their backend/credential prerequisite.
- The kanban toolset is workflow-gated and deliberately opt-in. Kanban
tools mutate shared board state, so they stay off by default even under
all/* — you must list 'kanban' by name (or be a dispatcher-spawned
worker with HERMES_KANBAN_TASK set).
This was the expectations gap behind #35581 — the docs previously said
all/* expands to 'every registered toolset' without noting the carve-out.
Closes#35581.
Follow-up to the salvaged #30728:
- Gateway already exports _HERMES_GATEWAY=1 at startup (gateway/run.py) and
cli.py already keys off it. Drop the redundant new HERMES_IN_GATEWAY var;
guard stop/restart on _HERMES_GATEWAY instead. One marker for one fact.
- Drop the greedy \bgateway.*restart alternation from the cron lifecycle
filter — it false-positived on legit prompts that merely mention an
unrelated gateway + a restart (API/payment gateway monitoring). The
specific 'hermes gateway (restart|stop|start)' pattern already covers the
real command.
- Rework the two negative guard tests to sentinel the first downstream call
so they don't drive real signal delivery (tripped the live-system guard).
- Add false-positive regression cases to test_safe_commands.
Three defenses against SIGTERM-respawn loops when agent schedules its
own gateway restart under launchd/systemd KeepAlive:
1. HERMES_IN_GATEWAY env var: gateway sets it at startup; stop/restart
subcommands refuse to run when set (exit 1 with clear message).
2. Cron create payload filter: regex pre-flight rejects prompts/scripts
containing hermes gateway restart/stop, launchctl kickstart/unload,
systemctl restart/stop, and pkill patterns.
3. 30 new tests: pattern matching (14), cron block (5), gateway guard (4),
safe command negatives (7).
The per-turn file-mutation verifier footer rendered failed-write paths as
bare absolute paths in the user-facing response. The gateway's
extract_local_files() scans response text for bare paths ending in a
deliverable extension (.yaml/.json/etc.), validates os.path.isfile(), and
auto-attaches matches as native uploads — so a denied write to
~/.hermes/config.yaml surfaced the path in the footer and got the
credential file silently uploaded to the messaging channel.
The gateway denylist (validate_media_delivery_path) already blocks the
config.yaml case after #35634. This is defense-in-depth at the source:
backtick-wrap every path the footer emits — both the bullet path and any
path echoed inside the tool's error preview (the protected-file denial
message embeds the path in single quotes, which do NOT block the
extractor regex). extract_local_files skips paths inside inline-code
spans, so wrapping defeats auto-attachment for ANY protected file while
keeping the path human-readable.
- run_agent.py: _format_file_mutation_failure_footer wraps bullet paths;
new _neutralize_footer_paths backticks any remaining bare path (covers
the preview echo). staticmethod -> classmethod (caller unaffected).
- tests: backtick-wrap assertion + end-to-end extract_local_files leak test.
When PTB's general httpx pool is exhausted, it converts httpx.PoolTimeout
into telegram.error.TimedOut whose message states the request was *not*
sent to Telegram. The send retry loop treated all non-connect TimedOut as
non-retryable, so a pool timeout raised immediately, skipped all 3 retry
attempts, and was returned as retryable=False -- silently dropping the
message (agent responses, cron reports, etc.).
A pool timeout means the request never left the process, making it the
safest case to retry. Add _looks_like_pool_timeout() and treat it like a
connect timeout in both the in-loop retry decision and the outer retryable
determination, so pool timeouts flow through the existing backoff loop and
stay retryable on exhaustion.
Reported-by: q3874758 (#35610)
* feat(models): add deepseek-v4-flash to OpenRouter + Nous curated lists
deepseek/deepseek-v4-flash was already in the native deepseek provider
catalog but missing from the curated OpenRouter and Nous Portal picker
lists. Added it to both and regenerated the model-catalog.json manifest
(drift guard requires same-PR regeneration).
* refactor(models): trim redundant variants, group curated lists by maker
Remove claude-opus-4.7/4.6, gpt-5.4-nano, gpt-5.3-codex,
gemini-3-pro-image-preview, gemini-3.1-flash-lite-preview, grok-4.20,
and the older gemini-3-pro-preview (Nous). Reorder both OPENROUTER_MODELS
and _PROVIDER_MODELS[nous] into contiguous per-maker blocks with comment
headers. Regenerated model-catalog.json (openrouter 27, nous 20).
* feat(models): add gemini-3-pro-preview to OpenRouter + Nous curated lists
Adds google/gemini-3-pro-preview to both curated pickers (new on
OpenRouter, restored on Nous). Regenerated model-catalog.json
(openrouter 28, nous 21).
* test(models): use claude-opus-4.8 in OpenRouter fetch fixtures
The two TestFetchOpenRouterModels tests mocked a live OpenRouter
response with claude-opus-4.6 and relied on it surviving the curated-list
filter. Since 4.6 was removed from OPENROUTER_MODELS, those models got
filtered out and the recommended tag shifted. Swap the fixture to
claude-opus-4.8 (still curated, still first in the Anthropic block).
Rewrite TestDiscordMentions as negative assertions (mentions survive the
redactor) and clean up the orphaned comment + dangling whitespace left by
removing _DISCORD_MENTION_RE. Follow-up to the salvaged #32259 fix for #35611.
The 'hermes update' config-migration prompt printed only counts ('1 new
config option available') then asked 'configure them now?' without ever
saying what the options were. Users said no because they couldn't tell what
they were agreeing to. For pure config-format version bumps (no new
env/config keys) it still asked the question, where saying yes just bumped
the version and looked like a no-op.
- List each new env var / config key by name + description before prompting
(cap at 8, then '… and N more'). The data was already available; we just
threw it away and printed a count.
- Pure version bump (no new options): apply the format migration
non-interactively and print what happened, instead of asking a misleading
yes/no.
Reported by ScottFive and Tt2021.
Some hosts (notably WSL) report a junk window size such as 131072 columns
by 1 row. Both the Ink fork and our components only guard against
0/null/undefined/NaN (stdout.columns || 80), so a positive-but-absurd
width sails through into createScreen(width*height), allocating tens to
hundreds of MB per frame and tripping the TUI memory monitor's hard exit.
Add clampStdoutDimensions(), installed in entry.tsx before ink.render: it
patches process.stdout.columns/rows with clamping getters (cols 1-2000,
rows 1-1000; out-of-range -> 80x24). One install point fixes the renderer,
its resize handler, and every component read. Live resizes still propagate
through the original descriptor, just clamped.
* fix(tui): swallow degraded mouse-burst noise so a stalled loop can't lock the composer
When the Node event loop blocks during a heavy render/tool-call burst, stdin
stops being drained. Mode-1003 any-motion mouse reports pile up in the kernel
buffer, get partially read, and arrive as text with the `\x1b[<` prefix AND
coordinate digits chewed off across many partial reads. The existing fragment
recovery (SGR_MOUSE_FRAGMENT_RE) only handles clean `button;col;row[Mm]`
triples, so the degraded shards leak into the composer as typed text — the user
can no longer type or exit until the stall clears.
Captured leak (Windows Terminal, during tool calls):
M6M35;220;56M6M35;218;56M169;48M;157;47M;44M20;43M79;40M78;40M0M7M35;49;41M
48;41M;47;40M9;15;32M[I;31M5;211;26M35;211;25M7M;220;1MM0M09;25M24M23M3;22M
M18M99;26M32MM38M63;44M47MM1;51M M4M54M
Add two recovery layers in parseTextWithSgrMouseFragments / the text-token path:
- MOUSE_BURST_NOISE_RE: whole-text fast path. If a text token is drawn only
from the mouse-leak alphabet (`[ ] < ; I M m`, digits, spaces) AND carries
the structural signature of mouse coordinates (>=3 M/m terminators, a digit,
and a `;`), swallow it wholesale.
- MOUSE_BURST_RESIDUE_RE: swallows pure-noise residue in the gaps between and
after recovered fragments, so a partially-recovered burst doesn't trail a
chewed-up tail into the prompt.
All three constraints together preserve real prose: `Mmm MMM mmm yummy` has no
digit/`;`, `see 1;2;3M for details` has disqualifying letters, and
`1234;56;78M9;10;11M` has only two terminators — none are swallowed.
This is defense-in-depth: it stops the leak/lockout regardless of what blocks
the loop. The underlying event-loop stall during streaming is a separate,
still-open issue that needs live-turn instrumentation to root-cause.
* fix(tui): check mouse-burst noise before fragment recovery; drop test cast
Copilot review on #35512:
- MOUSE_BURST_NOISE_RE was only evaluated when parseTextWithSgrMouseFragments
returned null. A noise blob that contains any intact `<b;c;r M` fragment makes
fragment recovery return non-null, so the whole-text swallow never fired and
the code emitted a pile of recovered mouse events instead of dropping the blob
wholesale (contradicting the comment, and doing extra work mid-stall). Move the
noise check ahead of fragment recovery so pure-noise tokens are dropped early.
Add a regression test for a noise blob carrying intact fragments.
- Drop the unnecessary `(e as { isPasted?: boolean })` cast in the test;
discriminated-union narrowing on `e.kind === 'key'` exposes isPasted directly.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
The installer's ensure_fts5() handled a no-FTS5 Python by running
'uv python install --reinstall', but WHICH Python builds a uv can
install is baked into the uv binary's download manifest. A stale uv
(e.g. 'pip install uv==0.7.20', which predates python-build-standalone
#694) only knows about pre-FTS5 builds, so --reinstall just pulls the
same FTS5-less interpreter — a no-op for FTS5. Result: 'Could not obtain
an FTS5-capable Python' and a broken session search even on the
supported installer path.
ensure_fts5() now escalates uv itself: reinstall with current uv ->
'uv self update' + reinstall (stale standalone uv) -> install a fresh
standalone uv into a temp dir and reinstall with that (externally-managed
uv that can't self-update, the reported case). Pythons live in uv's
shared store, so the fresh uv's --reinstall overwrites the stale
interpreter in place and the installer's later 'uv python find' resolves
to the FTS5-capable build.
Verified against the reporter's exact repro (ubuntu:24.04 +
pip install uv==0.7.20): Python 3.11.13 (no FTS5) -> 3.11.15 (FTS5).
Defense-in-depth on top of the EphemeralReply gate: even if a config.yaml
path reaches response text via some other path, it can never be delivered
as a native attachment. Matches existing protection for .env, auth.json,
and credentials/.
Co-authored-by: JezzaHehn <jezzahehn@gmail.com>
The compact "<n>|content" gutter from #35368 is now the sole behavior.
Removes the HERMES_READ_GUTTER=padded escape hatch and its env lookup —
no legacy fixed-width path to maintain. Padding was pure token overhead
(~48% more tokens than bare content, ~16% more than compact) with no
measured accuracy gain in the original A/B.
- file_operations.py: drop env lookup + os import; gutter always f"{i}|{line}"
- tests: drop the padded env-override test; compact assertions retained
Over SSH the OSC 11 background-color query round-trip routinely exceeds
the 100ms read budget, so _query_osc11_background() gives up and the late
reply lands after prompt_toolkit has grabbed the tty. prompt_toolkit then
injects the OSC payload as typed text and reads its BEL terminator
(\x07 = Ctrl+G) as a keystroke — Ctrl+G is the open-external-editor
binding, dropping the user into vi with garbage and no obvious way out.
- Skip the OSC 11 probe on remote sessions (SSH_CONNECTION/CLIENT/TTY);
fall back to COLORFGBG / env hints / the dark default.
- Restore the tty with TCSAFLUSH instead of TCSANOW so any partial/late
reply is scrubbed from the input buffer before pt reads it.
Relative paths in write_file/patch could resolve against the agent PROCESS cwd
instead of the terminal's working directory. In a git-worktree session with a
stale TERMINAL_CWD='.' (a relative base), early edits silently landed in the
MAIN checkout, verified there, and reported success — while the agent inspected
the worktree and saw nothing, misreading it as the patch tool no-op'ing.
- _resolve_base_dir(): resolution base is now ALWAYS absolute. A relative
TERMINAL_CWD is anchored to the process cwd once, deterministically, instead
of being left to resolve()-time cwd. Live terminal cwd stays authoritative.
- write_file/patch pass the resolved absolute path to the shell FileOps layer
so the tool layer and shell layer can't disagree about which file is edited.
- Responses now report the absolute resolved_path and files_modified, so a
wrong-cwd mismatch is visible on the first call.
- _path_resolution_warning(): emits a _warning when a relative path resolves
OUTSIDE the live terminal cwd (e.g. a worktree session writing into main).
Validation: 11 new unit tests + 43 live E2E assertions (worktree routing,
mid-session cd, V4A patches, divergence warning, absolute paths, consecutive
patches); 466 existing file/path/terminal tests green.
