A stream that drops mid-response after tokens are delivered (peer-closed
connection, stale-stream reconnect) is converted into a synthetic
finish_reason="length" stub. The conversation loop treated that network
stall as a max-output-tokens truncation: when the dropped content was a
tool call it retried exactly once, then hard-failed with "Response
truncated due to output length limit" — even on large-output models that
never hit any cap (e.g. Opus).
- Tool-call truncation now retries up to 3 times (was 1) with a
progressive max_tokens boost, and is stub-aware: a PARTIAL_STREAM_STUB_ID
stall prints "Stream interrupted mid tool-call — retrying (n/3)" instead
of the false "model hit max output tokens", and the give-up message
distinguishes a network drop from a real truncation.
- Length-continuation retries preserve the original request's output cap
as a floor, so a high provider/model default isn't silently downshifted
to 8K/12K on retry.
- Added _requested_output_cap_from_api_kwargs() helper.
Tests: stub-stall mid-tool-call recovery within 3 retries; continuation
preserves a large provider-default output cap.
Fixes#26425. Salvages the substance of #26427 (cap floor) and #9525
(retry bump), adapted to the post-refactor conversation_loop.py which
handles all three api_modes uniformly.
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: ygd58 <ygd58@users.noreply.github.com>
023149f665