fix(vision): detect vision-capable custom providers via ProviderProfile flag

_supports_media_in_tool_results() had a hardcoded provider allowlist
that missed custom providers and newer vision-capable providers like
xiaomi. Added ProviderProfile.supports_vision flag and made the
function check:

1. Registered provider profile (supports_vision flag)
2. Model capabilities from models.dev catalog (supports_vision)
3. Existing hardcoded allowlist (unchanged)

This fixes HTTP 400 "text is not set" errors when vision-capable
custom providers receive text-only tool results instead of
multipart image content.

Related: #25594
This commit is contained in:
Kewe63
2026-05-15 16:56:05 +03:00
committed by Teknium
parent 4a4b9bd2dc
commit f736d2be86
3 changed files with 34 additions and 1 deletions

View File

@ -56,6 +56,15 @@ class ProviderProfile:
auth_type: str = "api_key" # api_key|oauth_device_code|oauth_external|copilot|aws_sdk
supports_health_check: bool = True # False → doctor skips /models probe for this provider
# ── Vision support ────────────────────────────────────────
# True when the provider's API accepts image content inside
# tool-result messages natively. Set on providers that expose
# multimodal models via tool results (Anthropic Messages API,
# OpenAI Chat Completions, Gemini, Xiaomi, MiniMax, etc.).
# Falls back to model-catalog lookup when False and the provider
# has no registered profile.
supports_vision: bool = False
# ── Model catalog ─────────────────────────────────────────
# fallback_models: curated list shown in /model picker when live fetch fails.
# Only agentic models that support tool calling should appear here.