Skip to content

fix(openai): gate native tools by model support - #613

Open
solvemproblr wants to merge 2 commits into
mainfrom
asa/fix-openai-native-tool-model-gating
Open

fix(openai): gate native tools by model support#613
solvemproblr wants to merge 2 commits into
mainfrom
asa/fix-openai-native-tool-model-gating

Conversation

@solvemproblr

@solvemproblr solvemproblr commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Issue

The OpenAI Responses agent registered native shell and computer tools whenever an environment exposed the corresponding SSH or RFB capability, regardless of whether the selected model supports those provider-native tool types. In production, a rollout using gpt-4o-mini with a shell-capable environment failed before inference with Tool 'shell' is not supported with gpt-4o-mini.

The SDK already has the right ownership boundary: AgentToolSpec.supported_models, default_spec(model), and the generic tool assembly path that skips a tool when default_spec returns None. The model metadata and conditional return had been removed from both OpenAI native tools.

Solution

  • Restore explicit supported-model patterns for OpenAI native shell and computer tools.
  • Preserve the last known supported GPT-5.4 and GPT-5.5 families and add the current GPT-5.6 family.
  • Return None for unsupported models so the shared SDK tool builder omits the incompatible provider parameter before request construction.
  • Warn when an environment exposes a matching capability but the selected model cannot register its provider tool; supported models and environments without that capability remain quiet.
  • Add lifecycle-level regression coverage proving gpt-4o-mini receives neither native tool while supported base, variant, and versioned GPT-5 model IDs retain them.

Impact

Rollouts no longer fail merely because an environment exposes a capability that the selected OpenAI model cannot consume. The rollout continues without that capability and emits an operator-visible warning explaining the omission. Capability/model compatibility remains SDK-owned and applies to every caller of the OpenAI agent; no rollout-specific guard or model remapping is introduced.

Validation

  • uv run --extra dev --extra modal pytest -q — 1,122 passed, 17 deselected
  • uv run --extra dev pytest -q hud/agents/tests — 159 passed
  • uv run --extra dev ruff format . --check
  • uv run --extra dev ruff check .
  • uv run --extra dev ty check hud/agents/tool_agent.py hud/agents/openai/tools/coding.py hud/agents/openai/tools/computer.py hud/agents/tests/test_provider_native_tools.py

Note

Medium Risk
Changes which native tools are sent to OpenAI based on model ID patterns. Wrong allowlists could omit tools on supported models or still send them on unsupported ones.

Overview
Stops OpenAI native shell and computer tools from being registered when the selected model cannot use them (e.g. gpt-4o-mini with a shell-capable env).

default_spec now returns None unless the model matches GPT-5.4/5.5/5.6 (including variants). Shared _build_tools only warns when a matching capability is present and otherwise continues the rollout without those tools.

Reviewed by Cursor Bugbot for commit 785316d. Bugbot is set up for automated code reviews on this repo. Configure here.

@solvemproblr
solvemproblr marked this pull request as ready for review August 23, 2026 15:00

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9a101a0. Configure here.

Comment thread hud/agents/openai/tools/coding.py
@solvemproblr
solvemproblr marked this pull request as draft August 23, 2026 15:42
@solvemproblr
solvemproblr marked this pull request as ready for review August 23, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant