Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/concepts/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Used with `agent.with_mllm()` for the [MLLM flow](../guides/mllm-flow.md). These
| Class | Provider | Area | Required Parameters |
|---|---|---|---|
| `OpenAIRealtime` | OpenAI Realtime | Global | `api_key`; optional `turn_detection` |
| `AzureOpenAIRealtime` | Azure OpenAI Realtime | Global | `api_key`, `url`, `turn_detection`; optional `max_history` |
| `AzureOpenAIRealtime` | Azure OpenAI Realtime | Global | `api_key`, `url`; optional `turn_detection`, `max_history` |
| `GeminiLive` | Google Gemini Live API | Global | `api_key`, `model`; optional `turn_detection` |
| `VertexAI` | Vertex AI (Gemini Live) | Global | `model`, `project_id`, `location`, `adc_credentials_string`; optional `turn_detection` |
| `XaiGrok` | xAI Grok (`mllm.vendor`: `xai`) | Global | `api_key`; optional `voice`, `language`, `sample_rate`, `turn_detection` |
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -821,7 +821,7 @@ Global Azure OpenAI Realtime vendor (`mllm.vendor`: `"azure"`).
| `output_modalities` | `List[str]` | No | `None` | Output modalities |
| `messages` | `List[Dict]` | No | `None` | Conversation messages |
| `params` | `Dict[str, Any]` | No | `None` | Additional Azure OpenAI parameters |
| `turn_detection` | `MllmTurnDetectionConfig` | Yes | | Required MLLM turn detection configuration; overrides top-level `turn_detection` |
| `turn_detection` | `MllmTurnDetectionConfig` | No | `None` | Optional MLLM turn detection configuration; overrides top-level `turn_detection` when provided |

### `QwenOmni`

Expand Down
7 changes: 5 additions & 2 deletions src/agora_agent/agentkit/vendors/mllm.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ class AzureOpenAIRealtimeOptions(BaseModel):
output_modalities: Optional[List[str]] = Field(default=None, description="Output modalities")
messages: Optional[List[Dict[str, Any]]] = Field(default=None, description="Conversation messages")
params: Optional[Dict[str, Any]] = Field(default=None, description="Additional Azure OpenAI parameters")
turn_detection: MllmTurnDetectionConfig = Field(..., description="MLLM turn detection configuration")
turn_detection: Optional[MllmTurnDetectionConfig] = Field(
default=None, description="MLLM turn detection configuration"
)
failure_message: Optional[str] = Field(default=None, description="Message played on failure")


Expand Down Expand Up @@ -126,7 +128,8 @@ def to_config(self) -> Dict[str, Any]:
config["messages"] = self.messages
if self.failure_message is not None:
config["failure_message"] = self.failure_message
config["turn_detection"] = self.turn_detection
if self.turn_detection is not None:
config["turn_detection"] = self.turn_detection
return config


Expand Down
7 changes: 4 additions & 3 deletions tests/custom/test_request_body.py
Original file line number Diff line number Diff line change
Expand Up @@ -1326,9 +1326,10 @@ def test_azure_openai_realtime_rejects_input_modalities() -> None:
)


def test_azure_mllm_requires_turn_detection_and_qwen_mllm_requires_url() -> None:
with pytest.raises(ValidationError):
AzureOpenAIRealtime(url="AZURE_URL", api_key="APIKEY") # type: ignore[call-arg]
def test_azure_mllm_allows_omitting_turn_detection_and_qwen_mllm_requires_url() -> None:
agent = Agent(test_client()).with_mllm(AzureOpenAIRealtime(url="AZURE_URL", api_key="APIKEY"))
props = build_properties(agent)
assert "turn_detection" not in props["mllm"]

with pytest.raises(ValidationError):
QwenOmni( # type: ignore[call-arg]
Expand Down
Loading