From 816799f3184a811a3efa7e9e1b258d9e01f37ef1 Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 17:37:11 +0800 Subject: [PATCH 1/3] docs(skills): restore API7 Gateway Bedrock guidance --- skills/a7-plugin-ai-proxy/SKILL.md | 121 ++++++++++++++++++++++++----- 1 file changed, 102 insertions(+), 19 deletions(-) diff --git a/skills/a7-plugin-ai-proxy/SKILL.md b/skills/a7-plugin-ai-proxy/SKILL.md index 7f3915a..e5e9acb 100644 --- a/skills/a7-plugin-ai-proxy/SKILL.md +++ b/skills/a7-plugin-ai-proxy/SKILL.md @@ -3,8 +3,8 @@ name: a7-plugin-ai-proxy description: >- Skill for configuring the API7 Enterprise Edition ai-proxy plugin via the a7 CLI. Covers proxying requests to LLM providers (OpenAI, Azure OpenAI, DeepSeek, - Anthropic, Gemini, Vertex AI, and more), authentication per provider, - model configuration, streaming, logging, and route/service usage. + Anthropic, Gemini, Vertex AI, Amazon Bedrock, and more), authentication per + provider, model configuration, streaming, logging, and route/service usage. version: "1.0.0" author: API7.ai Contributors license: Apache-2.0 @@ -23,51 +23,66 @@ metadata: ## Overview -The `ai-proxy` plugin turns API7 Enterprise Edition (API7 EE) into an AI gateway. It proxies requests in -OpenAI-compatible format to LLM providers, handling authentication, endpoint -routing, and response streaming. Clients send a standard chat-completion -request; the plugin translates and forwards it to the configured provider. +The `ai-proxy` plugin turns API7 Enterprise Edition (API7 EE) into an AI +gateway. Clients can send requests in supported protocols to API7 EE instead +of handling provider authentication and endpoint selection themselves. The +plugin detects the client protocol, selects a compatible provider endpoint, +forwards the native format or converts it when an adapter is available, and +handles response streaming. ## When to Use -- Proxy chat-completion or embedding requests to any supported LLM provider +- Proxy Chat Completions, Responses API, Embeddings, Anthropic Messages, or + Bedrock Converse requests to a compatible provider - Centralize API keys at the gateway instead of distributing to clients - Add observability (token counts, latency) to LLM calls - Combine with `ai-prompt-template`, `ai-prompt-decorator`, or content moderation plugins for a full AI gateway pipeline - Apply consistent AI proxy configurations directly on services or routes +## Protocol Detection + +API7 Gateway uses the request URI as part of protocol detection. Anthropic +Messages requests must use a URI ending in `/v1/messages`, and Bedrock Converse +requests must use a URI ending in `/converse`. Without these suffixes, a request +body can match another protocol, such as OpenAI Chat. + +For Bedrock streaming, keep the client-facing URI ending in `/converse` and set +`stream: true` in the request body. API7 Gateway then selects the upstream +`/model/{modelId}/converse-stream` endpoint. + ## Supported Providers -| Provider | Value | Default Endpoint | -|----------|-------|------------------| -| OpenAI | `openai` | `https://api.openai.com/v1/chat/completions` | +| Provider | Value | Endpoint Behavior | +|----------|-------|-------------------| +| OpenAI | `openai` | Automatically selects `/v1/chat/completions`, `/v1/responses`, or `/v1/embeddings` on `https://api.openai.com` | | DeepSeek | `deepseek` | `https://api.deepseek.com/chat/completions` | | Azure OpenAI | `azure-openai` | Custom via `override.endpoint` | -| Anthropic | `anthropic` | `https://api.anthropic.com/v1/chat/completions` | +| Anthropic | `anthropic` | Automatically selects `/v1/chat/completions` or `/v1/messages` on `https://api.anthropic.com` | | AIMLAPI | `aimlapi` | `https://api.aimlapi.com/v1/chat/completions` | | OpenRouter | `openrouter` | `https://openrouter.ai/api/v1/chat/completions` | | Gemini | `gemini` | `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` | | Vertex AI | `vertex-ai` | `https://aiplatform.googleapis.com` | +| Amazon Bedrock | `bedrock` | Region- and model-specific Bedrock Runtime endpoint; available from API7 Enterprise 3.9.12 | | OpenAI-Compatible | `openai-compatible` | Custom via `override.endpoint` | ## Plugin Configuration Reference | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| -| `provider` | string | **Yes** | — | One of the 9 supported providers | +| `provider` | string | **Yes** | — | One of the 10 supported providers | | `auth` | object | **Yes** | — | Authentication config (see below) | | `options` | object | No | — | Model and generation parameters | | `options.model` | string | No | — | Model name (provider-specific) | | `options.temperature` | number | No | — | Sampling temperature | | `options.top_p` | number | No | — | Nucleus sampling | | `options.max_tokens` | integer | No | — | Maximum tokens to generate | -| `options.stream` | boolean | No | `false` | Enable SSE streaming | -| `override` | object | No | — | Override default endpoint | -| `override.endpoint` | string | No | — | Full URL for the provider API | -| `provider_conf` | object | No | — | Provider-specific config (Vertex AI) | -| `provider_conf.project_id` | string | No | — | GCP project ID (Vertex AI) | -| `provider_conf.region` | string | No | — | GCP region (Vertex AI) | +| `options.stream` | boolean | No | — | Override the outgoing `stream` field. For Bedrock Converse, `stream: true` on a `/converse` request selects `/model/{modelId}/converse-stream` and returns unmodified AWS EventStream binary frames with `Content-Type: application/vnd.amazon.eventstream`, not SSE; clients must parse EventStream responses. | +| `override` | object | No | — | Provider endpoint and request-body override settings | +| `override.endpoint` | string | No | — | Provider scheme and host, or a full URL including the path and query | +| `provider_conf` | object | No | — | Provider-specific config for Vertex AI or Amazon Bedrock | +| `provider_conf.project_id` | string | No | — | GCP project ID for Vertex AI; required with `region` unless `override.endpoint` is configured | +| `provider_conf.region` | string | No | — | GCP region for Vertex AI; required AWS region for Amazon Bedrock | | `logging` | object | No | — | Logging options | | `logging.summaries` | boolean | No | `false` | Log model, duration, tokens | | `logging.payloads` | boolean | No | `false` | Log request/response bodies | @@ -79,7 +94,7 @@ request; the plugin translates and forwards it to the configured provider. ## Authentication by Provider -### OpenAI / DeepSeek / Anthropic / AIMLAPI / OpenRouter +### OpenAI / DeepSeek / AIMLAPI / OpenRouter ```json { @@ -91,6 +106,22 @@ request; the plugin translates and forwards it to the configured provider. } ``` +### Anthropic + +```json +{ + "auth": { + "header": { + "x-api-key": "your-anthropic-api-key", + "anthropic-version": "2023-06-01" + } + } +} +``` + +Native Anthropic Messages requests require an `anthropic-version` header. +Configure it in `auth.header`, as shown, or require clients to send it. + ### Azure OpenAI ```json @@ -106,6 +137,18 @@ request; the plugin translates and forwards it to the configured provider. } ``` +### Gemini + +```json +{ + "auth": { + "header": { + "Authorization": "Bearer your-gemini-key" + } + } +} +``` + ### Vertex AI (GCP Service Account) ```json @@ -124,6 +167,46 @@ request; the plugin translates and forwards it to the configured provider. } ``` +The `service_account_json` can also be set through the +`GCP_SERVICE_ACCOUNT` environment variable. + +### Amazon Bedrock + +```json +{ + "auth": { + "aws": { + "access_key_id": "your-access-key-id", + "secret_access_key": "your-secret-access-key", + "session_token": "your-session-token" + } + }, + "provider_conf": { + "region": "us-east-1" + }, + "options": { + "model": "your-model-id" + } +} +``` + +The session token is required when you use temporary AWS credentials. + +### Custom OpenAI-Compatible API + +```json +{ + "auth": { + "header": { + "Authorization": "Bearer your-token" + } + }, + "override": { + "endpoint": "https://your-custom-llm.com/v1/chat/completions" + } +} +``` + ## Step-by-Step: Route to OpenAI ### 1. Create a route with ai-proxy From 5201da688bfafd5fa3dc7e6ca1fdcb506b4ced57 Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 17:44:38 +0800 Subject: [PATCH 2/3] docs(skills): correct Bedrock release boundary --- skills/a7-plugin-ai-proxy/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/a7-plugin-ai-proxy/SKILL.md b/skills/a7-plugin-ai-proxy/SKILL.md index e5e9acb..6682e32 100644 --- a/skills/a7-plugin-ai-proxy/SKILL.md +++ b/skills/a7-plugin-ai-proxy/SKILL.md @@ -63,7 +63,7 @@ For Bedrock streaming, keep the client-facing URI ending in `/converse` and set | OpenRouter | `openrouter` | `https://openrouter.ai/api/v1/chat/completions` | | Gemini | `gemini` | `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` | | Vertex AI | `vertex-ai` | `https://aiplatform.googleapis.com` | -| Amazon Bedrock | `bedrock` | Region- and model-specific Bedrock Runtime endpoint; available from API7 Enterprise 3.9.12 | +| Amazon Bedrock | `bedrock` | Region- and model-specific Bedrock Runtime endpoint; available from API7 Enterprise 3.9.11 | | OpenAI-Compatible | `openai-compatible` | Custom via `override.endpoint` | ## Plugin Configuration Reference From 11621801436cd44662c67a8c3138a1eccb53d3f9 Mon Sep 17 00:00:00 2001 From: kayx23 Date: Tue, 11 Aug 2026 17:47:36 +0800 Subject: [PATCH 3/3] docs(skills): document Responses protocol detection --- skills/a7-plugin-ai-proxy/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/a7-plugin-ai-proxy/SKILL.md b/skills/a7-plugin-ai-proxy/SKILL.md index 6682e32..867fefe 100644 --- a/skills/a7-plugin-ai-proxy/SKILL.md +++ b/skills/a7-plugin-ai-proxy/SKILL.md @@ -47,6 +47,10 @@ Messages requests must use a URI ending in `/v1/messages`, and Bedrock Converse requests must use a URI ending in `/converse`. Without these suffixes, a request body can match another protocol, such as OpenAI Chat. +OpenAI Responses requests with an `input` field must use a URI ending in +`/v1/responses`. Otherwise, API7 Gateway detects the body as OpenAI Embeddings; +use a URI ending in `/v1/embeddings` for embedding routes. + For Bedrock streaming, keep the client-facing URI ending in `/converse` and set `stream: true` in the request body. API7 Gateway then selects the upstream `/model/{modelId}/converse-stream` endpoint.