Dev

AGENTS.md

@@ -12,20 +12,19 @@ This file is the top-level orientation map. Depth lives under [docs/](docs/).
 | Path | Purpose |
 | --- | --- |
 | [oli_bot/chat.py](oli_bot/chat.py) | Textual TUI app (`OliBot`). Owns command handling, session UI, the `#command-suggestions` autocomplete `ListView`, and (when pooling is enabled) the "Active Sub-Agents" `Tree`. Tracks cumulative session token usage (persisted per session) and renders it in the `#status-bar`. Slash-command names come from the module-level `COMMANDS` tuple. |
-| [oli_bot/api_server.py](oli_bot/api_server.py) | FastAPI app exposing the harness over an OpenAI-compatible REST API (`GET /v1/models`, `POST /v1/chat/completions` streaming + non-streaming, `GET /health`). Stateless from the caller's POV; a single process-private `Agent` is shared across requests and serialised with a `threading.RLock`. Auto-approves permissions (no human), but offline/dry-run still apply. |
+| [oli_bot/api_server.py](oli_bot/api_server.py) | FastAPI app exposing the harness over an OpenAI-compatible REST API (`GET /v1/models`, `POST /v1/chat/completions` streaming + non-streaming, `GET /health`) plus a stateful `WS /v1/chat` WebSocket that relays every `AgentEvent` as a typed JSON envelope (`text_chunk`/`thinking`/`tool_call_executing`/`tool_call_result`/`assistant_response`/`usage`/`error`/`done`) for real-time browser UIs. REST is stateless from the caller's POV; the WebSocket keeps per-connection history (`{"content": "..."}` sends a turn, `{"action": "clear"}` resets). A single process-private `Agent` is shared across requests and serialised with a `threading.RLock`; the WebSocket holds the lock for the duration of each run. Auto-approves permissions (no human), but offline/dry-run still apply. |
 | [oli_bot/agent.py](oli_bot/agent.py) | `Agent` — mode + system prompt owner; orchestrates the tool-calling loop and streams typed events (`TextChunk`, `ThinkingChunk`, `ToolCallChunk`, `ToolCallExecuting`, `ToolCallResult`, `StreamChunk`, `UsageEvent`, `Error`, `Done`). Aggregates per-call `UsageChunk`s from each backend round into a single per-run `UsageEvent`. Also hosts `sanitize_tool_history`, `_merge_usage`, `stream_sub_agent_run`, and `AgentPool` (built from [oli_bot/agents.yaml](oli_bot/agents.yaml) when `--use-pool` is set). |
 | [oli_bot/backends/](oli_bot/backends/) | Backend package — `ModelBackend` ABC, `OllamaBackend`, `OpenAIBackend`, `HuggingFaceBackend`, `TransformersBackend`, and the `create_model_backend()` factory. Also hosts the shared `_StreamingThinkParser` and per-backend message formatting (Ollama native `images`, OpenAI `image_url` or Bedrock-native blocks via `openai_vision_style`, textual placeholder for text-only backends). Every backend surfaces a trailing `UsageChunk`: exact counts from provider usage where available (OpenAI `usage`/`stream_options`, Ollama `prompt_eval_count`/`eval_count`, HF `usage`), else a `~chars/4` estimate via `estimate_tokens`. See [docs/BACKENDS.md](docs/BACKENDS.md). |
 | [oli_bot/screens/](oli_bot/screens/) | All `ModalScreen` subclasses: `PermissionScreen`, `ConfirmScreen`, `ModelPickerScreen`, `ServerListScreen`, `MCPSetupScreen`, `SessionListScreen`, `WorkspaceListScreen`, `SubAgentViewScreen`, `ConfigScreen`, `InputPromptScreen`, plus `taglines.py` / `todo_widget.py`. |
 | [oli_bot/models.py](oli_bot/models.py) | Shared dataclasses: `Message`, `ToolCall`, `ModelResponse`, `HostConfig`, `MCPServerConfig`, `ProfileData`, `SubAgentRun`, `ImageAttachment`, `TodoItem` / `TodoListState`, plus `AgentEvent` variants and the `AgentRole` enum. Token accounting lives here too: `Usage` (prompt/completion/`estimated` flag), the per-call `UsageChunk` stream event, and the per-run `UsageEvent`. `Message.images` is in-memory only (dropped on session save); `ModelResponse.usage` is optionally set by backends. |
 | [oli_bot/config.py](oli_bot/config.py) | `AppConfig` — `pydantic_settings.BaseSettings`. Env vars prefixed `OLI_`, plus `.env` support and `OLI_TRUNCATION_SMALL` / `_LARGE` aliases via `AliasChoices`. Module-level `configs = AppConfig()` singleton. See [docs/CONFIGURE.md](docs/CONFIGURE.md). |
 | [oli_bot/settings.py](oli_bot/settings.py) | `SettingsManager` — load/save/merge `~/.config/oli/settings.json`; precedence `settings.json` > `OLI_*` env > SDK-standard env (`OPENAI_API_KEY`, `OPENAI_BASE_URL`, `HUGGINGFACE_API_KEY`, `HF_TOKEN`) > declared defaults. Empty API-key strings in JSON fall through to env. |
-| [oli_bot/profiles/](oli_bot/profiles/) | `ProfileManifest` / `PermissionsManifest` (Pydantic) in `schema.py`; `ProfilePermissionEnforcer` (layered allow/deny glob patterns, base-profile inheritance, deny-overrides-allow) in `permissions.py`. Built-in profiles ship as sibling directories. |
-| [oli_bot/profile_manager.py](oli_bot/profile_manager.py) | `ProfileManager` — profile CRUD, manifest loading, circular-dependency detection; delegates enforcement to `oli_bot/profiles/permissions.py`. |
+| [oli_bot/profiles/](oli_bot/profiles/) | `ProfileManifest` / `PermissionsManifest` (Pydantic) in `schema.py`; `ProfilePermissionEnforcer` (layered allow/deny glob patterns, base-profile inheritance, deny-overrides-allow) in `permissions.py`; `ProfileManager` (profile CRUD, manifest loading, circular-dependency detection) in `manager.py`. Built-in profiles ship as sibling directories. |
 | [oli_bot/mcp_client.py](oli_bot/mcp_client.py) | `MCPClientManager` — MCP server lifecycle (stdio/http via v2 `mcp.client.Client`, `mode="auto"` handshake), tool discovery + invocation, per-server tool-list cache, offline gating. Uses v2 snake_case fields (`Tool.input_schema`, `CallToolResult.is_error`/`structured_content`). |
 | [oli_bot/tools/manager.py](oli_bot/tools/manager.py) | `BuiltinToolManager` — registration, profile + session permission gating, dry-run gating, offline gating, and `TruncationManager` post-processing. Awaits coroutine handlers. |
 | [oli_bot/tools/](oli_bot/tools/) | Tool handlers: `files.py` (read/write/edit + `view_image` via Pillow), `directories.py` (glob/grep/list_directory/tree — filesystem work runs via `asyncio.to_thread` / `create_subprocess_exec`), `web.py` (search + fetch + specialised searches, all guarded by `_check_ssrf`), `shell.py` (allowlisted `run_command`, including read-only `git`), `parsing.py` (`compare`), `memory.py` (`think`, `todowrite`, `notebook`), `truncation.py` (per-tier char budgets), `permissions.py` (sensitive-path detection). See [docs/TOOLS.md](docs/TOOLS.md). |
 | [oli_bot/sessions.py](oli_bot/sessions.py) | `Session` (permission gating) + `ConversationStore` (per-server JSON persistence under `~/.config/oli/sessions/<server>/`) + `WorkspaceManager`. `save_session()` returns the (possibly new) id so callers can rebind after a corrupt-file rewrite. Persisted messages preserve `tool_call_id`; loads pass through `sanitize_tool_history` so poisoned histories self-heal. |
-| [oli_bot/server_manager.py](oli_bot/server_manager.py) | `ServerManager` — multi-server lifecycle persisted to `ollama_hosts.json`, URL validation. |
+| [oli_bot/backends/upstream_manager.py](oli_bot/backends/upstream_manager.py) | `UpstreamManager` — multi-server lifecycle persisted to `hosts.json`, URL validation. |
 | [oli_bot/voice.py](oli_bot/voice.py) | `VoiceEngine` — optional, lazy-loaded mic → STT → TTS engine for the `/voice` command (faster-whisper, Piper TTS, WebRTC VAD, pyaudio). All I/O is blocking; `chat.py` calls it via `asyncio.to_thread`. `record()` accepts a `threading.Event` so `chat.py` can interrupt an in-progress recording the instant voice mode is toggled off, instead of waiting out the silence/max-duration timeout. All seven tunables (whisper/piper models, sample rate, VAD frame duration, VAD aggressiveness, silence timeout, max record seconds) are `AppConfig` fields (`OLI_VOICE_*` env / `settings.json` `voice` section / `/config` screen); `chat.py` passes them explicitly when constructing the engine, and saving `/config` drops the engine so the next `/voice` picks up new values. |
 | [oli_bot/logger.py](oli_bot/logger.py) | Centralised NDJSON file logging (rotating, 10 MB × 5) under `AppConfig.log_file`. Deliberately no console handler — stray writes would corrupt the Textual TUI. |
 

oli_bot/agent.py

@@ -19,7 +19,7 @@
 )
 from .config import AppConfig, configs
 from .mcp_client import MCPClientManager
-from .profile_manager import ProfileData, ProfileManager
+from .profiles.manager import ProfileData, ProfileManager
 from .profiles.permissions import ProfilePermissionEnforcer
 from .models import (
     ToolCallExecuting,

oli_bot/api_server.py

@@ -1,9 +1,11 @@
 """OpenAI-compatible REST API over the oli agent harness.
 
 Serves ``/v1/models`` and ``/v1/chat/completions`` (streaming + non-streaming)
-using FastAPI, plugging the same ``Agent`` tool loop that powers the TUI into
-any workflow that speaks the OpenAI wire protocol (the ``openai`` Python SDK,
-curl, or any other HTTP client).
+with FastAPI, plus a stateful ``/v1/chat`` WebSocket that relays every agent
+event as a typed JSON frame for real-time browser UIs.  All routes plug the
+same ``Agent`` tool loop that powers the TUI into any client that speaks the
+OpenAI wire protocol (the ``openai`` Python SDK, curl, or any other HTTP
+client).
 
 The server is stateless from the caller's perspective: each
 ``/v1/chat/completions`` request carries the full message history, mirroring
@@ -19,6 +21,7 @@
 """
 
 import base64
+import dataclasses
 import json
 import logging
 import threading
@@ -29,10 +32,10 @@
 from typing import Any, AsyncIterator, Dict, List, Optional
 
 from art import text2art
-from fastapi import FastAPI
+from fastapi import FastAPI, WebSocket, WebSocketDisconnect
 from fastapi.responses import JSONResponse, StreamingResponse
 
-from .agent import Agent
+from .agent import Agent, AgentEvent
 from .backends import create_model_backend, ModelBackend
 from .config import AppConfig, configs
 from .logger import setup_logging
@@ -44,6 +47,10 @@
     ImageAttachment,
     Message,
     StreamChunk,
+    ThinkingChunk,
+    ToolCallExecuting,
+    ToolCallResult,
+    UsageEvent,
     ChatCompletionMessage,
     ChatCompletionRequest,
 )
@@ -343,6 +350,110 @@ def chunk(delta: Dict[str, Any], finish_reason: Any = None) -> str:
     yield "data: [DONE]\n\n"
 
 
+# --- WebSocket ------------------------------------------------------------- #
+
+
+def _event_to_frame(event: AgentEvent) -> Dict[str, Any]:
+    """Convert an ``AgentEvent`` into a typed JSON envelope for the browser.
+
+    The ``type`` field lets the client distinguish event kinds and render them
+    differently (streamed text, thinking blocks, tool calls, errors, etc.).
+    """
+    if isinstance(event, StreamChunk):
+        return {"type": "text_chunk", "data": {"text": event.text}}
+    if isinstance(event, ThinkingChunk):
+        return {"type": "thinking", "data": {"text": event.text}}
+    if isinstance(event, ToolCallExecuting):
+        return {
+            "type": "tool_call_executing",
+            "data": {"name": event.name, "parameters": event.parameters},
+        }
+    if isinstance(event, ToolCallResult):
+        return {
+            "type": "tool_call_result",
+            "data": {"name": event.name, "result": event.result},
+        }
+    if isinstance(event, AssistantResponse):
+        return {"type": "assistant_response", "data": {"content": event.content}}
+    if isinstance(event, UsageEvent):
+        return {"type": "usage", "data": dataclasses.asdict(event.usage)}
+    if isinstance(event, Error):
+        return {"type": "error", "data": {"message": event.message}}
+    if isinstance(event, Done):
+        return {"type": "done", "data": {"full_text": event.full_text}}
+    logger.warning("Unknown agent event in websocket relay: %r", event)
+    return {"type": "unknown", "data": {"event": repr(event)}}
+
+
+@app.websocket("/v1/chat")
+async def websocket_chat(websocket: WebSocket) -> None:
+    """Stateful WebSocket chat endpoint.
+
+    The server keeps a per-connection ``messages`` list so a client just sends
+    the next user turn (``{"content": "..."}``) and receives every ``AgentEvent``
+    back as a typed JSON frame.  ``{"action": "clear"}`` resets the history.
+    Runs are serialized on ``app.state.lock`` like the REST endpoints.
+    """
+    await websocket.accept()
+    messages: List[Message] = []
+    try:
+        await websocket.send_json({"type": "connected", "data": {}})
+        while True:
+            raw = await websocket.receive_text()
+            try:
+                data = json.loads(raw)
+            except json.JSONDecodeError:
+                await websocket.send_json(
+                    {"type": "error", "data": {"message": "Invalid JSON payload"}}
+                )
+                continue
+            if not isinstance(data, dict):
+                await websocket.send_json(
+                    {"type": "error", "data": {"message": "Expected a JSON object"}}
+                )
+                continue
+
+            if data.get("action") == "clear":
+                messages = []
+                await websocket.send_json({"type": "cleared", "data": {}})
+                continue
+
+            content = data.get("content")
+            if not content or not str(content).strip():
+                await websocket.send_json(
+                    {"type": "error", "data": {"message": "Empty message"}}
+                )
+                continue
+
+            messages.append(Message(role="user", content=str(content)))
+
+            try:
+                tools = await _resolve_tools(app.state.agent)
+            except Exception as e:
+                logger.warning("Failed to list tools: %s", e)
+                tools = None
+
+            with app.state.lock:
+                try:
+                    async for event in app.state.agent.process(
+                        messages, tools=tools, confirm_callback=_api_confirm
+                    ):
+                        await websocket.send_json(_event_to_frame(event))
+                        if isinstance(event, Done) and event.full_text:
+                            messages.append(
+                                Message(role="assistant", content=event.full_text)
+                            )
+                except WebSocketDisconnect:
+                    raise
+                except Exception as e:
+                    logger.exception("Agent process failed over websocket: %s", e)
+                    await websocket.send_json(
+                        {"type": "error", "data": {"message": str(e)}}
+                    )
+    except WebSocketDisconnect:
+        logger.debug("WebSocket client disconnected from /v1/chat")
+
+
 # --- Routes ----------------------------------------------------------------- #
 
 

oli_bot/server_manager.py → oli_bot/backends/upstream_manager.py

@@ -8,15 +8,15 @@
 
 from ollama import AsyncClient as OllamaAsyncClient
 
-from .models import HostConfig
+from ..models import HostConfig
 
 logger = logging.getLogger(__name__)
 
 CONFIG_DIR = Path.joinpath(Path.home(), ".config", "oli")
 CONFIG_FILE = Path.joinpath(CONFIG_DIR, "hosts.json")
 
 
-class ServerManager:
+class UpstreamManager:
     def __init__(self, config_path: str = CONFIG_FILE):
         self.config_path = config_path
         self.servers: List[HostConfig] = []