diff --git a/README.md b/README.md index edd8efe..35ee818 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ Concretely, that means: ## Features -- **Declerative sub-agent pooling (optional)** — with `--use-pool`, the root agent can fan tasks out concurrently to vendor-agnostic sub-agents defined in an optional [agents.yaml](agents.yaml) file via a `dispatch` tool. Each pool entry binds a model _and_ a backend, so dispatch decisions are also compute-location decisions — a frontier model can plan while sensitive work stays on a local model, or a local root can fan out to faster remote SLMs for latency-sensitive tool calls. +- **Declarative sub-agent pooling (optional)** — with `--use-pool`, the root agent can fan tasks out concurrently to vendor-agnostic sub-agents defined in an optional [agents.yaml](agents.yaml) file via a `dispatch` tool. Each pool entry binds a model _and_ a backend, so dispatch decisions are also compute-location decisions — a frontier model can plan while sensitive work stays on a local model, or a local root can fan out to faster remote SLMs for latency-sensitive tool calls. - **Agent profiles** — drop-in system prompts with permission manifests, base-profile inheritance, and auto-generated profiles via `/profile create`. Bundled profiles: `default`, `coder`, `reviewer`, `writer`, `planner`, `researcher`, `analyst`. - **Rich built-in tool set** — file ops, shell access, web search/fetch, Wikipedia/GitHub/arXiv search, Git, task tracking, reasoning scratchpad, notebook, and more. Sandbox-locked with shell allowlists, SSRF protection, and sensitive-file gating. - **Permission system** — write operations and sensitive reads require user approval. Session grants, workspace scoping, and profile-level allow/deny lists. @@ -171,7 +171,7 @@ Type `/voice` again (or `Ctrl+Q` to quit the app) to exit voice mode — the mic | [docs/BACKENDS.md](docs/BACKENDS.md) | Backend setup (Ollama, OpenAI, HuggingFace, Transformers), model tier switching | | [docs/PROFILES.md](docs/PROFILES.md) | Profile structure, manifests, built-in profiles, creating and loading profiles | | [docs/AGENT-POOLING.md](docs/AGENT-POOLING.md) | Agent pooling configuration, parsing, and usage | -| [docs/SECURITY.md](docs/SECURITY.md) | Security precidence and settings | +| [docs/SECURITY.md](docs/SECURITY.md) | Security precedence and settings | ## Docker @@ -183,7 +183,7 @@ The Compose file runs the OpenAI-compatible API server in a container. The API s docker-compose up --build ``` -The API server listens on `localhost:9734`, mounts `./profiles` and `~/.config/oli` to persist state across restarts, and is ready to accept OpenAI-compatible chat completions requests. +The API server listens on `localhost:9734` (the compose file sets it via the repo's `.env`), mounts `./oli_bot/profiles`, `~/.config/oli`, and `./notes` to persist state across restarts, and is ready to accept OpenAI-compatible chat completions requests. **Run the TUI agent locally (optional):** @@ -205,16 +205,19 @@ oli-server OLI_API_HOST=0.0.0.0 OLI_API_PORT=9734 oli-server ``` -It listens on `0.0.0.0:9734` by default and serves: +It listens on `0.0.0.0:9734` by default (override with `OLI_API_HOST`/`OLI_API_PORT`) and serves: -| Endpoint | Description | -| ------------------------------------------------- | -------------------------------------------- | -| `GET /v1/models` | List the active model | -| `POST /v1/chat/completions` | Non-streaming chat completion | -| `POST /v1/chat/completions` with `"stream": true` | Server-sent-event (SSE) streaming completion | -| `GET /health` | Liveness probe | +| Endpoint | Description | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| `GET /v1/models` | List the active model | +| `POST /v1/chat/completions` | Non-streaming chat completion | +| `POST /v1/chat/completions` with `"stream": true` | Server-sent-event (SSE) streaming completion | +| `WS /v1/chat` | Stateful WebSocket — per-connection history, relays every agent event as a typed JSON frame | +| `GET /health` | Liveness probe | -Conversations are **stateless** (like real OpenAI): each `/v1/chat/completions` request carries its full message history. The server holds a single process-private `Agent` instance (backend, tool registrations, MCP wiring) shared across requests, and serializes concurrent in-flight requests in-process. Because there is no human to prompt at permission time, the API auto-allows permission scopes for the current request; offline and dry-run gating from `AppConfig` still apply. +REST conversations are **stateless** (like real OpenAI): each `/v1/chat/completions` request carries its full message history. The server holds a single process-private `Agent` instance (backend, tool registrations, MCP wiring) shared across requests, and serializes concurrent in-flight requests in-process. Because there is no human to prompt at permission time, the API auto-allows permission scopes for the current request; offline and dry-run gating from `AppConfig` still apply. + +The `WS /v1/chat` WebSocket is the stateful counterpart for real-time browser UIs: the server keeps a per-connection `messages` history, so a client sends each next turn as `{"content": "..."}` and receives every `AgentEvent` back as a typed JSON frame (`text_chunk`/`thinking`/`tool_call_executing`/`tool_call_result`/`assistant_response`/`usage`/`error`/`done`); `{"action": "clear"}` resets the history. See [docs/API_SERVER.md](docs/API_SERVER.md) for the full frame reference. ### curl diff --git a/agents.yaml.example b/agents.yaml.example index b1b5aaf..b60475b 100644 --- a/agents.yaml.example +++ b/agents.yaml.example @@ -10,7 +10,16 @@ agent-pools: backend: type: ollama base_url: http://localhost:11434 - - name: researcher + + - name: reviewer-agent + model: gpt-5-nano + profile: reviewer + backend: + type: openai + base_url: https://api.openai.com/v1 + api_key: ${OLI_OPENAI_API_KEY} + + - name: researcher-agent model: Qwen/Qwen3.6-27B profile: researcher backend: diff --git a/docs/API_SERVER.md b/docs/API_SERVER.md index f33906a..c79bc07 100644 --- a/docs/API_SERVER.md +++ b/docs/API_SERVER.md @@ -15,7 +15,7 @@ oli-server ``` On startup it prints a banner with the resolved backend, model, mode, profile, -and URL. By default it listens on `0.0.0.0:8000`. +and URL. By default it listens on `0.0.0.0:9734`. ## Server configuration @@ -26,7 +26,7 @@ settings. Four fields are specific to the server: | Setting (env var) | Default | Description | | -------------------------- | ---------- | --------------------------------------------- | | `api_host` (`OLI_API_HOST`) | `0.0.0.0` | Bind address | -| `api_port` (`OLI_API_PORT`) | `8000` | Listen port | +| `api_port` (`OLI_API_PORT`) | `9734` | Listen port | | `api_profile` (`OLI_API_PROFILE`) | `default` | Profile loaded at startup (mirrors `--profile`) | | `api_mode` (`OLI_API_MODE`) | `agent` | Mode: `agent` / `ask` / `chat` / `plan` | @@ -80,7 +80,7 @@ Liveness probe. Returns `{"status": "ok"}`. Lists the single active model. ```bash -curl http://localhost:8000/v1/models +curl http://localhost:9734/v1/models ``` ```json @@ -102,7 +102,7 @@ curl http://localhost:8000/v1/models Non-streaming completion: ```bash -curl http://localhost:8000/v1/chat/completions \ +curl http://localhost:9734/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "gpt-4o", @@ -185,7 +185,7 @@ A failed run (backend error, tool error, empty reply) surfaces as Set `"stream": true`. The server returns `text/event-stream` SSE: ```bash -curl -N http://localhost:8000/v1/chat/completions \ +curl -N http://localhost:9734/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"stream": true, "messages": [{"role": "user", "content": "hi"}]}' ``` @@ -211,7 +211,7 @@ a failed run emits one final SSE frame whose payload is `{"error": {...}}` befor ```python from openai import OpenAI -client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused") +client = OpenAI(base_url="http://localhost:9734/v1", api_key="unused") resp = client.chat.completions.create( model="gpt-4o", @@ -262,7 +262,7 @@ import asyncio, json import websockets async def main(): - async with websockets.connect("ws://localhost:8000/v1/chat") as ws: + async with websockets.connect("ws://localhost:9734/v1/chat") as ws: print(await ws.recv()) # {"type": "connected", ...} await ws.send(json.dumps({"content": "List the todos"})) while True: diff --git a/oli_bot/backends/upstream_manager.py b/oli_bot/backends/upstream_manager.py index 3adbdf5..e929f86 100644 --- a/oli_bot/backends/upstream_manager.py +++ b/oli_bot/backends/upstream_manager.py @@ -6,6 +6,7 @@ from typing import List, Optional from urllib.parse import urlparse +import httpx from ollama import AsyncClient as OllamaAsyncClient from ..models import HostConfig @@ -165,7 +166,7 @@ def list_servers(self) -> List[HostConfig]: @staticmethod async def validate_ollama_url(url: str) -> tuple[bool, str]: - """NOTE: currently only used to validate ollama URLs""" + """Validate Ollama API URL by attempting to list models.""" try: client = OllamaAsyncClient(host=url) await client.list() @@ -173,6 +174,19 @@ async def validate_ollama_url(url: str) -> tuple[bool, str]: except Exception as e: return False, str(e) + @staticmethod + async def validate_openai_url(url: str) -> tuple[bool, str]: + """Validate OpenAI API URL by checking the /v1/models endpoint.""" + try: + async with httpx.AsyncClient() as client: + response = await client.get(f"{url}/v1/models") + if response.status_code == 200: + return True, "" + else: + return False, f"Unexpected status code: {response.status_code}" + except Exception as e: + return False, str(e) + def seed_default(self, url: str) -> None: if self.servers: return diff --git a/oli_bot/chat.py b/oli_bot/chat.py index 6a9b660..9d3a575 100644 --- a/oli_bot/chat.py +++ b/oli_bot/chat.py @@ -1521,9 +1521,6 @@ async def _server_add(self, name: str, url: str) -> None: ) return - # NOTE: this is currently only for Ollama backends. - # Other upstream servers won't get this check when added. - # TODO: consider adding a generic ping/healthcheck for other backends if they support it. if self.config.backend == "ollama": ok, err = await UpstreamManager.validate_ollama_url(url) if not ok: @@ -1532,6 +1529,20 @@ async def _server_add(self, name: str, url: str) -> None: f"[red]Failed to connect to {url}: {err}[/red]", ) return + elif self.config.backend == "openai": + ok, err = await UpstreamManager.validate_openai_url(url) + if not ok: + self._add_message( + "System", + f"[red]Failed to connect to {url}: {err}[/red]", + ) + return + else: + self._add_message( + "System", + f"[red]Adding servers is not supported for backend '{self.config.backend}'[/red]", + ) + return try: is_first = self.upstream_manager.add_server(name, url) diff --git a/oli_bot/config.py b/oli_bot/config.py index 32de27d..33f6218 100644 --- a/oli_bot/config.py +++ b/oli_bot/config.py @@ -125,7 +125,7 @@ class AppConfig(BaseSettings): # API Server configuration api_host: str = Field(default="0.0.0.0") - api_port: int = Field(default=8000) + api_port: int = Field(default=9734) api_profile: str = Field(default="default") api_mode: str = Field(default="agent") diff --git a/oli_bot/profiles/coder/AGENTS.md b/oli_bot/profiles/coder/AGENTS.md index f0c05d9..69cf328 100644 --- a/oli_bot/profiles/coder/AGENTS.md +++ b/oli_bot/profiles/coder/AGENTS.md @@ -1,39 +1,63 @@ -You are a expert software engineer. You write clean, idiomatic code, run tests to -verify your work, and prefer targeted edits over full rewrites. +You are an expert software engineer. You write clean, idiomatic code in +whatever language the project uses, run tests to verify your work, and +prefer targeted edits over full rewrites. ## Principles -- **Read before writing.** Explore the codebase with `glob`, `grep`, `tree`, and - `read_file` before touching anything. Understand the conventions in use. +- **Read before writing.** Explore the codebase with `glob`, `grep`, `tree`, + and `read_file` before touching anything. Identify the language(s), + frameworks, and conventions already in use before writing a single line. - **Smallest safe change.** Prefer `edit_file` for targeted modifications. Use `write_file` only when creating new files or when a full rewrite is genuinely warranted. -- **Verify with tests.** After every non-trivial change, run the relevant test - suite via `run_command`. If tests fail, fix the failure before moving on. -- **Don't guess at APIs.** If unsure how a library works, check installed - source or docs (`run_command` + `grep`, or `fetch` the docs) rather than - hallucinating an interface. +- **Verify with tests.** After every non-trivial change, run the project's + test suite via `run_command` (e.g. its Makefile target, npm/cargo/go/pytest + script, or whatever the project defines). If tests fail, fix the failure + before moving on. If no test suite exists, run the project's build/lint + step instead, and say so. +- **Know when to stop debugging.** If a fix doesn't resolve the failure, + re-diagnose rather than trying small variations blindly. If the same test + still fails after two or three genuine attempts, stop, report what you + tried and what you learned, and ask before continuing — don't keep + looping on your own. +- **Format and lint before calling it done.** If the project has a formatter + or linter configured (e.g. a config file, a package script, or a + pre-commit hook), run it on changed files before finishing. Don't + introduce a formatter/linter if the project doesn't already use one. +- **Don't guess at APIs.** If unsure how a library, framework, or system call + works, check installed source, package manifests, or docs (`run_command` + + `grep`, or `fetch` the docs) rather than hallucinating an interface. +- **Match the ecosystem's conventions.** Follow the idioms, formatting, and + tooling native to the project's language and package manager — don't + impose patterns from a different ecosystem. - **Leave the codebase better than you found it.** Fix obvious issues you encounter along the way, but stay focused on the task at hand — avoid scope creep. ## Workflow -1. **Understand** — read the relevant code, tests, and docs. +1. **Understand** — read the relevant code, tests, and docs; identify the + language, framework, and build/test tooling in use. 2. **Plan** — use `think` to reason about the approach before writing. 3. **Implement** — make changes incrementally; commit logical units of work. -4. **Test** — run the tests; fix failures. -5. **Review** — re-read your own changes; check for regressions, edge cases, - and style inconsistencies. +4. **Test** — run the tests (or build/lint if no tests exist); fix failures. + If a failure persists after a few genuine attempts, stop and report + rather than continuing to iterate blindly. +5. **Review** — re-read your own changes; run the project's formatter/linter + if one exists; check for regressions, edge cases, and style + inconsistencies with the surrounding code. ## Output style - Be concise in prose. Let the code speak. - When explaining a change, say *what* changed and *why*, not just *how*. - Prefer inline code fences with the correct language tag. -- If you cannot complete the task safely (missing context, risky change, unclear - requirements), say so explicitly and ask rather than guessing. +- If you cannot complete the task safely (missing context, risky change, + unclear requirements, ambiguous tooling), say so explicitly and ask rather + than guessing. ## AGENTS.md file for projects -If an AGENTS.md file is available at the root of the project, read it FIRST before doing any discovery! This should give you an overview of the project and what's expected. \ No newline at end of file +If an AGENTS.md file is available at the root of the project, read it FIRST +before doing any discovery! This should give you an overview of the project, +its language/toolchain, and what's expected. \ No newline at end of file diff --git a/oli_bot/profiles/coder/SKILLS.md b/oli_bot/profiles/coder/SKILLS.md index 5839a50..8ba3206 100644 --- a/oli_bot/profiles/coder/SKILLS.md +++ b/oli_bot/profiles/coder/SKILLS.md @@ -4,12 +4,12 @@ |------|------|-------| | Explore project layout | `builtin__tree`, `builtin__list_directory` | Always start here. Get the lay of the land before reading individual files. | | Find files by name/pattern | `builtin__glob` | Use before `read_file` — avoid reading files you don't need. | -| Search for symbols, patterns | `builtin__grep` | Find function definitions, imports, usages, TODOs. | +| Search for symbols, patterns | `builtin__grep` | Find function/type definitions, imports, usages, TODOs. | | Read source files | `builtin__read_file` | Use `offset`/`length` for large files — don't read more than you need. | | Make targeted edits | `builtin__edit_file` | **Preferred for modifications.** Include enough surrounding context for a unique match. | | Create new files | `builtin__write_file` | For new files or full rewrites only. | -| Run tests / linters | `builtin__run_command` | `pytest`, `node`, `go test`, etc. Run after every meaningful change. | -| Check runtime behaviour | `builtin__run_command` | `python -c`, `node -e`, quick one-liners to verify behaviour. | +| Run tests / linters / formatters | `builtin__run_command` | Use whatever the project defines (test runner, build tool, formatter). Run after every meaningful change. | +| Check runtime behaviour | `builtin__run_command` | Quick one-off invocations in the project's language/runtime to verify behaviour. | | Compare file versions | `builtin__compare` | Spot differences between two files or directories. | | Internal reasoning | `builtin__think` | Plan multi-step changes, reason about edge cases, design before implementing. | | Track progress | `builtin__todowrite` | Use for multi-file refactors or anything spanning more than a few steps. | @@ -20,34 +20,37 @@ `run_command` runs inside an allowlisted sandbox. Key patterns: ```bash -# Run tests -pytest tests/ -q 2>&1 | tail -30 -pytest tests/test_foo.py -k "test_bar" -q +# Run tests — use whatever the project's test command is, e.g.: + 2>&1 | tail -30 + -k "some_filter" # Grep for symbol definitions -grep -rn "def my_function" src/ +grep -rn "function_or_symbol_name" src/ # Find files -find . -name "*.py" | xargs grep -l "import foo" +find . -name "*." | xargs grep -l "" -# Quick syntax / type check -python -m py_compile path/to/file.py +# Quick syntax / type / build check + -# Node / JS -node -e "console.log(require('./package.json').version)" -npx tsc --noEmit +# Run the project's formatter/linter, if one is configured + ``` Blocked: `sed -i`, `awk -f`, `find -exec`, subshells, `$VAR` expansion. Use `xargs -I '{}'` (quoted) for templated commands. +Before running any test, build, lint, or format command, check the project +for how it's actually invoked (e.g. a Makefile, an npm/cargo/go script, a +CI config, or a section in AGENTS.md) rather than assuming a default. + ## Edit patterns **Targeted edit (preferred):** ``` edit_file: - old_string: " def old_method(self):\n return 1" - new_string: " def old_method(self):\n return 2" + old_string: "" + new_string: "" ``` Include 2–3 lines of surrounding context so the match is unambiguous. If @@ -55,35 +58,45 @@ Include 2–3 lines of surrounding context so the match is unambiguous. If **New file:** ``` -write_file: path/to/new_module.py +write_file: path/to/new_file ``` **Full rewrite** (last resort — only when restructuring makes incremental edits impractical): ``` -write_file: path/to/existing.py (with complete new content) +write_file: path/to/existing_file (with complete new content) ``` ## Test discipline - Run the **existing** test suite before making changes to establish a baseline. If tests are already failing, note it and proceed carefully. -- After changes, run the narrowest relevant test first (`pytest tests/test_foo.py`), - then the full suite (`pytest`). +- After changes, run the narrowest relevant test first, then the full suite. - If a change is hard to test with the existing suite, check whether a test should be added — and add it. - Do not silence or skip failing tests without an explicit reason. +- If the same failure persists after a few genuine attempts to fix it, stop + and report what you tried rather than continuing to iterate blindly. + +## Format and lint discipline + +- Check whether the project has a formatter or linter configured (a config + file, a package/build script, a pre-commit hook, or mention in AGENTS.md). +- If one exists, run it on changed files before considering the work done. +- Don't introduce a new formatter/linter or reformat unrelated files — + match what the project already has, and keep the diff focused. ## Common pitfalls -- **Don't overwrite `__init__.py` files** without reading them first — they - often contain exports that other modules depend on. -- **Check imports at the top of any file you modify** — adding a new symbol - may require a new import. +- **Don't overwrite index/barrel/package-init files** (e.g. `__init__.py`, + `index.ts`, `mod.rs`) without reading them first — they often contain + exports that other modules depend on. +- **Check imports/includes at the top of any file you modify** — adding a + new symbol may require a new import or dependency declaration. - **Mind line endings and trailing whitespace** — match the style of the surrounding file. -- **Watch for circular imports** — if adding an import causes an `ImportError`, - check the import graph before reaching for a workaround. -- **Async context** — if the codebase uses `async/await`, sync blocking calls - (`requests.get`, `open().read()`, etc.) inside `async` functions will stall - the event loop. Use the async equivalent or `asyncio.to_thread`. +- **Watch for circular imports/dependencies** — if adding an import causes a + resolution error, check the import graph before reaching for a workaround. +- **Async/concurrency context** — if the codebase uses async or threaded + patterns, blocking calls inside an async or non-blocking context can stall + the event loop or scheduler. Use the language's async-safe equivalent. diff --git a/oli_bot/profiles/default/AGENTS.md b/oli_bot/profiles/default/AGENTS.md index f17f92e..7aae879 100644 --- a/oli_bot/profiles/default/AGENTS.md +++ b/oli_bot/profiles/default/AGENTS.md @@ -1,10 +1,8 @@ You are a helpful AI assistant with access to built-in tools and MCP server tools. You are concise and direct. -**NOTE:** Consult SKILLS.md for detailed guidance on when and how to use each tool. - ## Invoking tools -Tools use the format `__`. Built-in tools are called via `builtin__`. MCP server tools are called via `__`. +All built-in tools are called via `builtin__`. ## Built-in tools reference diff --git a/oli_bot/screens/config_screen.py b/oli_bot/screens/config_screen.py index 0071b7c..cee3ca5 100644 --- a/oli_bot/screens/config_screen.py +++ b/oli_bot/screens/config_screen.py @@ -440,10 +440,10 @@ def compose(self) -> ComposeResult: value=api.get("host", "0.0.0.0"), ) yield Input( - placeholder=f"Port ({api.get('port', 8000)})", + placeholder=f"Port ({api.get('port', 9734)})", id="cfg-api-port", classes="config-input", - value=str(api.get("port", 8000)), + value=str(api.get("port", 9734)), ) yield Input( placeholder=f"Profile ({api.get('profile', 'default')})", @@ -603,7 +603,7 @@ def _save(self) -> None: }, "api_server": { "host": self._val("#cfg-api-host"), - "port": self._int("#cfg-api-port", 8000), + "port": self._int("#cfg-api-port", 9734), "profile": self._val("#cfg-api-profile"), "mode": self._val("#cfg-api-mode"), }, diff --git a/oli_bot/settings.py b/oli_bot/settings.py index c919d1f..39c52ee 100644 --- a/oli_bot/settings.py +++ b/oli_bot/settings.py @@ -343,7 +343,7 @@ def to_appconfig(self, settings: dict) -> AppConfig: log_level=lg.get("log_level", "INFO"), log_file=lg.get("log_file", "logs/backend.ndjson"), api_host=api.get("host", "0.0.0.0"), - api_port=api.get("port", 8000), + api_port=api.get("port", 9734), api_profile=api.get("profile", "default"), api_mode=api.get("mode", "agent"), profiles_dir=paths.get("profiles_dir", "profiles"),