null-create · null-create · Sep 11, 2026 · Sep 10, 2026 · Sep 10, 2026 · Sep 11, 2026
diff --git a/AGENTS.md b/AGENTS.md
@@ -26,6 +26,7 @@ This file is the top-level orientation map. Depth lives under [docs/](docs/).
 | [oli_bot/tools/](oli_bot/tools/) | Tool handlers: `files.py` (read/write/edit + `view_image` via Pillow), `directories.py` (glob/grep/list_directory/tree — filesystem work runs via `asyncio.to_thread` / `create_subprocess_exec`), `web.py` (search + fetch + specialised searches, all guarded by `_check_ssrf`), `shell.py` (allowlisted `run_command`, including read-only `git`), `parsing.py` (`compare`), `memory.py` (`think`, `todowrite`, `notebook`), `truncation.py` (per-tier char budgets), `permissions.py` (sensitive-path detection). See [docs/TOOLS.md](docs/TOOLS.md). |
 | [oli_bot/sessions.py](oli_bot/sessions.py) | `Session` (permission gating) + `ConversationStore` (per-server JSON persistence under `~/.config/oli/sessions/<server>/`) + `WorkspaceManager`. `save_session()` returns the (possibly new) id so callers can rebind after a corrupt-file rewrite. Persisted messages preserve `tool_call_id`; loads pass through `sanitize_tool_history` so poisoned histories self-heal. |
 | [oli_bot/server_manager.py](oli_bot/server_manager.py) | `ServerManager` — multi-server lifecycle persisted to `ollama_hosts.json`, URL validation. |
+| [oli_bot/voice.py](oli_bot/voice.py) | `VoiceEngine` — optional, lazy-loaded mic → STT → TTS engine for the `/voice` command (faster-whisper, Piper TTS, WebRTC VAD, pyaudio). All I/O is blocking; `chat.py` calls it via `asyncio.to_thread`. `record()` accepts a `threading.Event` so `chat.py` can interrupt an in-progress recording the instant voice mode is toggled off, instead of waiting out the silence/max-duration timeout. All seven tunables (whisper/piper models, sample rate, VAD frame duration, VAD aggressiveness, silence timeout, max record seconds) are `AppConfig` fields (`OLI_VOICE_*` env / `settings.json` `voice` section / `/config` screen); `chat.py` passes them explicitly when constructing the engine, and saving `/config` drops the engine so the next `/voice` picks up new values. |
 | [oli_bot/logger.py](oli_bot/logger.py) | Centralised NDJSON file logging (rotating, 10 MB × 5) under `AppConfig.log_file`. Deliberately no console handler — stray writes would corrupt the Textual TUI. |
 
 ## Modes
@@ -74,6 +75,7 @@ See [docs/AGENT-POOLING.md](docs/AGENT-POOLING.md) for the `agents.yaml` schema,
 - **Permission gating** — `BuiltinToolManager.call_tool()` runs the profile enforcer, then `Session.needs_permission()`, then (if needed) `confirm_callback(description)`; the TUI shows `PermissionScreen` and the callback returns `"once"`, `"session"`, or `"deny"`. Session grants persist per scope for the process lifetime. `glob`/`grep` targeting patterns like `.env*`, `*.pem`, `*secret*` trigger the `workspace_sensitive` scope even inside the workspace.
 - **Built-in tool naming** — registered as `builtin__<name>` and dispatched by `MCPClientManager.call_tool`.
 - **Settings round-trip** — runtime model changes (`/model set-large|set-small`, `/servers set-default-model`) persist back to `settings.json` via `OliBot._persist_model_to_settings`; the `/config` form pre-populates from effective runtime values (server overrides included) via `_sync_settings_from_runtime`. Backend construction honours `self.config` overrides so JSON beats env at startup.
+- **Voice mode** — `/voice` toggles a background `@work(exclusive=False)` loop (`_run_voice_loop`) that cycles mic record → `VoiceEngine.transcribe` → the normal `_handle_user_message` chat pipeline → `VoiceEngine.speak`. Fully local (no network calls). A `threading.Event` (`_voice_stop_event`) is set the instant `/voice` toggles off so the blocking `record()` call returns immediately instead of running out the silence/max-duration timeout.
 
 For sequence diagrams and the full state machines, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
 
@@ -102,6 +104,7 @@ Runtime deps are declared in [pyproject.toml](pyproject.toml) `[project.dependen
 - **Backends** — `ollama`, `openai`, `huggingface_hub`, `transformers`, `accelerate`
 - **MCP + API** — `mcp` (v2 SDK — pulls in `httpx2`, `mcp-types`, `opentelemetry-api`), `fastapi`, `uvicorn[standard]`
 - **Tools** — `Pillow` (image handling), `httpx` / `requests` / `aiohttp`, `beautifulsoup4`, `ddgs`, `wikipedia`, `arxiv`, `googlesearch_python`, `stackapi`, `search_engine_parser`, `gnews`, `newspaper4k`
+- **Voice (optional)** — `[project.optional-dependencies].voice`: `pyaudio` (needs the PortAudio system library — `brew install portaudio` on macOS), `webrtcvad-wheels`, `faster-whisper`, `piper-tts`, `simpleaudio`. Install via `pip install -e '.[voice]'`.
 - **Python < 3.11 only** — `exceptiongroup`
 
 ## Run
@@ -175,4 +178,4 @@ Tests under `tests/` cover: `AgentPool` scaffolding (lookup, `${VAR}` expansion,
 
 ## Commands (in-app)
 
-`/help`, `/models [name]`, `/model large|small`, `/model set-large|set-small <name>`, `/config`, `/context`, `/servers add|list|remove|default|switch|use-model`, `/mcp add|list|remove`, `/mode [ask|agent|chat|plan]`, `/profile list|load|create`, `/sessions [list|switch|delete|rename|purge]`, `/workspace list|set|unset`, `/offline`, `/dry-run`, `/clear`, `/home`, `Ctrl+Q`, `Ctrl+L`, `Ctrl+Y`
+`/help`, `/models [name]`, `/model large|small`, `/model set-large|set-small <name>`, `/config`, `/context`, `/servers add|list|remove|default|switch|use-model`, `/mcp add|list|remove`, `/mode [ask|agent|chat|plan]`, `/profile list|load|create`, `/sessions [list|switch|delete|rename|purge]`, `/workspace list|set|unset`, `/offline`, `/dry-run`, `/voice`, `/clear`, `/home`, `Ctrl+Q`, `Ctrl+L`, `Ctrl+Y`
diff --git a/README.md b/README.md
@@ -37,13 +37,14 @@ Concretely, that means:
 
 ## Features
 
-- **Multi-backend support** — Ollama, OpenAI, HuggingFace (remote or local), and Transformers (local GPU/CPU). Switch at runtime.
-- **Agent pooling (optional)** — with `--use-pool`, the root agent can fan tasks out concurrently to vendor-agnostic sub-agents defined in [agents.yaml](agents.yaml) via a `dispatch` tool. Each pool entry binds a model _and_ a backend, so dispatch decisions are also compute-location decisions — a frontier model can plan while sensitive work stays on a local model, or a local root can fan out to faster remote SLMs for latency-sensitive tool calls.
+- **Declerative sub-agent pooling (optional)** — with `--use-pool`, the root agent can fan tasks out concurrently to vendor-agnostic sub-agents defined in an optional [agents.yaml](agents.yaml) file via a `dispatch` tool. Each pool entry binds a model _and_ a backend, so dispatch decisions are also compute-location decisions — a frontier model can plan while sensitive work stays on a local model, or a local root can fan out to faster remote SLMs for latency-sensitive tool calls.
 - **Agent profiles** — drop-in system prompts with permission manifests, base-profile inheritance, and auto-generated profiles via `/profile create`. Bundled profiles: `default`, `coder`, `reviewer`, `writer`, `planner`, `researcher`, `analyst`.
 - **Rich built-in tool set** — file ops, shell access, web search/fetch, Wikipedia/GitHub/arXiv search, Git, task tracking, reasoning scratchpad, notebook, and more. Sandbox-locked with shell allowlists, SSRF protection, and sensitive-file gating.
 - **Permission system** — write operations and sensitive reads require user approval. Session grants, workspace scoping, and profile-level allow/deny lists.
 - **OpenAI-compatible API server** — run the same agent harness behind `/v1/models` and `/v1/chat/completions` (streaming + non-streaming) so any workflow that speaks the OpenAI wire protocol (the `openai` Python SDK, curl, or plain REST) can drive the agent.
+- **Multi-backend support** — Ollama, OpenAI, HuggingFace (remote or local), and Transformers (local GPU/CPU). Switch at runtime.
 - **MCP integration** — add stdio or HTTP MCP servers at runtime for custom tools.
+- **Voice mode (optional)** — `/voice` toggles a hands-free mic → STT → LLM → TTS loop (faster-whisper, Piper TTS, WebRTC VAD) for the TUI. Fully local; requires the `voice` extras and a downloaded Piper model.
 
 ## Roadmap / areas of active exploration
 
@@ -128,10 +129,30 @@ can pick up where you left off.
 | `/workspace list\|set\|unset`                            | Manage workspace directory                                                               |
 | `/offline`                                               | Toggle offline mode                                                                      |
 | `/dry-run`                                               | Toggle dry-run mode                                                                      |
+| `/voice`                                                 | Toggle voice mode (mic → STT → LLM → TTS)                                                |
 | `/clear`                                                 | Clear the conversation                                                                   |
 | `/home`                                                  | Return to the home screen                                                                |
 | `Ctrl+Q` / `Ctrl+L` / `Ctrl+Y`                           | Quit / Clear / Copy last message                                                         |
+## Voice mode
+
+`/voice` toggles a hands-free loop: listen on the mic (WebRTC VAD auto-detects speech/silence), transcribe with faster-whisper, send the text through the normal chat pipeline, then speak the response back with Piper TTS. Everything runs locally — no network calls.
+
+Install the extras and the PortAudio system library (required to build `pyaudio`):
+
+```bash
+brew install portaudio        # macOS; use your distro's package manager on Linux
+pip install -e '.[voice]'
+```
+
+Download a Piper voice model (one-time):
+
+```bash
+wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx
+wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json
+export OLI_VOICE_PIPER_MODEL=/path/to/en_US-lessac-medium.onnx
+```
 
+Type `/voice` again (or `Ctrl+Q` to quit the app) to exit voice mode — the mic loop is interrupted immediately rather than waiting for the current recording to time out. All voice settings are configurable via env vars, `.env`, or the `/config` screen: `OLI_VOICE_WHISPER_MODEL` (default `base`), `OLI_VOICE_PIPER_MODEL`, plus VAD tunables (`OLI_VOICE_SAMPLE_RATE`, `OLI_VOICE_FRAME_DURATION_MS`, `OLI_VOICE_VAD_AGGRESSIVENESS`, `OLI_VOICE_SILENCE_TIMEOUT_MS`, `OLI_VOICE_MAX_RECORD_SECONDS`). See [docs/CONFIGURE.md](docs/CONFIGURE.md) for the full table.
 ## Documentation
 
 | Document                                       | Contents                                                                        |

diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
@@ -325,6 +325,68 @@ Tool Call Generated by Agent
 
 ---
 
+## Voice Mode Loop
+
+`/voice` toggles a background `@work(exclusive=False)` worker (`_run_voice_loop`)
+that runs alongside the normal `_generate_response` worker. All audio I/O is
+blocking and offloaded via `asyncio.to_thread` so the Textual event loop never
+stalls. Everything is local — no network calls.
+
+```
+/voice (typed)
+    │
+    ▼
+_handle_voice()
+    │
+    ├─ voice OFF → ON: create VoiceEngine (lazy), threading.Event, start
+    │              _run_voice_loop() as a background worker
+    │
+    └─ voice ON → OFF: set _voice_stop_event ─────────────┐
+                                                            │
+                                                            ▼
+┌───────────────────────────────────────────────────────────────────┐
+│ _run_voice_loop()  (while self._voice_active)                     │
+│                                                                     │
+│  mount "🎙 Listening…" ──▶ engine.record(stop_event)               │
+│         │                        │                                │
+│         │           stop_event.set() mid-recording ──▶ return None │
+│         │                        │ (checked every audio frame)     │
+│         ▼                        ▼                                │
+│  remove listening widget   audio_path or None                     │
+│         │                        │                                │
+│         │              None (silence) ──▶ loop back                │
+│         ▼                                                          │
+│  engine.transcribe(audio_path)  (faster-whisper)                  │
+│         │                                                          │
+│         ▼                                                          │
+│  _handle_user_message(text)  ──▶ normal chat pipeline               │
+│         │                        (_generate_response, tool loop)   │
+│         ▼                                                          │
+│  poll self.agent.generating until False                           │
+│         │                                                          │
+│         ▼                                                          │
+│  engine.speak(last_assistant_message)  (Piper TTS)                │
+│         │                                                          │
+│         └──▶ loop back to "Listening…" (if still active)          │
+└───────────────────────────────────────────────────────────────────┘
+```
+
+Key points:
+
+- **Immediate interruption** — `VoiceEngine.record()` polls the shared
+  `threading.Event` every audio frame, so toggling `/voice` off during an
+  in-progress recording returns `None` right away instead of waiting out the
+  silence/max-duration timeout (`SILENCE_TIMEOUT_MS` / `MAX_RECORD_SECONDS`).
+- **Shares the normal pipeline** — transcribed text is fed through
+  `_handle_user_message`, so tool calls, permission prompts, and session
+  persistence all behave exactly as they do for typed input.
+- **Lazy model loading** — `VoiceEngine.load()` (Whisper, Piper, WebRTC VAD)
+  only runs on first activation, keeping TUI startup fast.
+- **Also stopped on app exit** — `on_unmount` clears `_voice_active` and sets
+  the stop event so a lingering recording doesn't block shutdown.
+
+---
+
 ## State Machine: Chat Session
 
 ```

diff --git a/docs/CONFIGURE.md b/docs/CONFIGURE.md
@@ -54,6 +54,13 @@ If `~/.config/oli/settings.json` does not exist, it is auto-created on first loa
 | `offline_mode`               | `true`                                 | `OLI_OFFLINE_MODE`            | Block network access for web tools and MCP servers                                           |
 | `log_file`                   | `logs/backend.ndjson`                  | `OLI_LOG_FILE`                | Path for NDJSON backend log file                                                             |
 | `log_level`                  | `INFO`                                 | `OLI_LOG_LEVEL`               | Logging level: `DEBUG`, `INFO`, `WARNING`, `ERROR`                                           |
+| `voice_whisper_model`        | `base`                                 | `OLI_VOICE_WHISPER_MODEL`     | faster-whisper model size: `tiny`/`base`/`small`/`medium`/`large`                            |
+| `voice_piper_model`          | `en_US-lessac-medium.onnx`             | `OLI_VOICE_PIPER_MODEL`       | Path to the local Piper TTS `.onnx` model file                                               |
+| `voice_sample_rate`          | `16000`                                | `OLI_VOICE_SAMPLE_RATE`       | Mic sample rate in Hz (required by WebRTC VAD)                                               |
+| `voice_frame_duration_ms`    | `30`                                   | `OLI_VOICE_FRAME_DURATION_MS` | VAD frame size; must be `10`, `20`, or `30`                                                  |
+| `voice_vad_aggressiveness`   | `2`                                    | `OLI_VOICE_VAD_AGGRESSIVENESS`| WebRTC VAD aggressiveness `0`–`3`; higher rejects more noise                                 |
+| `voice_silence_timeout_ms`   | `800`                                  | `OLI_VOICE_SILENCE_TIMEOUT_MS`| Stop recording after this much consecutive silence                                           |
+| `voice_max_record_seconds`   | `15`                                   | `OLI_VOICE_MAX_RECORD_SECONDS`| Hard cap on a single recording                                                               |
 
 ## Quick examples