From 15d5f769a7baabe735b3439b805666fbaf5b556e Mon Sep 17 00:00:00 2001 From: Manish Kumar Date: Mon, 31 Aug 2026 17:16:25 -0500 Subject: [PATCH] fix(celery-worker): add OLLAMA_BASE_URL + OMNIBIOAI_MYSQL_PASSWORD, matching workbench Live audit finding (2026-08-31, plugins/bio_agent post-#518 scan): celery-worker runs the exact same omnibioai-workbench image as `workbench` (only 2 services in this compose file build from that image, confirmed by grepping every `context:` line), but was missing two env vars workbench already has. - Without OLLAMA_BASE_URL, LLMService fell back to the ollama client's own default (127.0.0.1:11434) -- nothing inside this container -- so every real BioQueryAI analysis Celery task failed immediately with "Failed to connect to Ollama" as soon as it reached the LLM step. - Without OMNIBIOAI_MYSQL_PASSWORD, Workflow Planner's MySQL workflow-candidate scan (_resolve_mysql_config in workflow_planner_service.py) failed with "MySQL password missing". Traced the real fallback chain before adding this (not just the two var names the error message named): host/port/user/db all already resolve correctly via this block's existing DB_HOST and hardcoded defaults -- only password had no working fallback. Verified post-fix: sha256 of all 7 relevant source files (views.py, 5 bio_agent handlers, workflow_planner_service.py) match exactly between celery-worker and the already-verified workbench/local source. Real end-to-end BioQueryAI + Workflow Planner submissions confirm the fix itself works -- Workflow Planner's MySQL scan genuinely succeeds and produces a real multi-block plan (301s, no MySQL error); BioQueryAI's Ollama call genuinely connects and returns a real 200 (no connection error). Two NEW, separate, real bugs surfaced by getting past this blocker (not fixed here, filed as follow-ups): 1. Neither `workbench` nor `celery-worker` mounts /app/work/planner or /app/work/bioqueryai as a shared volume (unlike /app/work/runs, /app/work/objects, etc., which are). Both status-polling endpoints (planner_ask_status, bioquery_ask_status) read from that path in workbench's own container, but the Celery task that writes the completion status file runs in celery-worker's container -- so a genuinely-completed task's result is written somewhere the poller can never see it. Confirmed live: Workflow Planner's task succeeded in celery-worker's log with a full real plan, but polling from workbench showed PENDING forever. 2. BioQueryAI's Celery worker process SIGSEGV'd immediately after a successful Ollama response, losing the task result via WorkerLostError. Circumstantial cause: two large models (deepseek-r1:32b ~55GB + llama3.1:70b) loaded concurrently, host swap at 15Gi/15Gi (100%) at crash time. Root cause not confirmed -- dmesg is permission-denied in this environment, so OOM-kill correlation could not be directly verified. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01VxRzmWKquRKGwebL21d6wH --- docker-compose.yml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docker-compose.yml b/docker-compose.yml index be64651..b12430f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -793,6 +793,25 @@ services: MODEL_REGISTRY_BASE_URL: http://model-registry:8095 ORCHESTRATOR_MAX_RETRIES: "10" RAG_BASE_URL: http://rag:8096 + # Live audit finding (2026-08-31, plugins/bio_agent post-#518 scan): + # workbench has both of these already (see this file's `workbench` + # service block above); celery-worker didn't, same DB_HOST/TES_BASE_URL + # kind of parity gap as GATEWAY_URL above, just never caught until a + # real end-to-end BioQueryAI/Workflow Planner run actually reached the + # LLM/MySQL step from inside THIS container instead of workbench's. + # Without OLLAMA_BASE_URL, LLMService falls back to the ollama client's + # own default (127.0.0.1:11434), which is nothing inside this + # container -- every real BioQueryAI analysis Celery task failed with + # "Failed to connect to Ollama" as soon as it reached the LLM step. + # Without OMNIBIOAI_MYSQL_PASSWORD, Workflow Planner's MySQL + # workflow-candidate scan (_resolve_mysql_config in + # workflow_planner_service.py) failed with "MySQL password missing" -- + # host/port/user/db all already resolve correctly via this block's + # existing DB_HOST/hardcoded defaults, only password had no working + # fallback (checked _resolve_mysql_config's real fallback chain before + # adding this, not just the two vars the error message named). + OLLAMA_BASE_URL: http://ollama:11434 + OMNIBIOAI_MYSQL_PASSWORD: ${MYSQL_ROOT_PASSWORD:-omnibioai} ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-} OPENAI_API_KEY: ${OPENAI_API_KEY:-} CODE_LLM_MODEL: ${CODE_LLM_MODEL:-qwen2.5-coder:32b}