diff --git a/guides/workflows/deploy-to-your-compute.mdx b/guides/workflows/deploy-to-your-compute.mdx index d79c100..c4181eb 100644 --- a/guides/workflows/deploy-to-your-compute.mdx +++ b/guides/workflows/deploy-to-your-compute.mdx @@ -131,10 +131,10 @@ docker run --rm \ Scale the number of runner containers or virtual machine instances when you want more parallelism. In Kubernetes, increase the `Deployment` replica count. In GCP or AWS, use the scaling controls of the service that runs the container, such as GKE, ECS, EKS, or an auto-scaling VM group. Each runner process connects to the same cluster and claims compatible tasks independently. -As an alternative for local testing or constrained environments, you can run multiple runner processes inside one container or shell session. Use `tilebox parallel` only for that case. +As an alternative for local testing or constrained environments, use `-n` to run multiple independent release runners inside one CLI process. ```bash -tilebox parallel -n 4 -- tilebox runner start --cluster workflow-dev +tilebox runner start -n 3 --cluster workflow-dev ``` Tilebox does not require the runner process to run in Tilebox-managed infrastructure. Use the process manager, scheduler, or container platform that fits your compute environment. diff --git a/guides/workflows/execute-tasks-in-parallel.mdx b/guides/workflows/execute-tasks-in-parallel.mdx index 5810515..f59d255 100644 --- a/guides/workflows/execute-tasks-in-parallel.mdx +++ b/guides/workflows/execute-tasks-in-parallel.mdx @@ -4,7 +4,7 @@ description: Submit multiple workflow subtasks and process them faster by runnin icon: arrows-split-up-and-left --- -Use this guide when a workflow can split work into independent tasks. You will create a small workflow that submits 20 sleep subtasks, submit one job, run it with one direct runner, and then run the same workflow with five direct runners in parallel. +Use this guide when a workflow can split work into independent tasks. You will create a small workflow that submits 20 sleep subtasks, submit one job, run it with one direct runner, and then run the same workflow with three direct runners in parallel. The example is intentionally simple. `time.sleep` stands in for real work such as downloading scenes, processing tiles, calling a model, or writing output files. @@ -124,19 +124,19 @@ uv run parallel_workflow.py The runner executes tasks, but only one after the other, and exits when no more work is available. With one runner, the sleep subtasks don't run in parallel at all. -## Run five direct runners +## Run three direct runners -Submit another job, then start five runner processes for the same workflow file. +Submit another job, then start three runner processes for the same workflow file. ```bash uv run parallel_workflow.py --submit --count 20 --seconds 5 -tilebox parallel -n 5 -- uv run parallel_workflow.py +tilebox parallel -n 3 -- uv run parallel_workflow.py ``` -This starts five direct runners. Each process registers the same task classes and asks Tilebox for work. Tilebox assigns queued tasks across the available runners, so multiple `SleepTask` instances run at the same time. +This starts three direct runners. Each process registers the same task classes and asks Tilebox for work. Tilebox assigns queued tasks across the available runners, so multiple `SleepTask` instances run at the same time. - Takeaway: use `tilebox parallel -n 5 -- uv run parallel_workflow.py` to start five local direct runners for the same workflow file. + Takeaway: use `tilebox parallel -n 3 -- uv run parallel_workflow.py` to start three local direct runners for the same workflow file. ## What to expect @@ -147,7 +147,7 @@ In the Console, you should see: - one root task that submits the subtask fan-out - many `SleepTask(index)` tasks -- multiple tasks running at overlapping times when five runners are active +- multiple tasks running at overlapping times when three runners are active - logs from each task attached to the same job For command-line inspection, query logs or spans for the job: diff --git a/workflows/concepts/runners.mdx b/workflows/concepts/runners.mdx index 3e9848f..9222df1 100644 --- a/workflows/concepts/runners.mdx +++ b/workflows/concepts/runners.mdx @@ -117,14 +117,14 @@ Release runners advertise the task identifiers from workflow releases currently Start multiple runner processes to execute tasks in parallel. Each runner process claims and executes tasks independently. You can run multiple release runners, multiple direct runners, or a mix of both in the same cluster. This increases parallelism and helps handle large workloads. -To test this, run multiple instances of the runner script in different terminal windows on your local machine, or use the [CLI](/agents-and-ai-tools/tilebox-cli) built-in `parallel` subcommand to start multiple runners in parallel. +For release runners, use `tilebox runner start -n ` to run multiple independent runners in one CLI process. For direct runners, run multiple instances of the runner script in different terminal windows or use the [CLI](/agents-and-ai-tools/tilebox-cli) built-in `parallel` subcommand. ```bash -# start multiple release runners in parallel -> tilebox parallel -n 5 -- tilebox runner start --cluster +# release runner mode +> tilebox runner start -n 3 --cluster -# or direct runner mode -> tilebox parallel -n 5 -- python your_direct_runner.py +# direct runner mode +> tilebox parallel -n 3 -- python your_direct_runner.py ``` ## Scaling