Skip to content

bake: add execution mode - #4044

Open
crazy-max wants to merge 5 commits into
docker:masterfrom
crazy-max:bake-execution
Open

crazy-max wants to merge 5 commits into
docker:masterfrom
crazy-max:bake-execution

Conversation

@crazy-max

@crazy-max crazy-max commented Aug 31, 2026

Copy link
Copy Markdown
Member

This carries forward the synchronized-output work from #1197 and adds explicit execution modes and target concurrency control to Bake. --execution=fail-fast remains the default, preserving cancellation of other targets when a build fails.

--execution=defer-output waits until all participating targets have successfully evaluated their build results before allowing exports to begin. This addresses the premature publication scenario in #1089, including failures discovered during evaluation rather than Dockerfile parsing. Exports are not transactional: export failures can still leave partial output, completed exports are not rolled back, and Buildx-side multi-node manifest merging and pushing remain outside the synchronization barrier. It does not combine multiple targets into a shared archive, as requested in #1668.

--execution=defer-error lets independent targets finish even when another target fails, covering the workflows in #3297 and #3428. Bake still exits with an error, but successful targets can export their outputs and retain their entries in --metadata-file. Failed multi-target builds end with an [internal] target results progress step distinguishing succeeded, failed, aborted, and unstarted targets. These outcomes are also available through --progress=rawjson; quiet mode remains silent. This partially addresses #1320, without adding its broader proposed summary of failing stages, source lines, and command logs.

--jobs=N, also available as -j=N, controls target concurrency independently of the execution mode. -j=1 runs independent targets sequentially, while 0 retains unlimited concurrency. This provides the per-invocation Bake control requested in #3989 without changing BuildKit internal step parallelism or requiring builder reconfiguration. Bake-file defaults and Compose passthrough are outside this change. With deferred output or linked targets, a nonzero limit smaller than the total participating target count is rejected to avoid deadlocks; that count includes implicit targets referenced through target: contexts.

@crazy-max crazy-max changed the title bake: add sync-output execution mode bake: add execution mode Aug 31, 2026
@crazy-max
crazy-max force-pushed the bake-execution branch 5 times, most recently from 7a76c27 to 1311cec Compare August 31, 2026 15:09
Comment thread build/build.go
Comment thread build/build.go
Comment on lines +652 to +654
if bh.Execution.Parallel < len(opts) && linkedTargets.hasLinks() {
return nil, errors.Errorf("limited parallelism is not supported with linked targets")
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rejected limited target parallelism when linked targets are present because linked target evaluation can require parent and child targets to be alive together.

Does it make sense for this first version or should linked targets be allowed when the requested limit is high enough to cover each linked group?

Comment thread commands/bake.go Outdated
@crazy-max
crazy-max marked this pull request as ready for review September 2, 2026 10:35
@crazy-max
crazy-max force-pushed the bake-execution branch 4 times, most recently from fa7a54d to d552196 Compare September 2, 2026 12:23
@crazy-max crazy-max added this to the v0.38.0 milestone Sep 2, 2026
Defer target exports until all targets successfully reach the output boundary.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Add --jobs and -j to limit concurrent Bake targets.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Report succeeded, failed, aborted, and unstarted targets through the progress stream when a multi-target
defer-error build fails. Include the summary in rawjson output while keeping quiet mode and single-target
builds unchanged.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants