Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 131 additions & 43 deletions .github/workflows/build_documentdb_images.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
name: RELEASE - Build DocumentDB Candidate Images

# Builds documentdb extension and gateway images from public DocumentDB release artifacts.
# - documentdb image: public deb13 PostgreSQL 18 extension package
# Builds DocumentDB extension and gateway images from released DocumentDB sources.
# - documentdb image: Debian 13 PostgreSQL 18 package built from source for 0.116+
# (older releases use their published package)
# - gateway image: public documentdb-local image payload
# These images follow the DATABASE version track (documentDbVersion in values.yaml).
# For operator/sidecar images, see build_operator_images.yml.
Expand All @@ -10,11 +11,11 @@ on:
workflow_dispatch:
inputs:
version:
description: 'Released DocumentDB version to package (for example 0.113.0)'
description: 'Released DocumentDB version to build (for example 0.113.0)'
required: false
default: '0.113.0'
documentdb_extension_github_repo:
description: 'GitHub owner/repo for DocumentDB extension releases'
description: 'GitHub owner/repo containing DocumentDB source and releases'
required: false
default: 'documentdb/documentdb'
documentdb_gateway_image_repo:
Expand All @@ -31,23 +32,23 @@ permissions:
id-token: write

env:

DEFAULT_DOCUMENTDB_VERSION: '0.113.0'
DOCUMENTDB_EXTENSION_GITHUB_REPO: ${{ github.event.inputs.documentdb_extension_github_repo || 'documentdb/documentdb' }}
DOCUMENTDB_SOURCE_GITHUB_REPO: ${{ github.event.inputs.documentdb_extension_github_repo || 'documentdb/documentdb' }}
DOCUMENTDB_GATEWAY_IMAGE_REPO: ${{ github.event.inputs.documentdb_gateway_image_repo || 'ghcr.io/documentdb/documentdb/documentdb-local' }}


jobs:
# ---------------------------------------------------------------------------
# Resolve public release artifacts
# Resolve and pin released DocumentDB sources
# ---------------------------------------------------------------------------
resolve-public-artifacts:
name: Resolve Public DocumentDB Sources
resolve-sources:
name: Resolve DocumentDB Sources
runs-on: ubuntu-22.04
outputs:
documentdb_version: ${{ steps.version.outputs.documentdb_version }}
documentdb_version_dash: ${{ steps.version.outputs.documentdb_version_dash }}
image_tag: ${{ steps.version.outputs.image_tag }}
source_ref: ${{ steps.version.outputs.source_ref }}
source_sha: ${{ steps.source.outputs.source_sha }}
gateway_source_image: ${{ steps.version.outputs.gateway_source_image }}
steps:
- name: Resolve released DocumentDB version
Expand All @@ -61,48 +62,62 @@ jobs:
VERSION="$RAW_VERSION"
fi
if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Version must use dotted semver format (for example 0.113.0), got: $RAW_VERSION" >&2
echo "Version must use dotted semver format (for example 0.116.0), got: $RAW_VERSION" >&2
exit 1
fi
VERSION_DASH=$(echo "$VERSION" | sed -E 's/^([0-9]+\.[0-9]+)\.([0-9]+)$/\1-\2/')
SOURCE_REF="v${VERSION_DASH}"
SHORT_SHA=$(echo "$GITHUB_SHA" | cut -c1-7)
IMAGE_TAG="${VERSION}-build-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${SHORT_SHA}"
GATEWAY_SOURCE_IMAGE="${{ env.DOCUMENTDB_GATEWAY_IMAGE_REPO }}:pg17-${VERSION}"

{
echo "documentdb_version=$VERSION"
echo "documentdb_version_dash=$VERSION_DASH"
echo "image_tag=$IMAGE_TAG"
echo "source_ref=$SOURCE_REF"
echo "gateway_source_image=$GATEWAY_SOURCE_IMAGE"
} >> "$GITHUB_OUTPUT"

echo "DocumentDB version: $VERSION"
echo "Release tag: v$VERSION_DASH"
echo "DocumentDB source: ${{ env.DOCUMENTDB_SOURCE_GITHUB_REPO }}@$SOURCE_REF"
echo "Candidate image tag: $IMAGE_TAG"
echo "Gateway source image: $GATEWAY_SOURCE_IMAGE"

- name: Verify public extension release assets
- name: Resolve source ref to immutable commit
id: source
env:
VERSION_DASH: ${{ steps.version.outputs.documentdb_version_dash }}
SOURCE_REF: ${{ steps.version.outputs.source_ref }}
shell: bash
run: |
set -euo pipefail
for ARCH in amd64 arm64; do
ASSET_URL="https://github.com/${{ env.DOCUMENTDB_EXTENSION_GITHUB_REPO }}/releases/download/v${VERSION_DASH}/deb13-postgresql-18-documentdb_${VERSION_DASH}_${ARCH}.deb"
echo "Checking $ASSET_URL"
curl -fsI -L "$ASSET_URL" >/dev/null
done
REMOTE="https://github.com/${{ env.DOCUMENTDB_SOURCE_GITHUB_REPO }}.git"
REFS=$(git ls-remote "$REMOTE" \
"$SOURCE_REF" \
"refs/heads/$SOURCE_REF" \
"refs/tags/$SOURCE_REF" \
"refs/tags/$SOURCE_REF^{}")
PEELED_SHA=$(echo "$REFS" | awk '$2 ~ /\^\{\}$/ { print $1; exit }')
DIRECT_SHA=$(echo "$REFS" | awk 'NR == 1 { print $1 }')
SOURCE_SHA="${PEELED_SHA:-$DIRECT_SHA}"
if [[ ! "$SOURCE_SHA" =~ ^[0-9a-f]{40}$ ]]; then
echo "Unable to resolve ${{ env.DOCUMENTDB_SOURCE_GITHUB_REPO }}@$SOURCE_REF to a commit" >&2
exit 1
fi
echo "source_sha=$SOURCE_SHA" >> "$GITHUB_OUTPUT"
echo "Pinned DocumentDB source commit: $SOURCE_SHA"

- name: Verify public gateway source image
env:
SOURCE_IMAGE: ${{ steps.version.outputs.gateway_source_image }}
run: |
set -euo pipefail
docker manifest inspect "$SOURCE_IMAGE" >/dev/null
run: docker manifest inspect "$SOURCE_IMAGE" >/dev/null

# ---------------------------------------------------------------------------
# Build and push documentdb + gateway images (per-arch)
# Build source packages and images on native runners
# ---------------------------------------------------------------------------
build-and-push:
name: Build and Push ${{ matrix.image.name }} (${{ matrix.arch }})
needs: [resolve-public-artifacts]
needs: [resolve-sources]
strategy:
matrix:
arch: [amd64, arm64]
Expand All @@ -118,22 +133,94 @@ jobs:
runner: ubuntu-22.04-arm
runs-on: ${{ matrix.runner }}
env:
IMAGE_TAG: ${{ needs.resolve-public-artifacts.outputs.image_tag }}
IMAGE_TAG: ${{ needs.resolve-sources.outputs.image_tag }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
persist-credentials: false

- name: Download public extension package
- name: Checkout pinned DocumentDB source
if: matrix.image.name == 'documentdb'
uses: actions/checkout@v4
with:
repository: ${{ env.DOCUMENTDB_SOURCE_GITHUB_REPO }}
ref: ${{ needs.resolve-sources.outputs.source_sha }}
path: documentdb-source
persist-credentials: false

- name: Verify source version
if: matrix.image.name == 'documentdb'
working-directory: documentdb-source
env:
EXPECTED_VERSION: ${{ needs.resolve-sources.outputs.documentdb_version_dash }}
shell: bash
run: |
set -euo pipefail
CONTROL_FILE="pg_documentdb_core/documentdb_core.control"
SOURCE_VERSION=$(sed -nE "s/^default_version[[:space:]]*=[[:space:]]*'([^']+)'.*/\1/p" "$CONTROL_FILE")
if [[ "$SOURCE_VERSION" != "$EXPECTED_VERSION" ]]; then
echo "DocumentDB source declares version '$SOURCE_VERSION', expected '$EXPECTED_VERSION'" >&2
exit 1
fi

- name: Prepare and validate Debian 13 PostgreSQL 18 package
if: matrix.image.name == 'documentdb'
working-directory: documentdb-source
env:
DOCUMENTDB_VERSION: ${{ needs.resolve-sources.outputs.documentdb_version }}
DOCUMENTDB_VERSION_DASH: ${{ needs.resolve-sources.outputs.documentdb_version_dash }}
EXPECTED_ARCH: ${{ matrix.arch }}
shell: bash
run: |
set -euo pipefail
mkdir -p packages
DEB_FILE="deb13-postgresql-18-documentdb_${{ needs.resolve-public-artifacts.outputs.documentdb_version_dash }}_${{ matrix.arch }}.deb"
ASSET_URL="https://github.com/${{ env.DOCUMENTDB_EXTENSION_GITHUB_REPO }}/releases/download/v${{ needs.resolve-public-artifacts.outputs.documentdb_version_dash }}/${DEB_FILE}"
curl -fsSL -o "packages/${DEB_FILE}" -L "$ASSET_URL"
ls -lh packages/
if dpkg --compare-versions "$DOCUMENTDB_VERSION" ge "0.116.0"; then
BUILD_ARGS=(
--os deb13
--pg 18
--version "$DOCUMENTDB_VERSION"
--output-dir packages
)
if grep -q -- '--no-dbgsym' packaging/build_packages.sh; then
BUILD_ARGS+=(--no-dbgsym)
fi
./packaging/build_packages.sh "${BUILD_ARGS[@]}"
else
mkdir -p packages
DEB_FILE="deb13-postgresql-18-documentdb_${DOCUMENTDB_VERSION_DASH}_${EXPECTED_ARCH}.deb"
ASSET_URL="https://github.com/${DOCUMENTDB_SOURCE_GITHUB_REPO}/releases/download/v${DOCUMENTDB_VERSION_DASH}/${DEB_FILE}"
echo "Source does not support Debian 13 builds; downloading $ASSET_URL"
curl -fsSL -o "packages/${DEB_FILE}" -L "$ASSET_URL"
fi

DEB_FILE="packages/deb13-postgresql-18-documentdb_${DOCUMENTDB_VERSION_DASH}_${EXPECTED_ARCH}.deb"
if [[ ! -f "$DEB_FILE" ]]; then
echo "Expected package was not produced: $DEB_FILE" >&2
find packages -maxdepth 1 -type f -printf '%f\n' >&2
exit 1
fi

PACKAGE_NAME=$(dpkg-deb -f "$DEB_FILE" Package)
PACKAGE_VERSION=$(dpkg-deb -f "$DEB_FILE" Version)
PACKAGE_ARCH=$(dpkg-deb -f "$DEB_FILE" Architecture)
[[ "$PACKAGE_NAME" == "postgresql-18-documentdb" ]] || {
echo "Unexpected package name: $PACKAGE_NAME" >&2
exit 1
}
[[ "$PACKAGE_VERSION" == "$DOCUMENTDB_VERSION_DASH" ]] || {
echo "Unexpected package version: $PACKAGE_VERSION" >&2
exit 1
}
[[ "$PACKAGE_ARCH" == "$EXPECTED_ARCH" ]] || {
echo "Unexpected package architecture: $PACKAGE_ARCH" >&2
exit 1
}

mkdir -p "$GITHUB_WORKSPACE/packages"
cp "$DEB_FILE" "$GITHUB_WORKSPACE/packages/"

cd "$GITHUB_WORKSPACE"
rm -rf documentdb-source

- name: Login to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
Expand All @@ -148,13 +235,13 @@ jobs:

case "${{ matrix.image.name }}" in
documentdb)
DEB_FILE="deb13-postgresql-18-documentdb_${{ needs.resolve-public-artifacts.outputs.documentdb_version_dash }}_${{ matrix.arch }}.deb"
DEB_FILE="deb13-postgresql-18-documentdb_${{ needs.resolve-sources.outputs.documentdb_version_dash }}_${{ matrix.arch }}.deb"
echo "Using deb: $DEB_FILE"
BUILD_ARGS="--build-arg PG_MAJOR=18 --build-arg DEB_PACKAGE_REL_PATH=packages/$DEB_FILE"
;;
gateway)
echo "Using public gateway source image: ${{ needs.resolve-public-artifacts.outputs.gateway_source_image }}"
BUILD_ARGS="--build-arg SOURCE_IMAGE=${{ needs.resolve-public-artifacts.outputs.gateway_source_image }}"
echo "Using public gateway source image: ${{ needs.resolve-sources.outputs.gateway_source_image }}"
BUILD_ARGS="--build-arg SOURCE_IMAGE=${{ needs.resolve-sources.outputs.gateway_source_image }}"
;;
esac

Expand All @@ -172,9 +259,9 @@ jobs:
matrix:
image: [documentdb, gateway]
runs-on: ubuntu-22.04
needs: [resolve-public-artifacts, build-and-push]
needs: [resolve-sources, build-and-push]
env:
IMAGE_TAG: ${{ needs.resolve-public-artifacts.outputs.image_tag }}
IMAGE_TAG: ${{ needs.resolve-sources.outputs.image_tag }}
steps:
- name: Login to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
Expand Down Expand Up @@ -211,21 +298,22 @@ jobs:
summary:
name: Build Summary
runs-on: ubuntu-22.04
needs: [resolve-public-artifacts, create-manifest]
needs: [resolve-sources, create-manifest]
if: always()
steps:
- name: Summary
run: |
{
echo "## DocumentDB Image Build Summary"
echo ""
echo "- **DocumentDB Version**: \`${{ needs.resolve-public-artifacts.outputs.documentdb_version }}\`"
echo "- **Candidate Image Tag**: \`${{ needs.resolve-public-artifacts.outputs.image_tag }}\`"
echo "- **Extension Package Source**: \`https://github.com/${{ env.DOCUMENTDB_EXTENSION_GITHUB_REPO }}/releases/download/v${{ needs.resolve-public-artifacts.outputs.documentdb_version_dash }}/deb13-postgresql-18-documentdb_${{ needs.resolve-public-artifacts.outputs.documentdb_version_dash }}_{amd64,arm64}.deb\`"
echo "- **Gateway Source Image**: \`${{ needs.resolve-public-artifacts.outputs.gateway_source_image }}\`"
echo "- **DocumentDB Version**: \`${{ needs.resolve-sources.outputs.documentdb_version }}\`"
echo "- **DocumentDB Source**: \`${{ env.DOCUMENTDB_SOURCE_GITHUB_REPO }}@${{ needs.resolve-sources.outputs.source_ref }}\`"
echo "- **Pinned Source Commit**: \`${{ needs.resolve-sources.outputs.source_sha }}\`"
echo "- **Candidate Image Tag**: \`${{ needs.resolve-sources.outputs.image_tag }}\`"
echo "- **Gateway Source Image**: \`${{ needs.resolve-sources.outputs.gateway_source_image }}\`"
echo "- **Images**: documentdb, gateway"
echo ""
echo "To release these images, run \`release_documentdb_images.yml\` with:"
echo "- candidate_version: \`${{ needs.resolve-public-artifacts.outputs.image_tag }}\`"
echo "- version: \`${{ needs.resolve-public-artifacts.outputs.documentdb_version }}\`"
echo "- candidate_version: \`${{ needs.resolve-sources.outputs.image_tag }}\`"
echo "- version: \`${{ needs.resolve-sources.outputs.documentdb_version }}\`"
} >> "$GITHUB_STEP_SUMMARY"
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### Major Features
- **Fail-fast ImageVolume capability check**: The operator now depends on the Kubernetes [ImageVolume](https://kubernetes.io/docs/concepts/storage/volumes/#image) feature to mount the DocumentDB extension into PostgreSQL pods. Instead of gating on a Kubernetes version number, the validating webhook performs a capability probe (a server-side dry-run) when a `DocumentDB` is created and **rejects the resource with an actionable error if ImageVolume is unavailable**, so you find out immediately instead of waiting for pods that never become ready. ImageVolume is GA (on by default) in Kubernetes **1.35+**; on **1.33/1.34** it is beta and must be enabled via the `ImageVolume` feature gate on a containerd/CRI-O runtime. The Helm chart's `kubeVersion` floor is relaxed to `>= 1.33.0-0` accordingly. See [Before you start](docs/operator-public-documentation/preview/getting-started/before-you-start.md).
- **Source-built Debian 13 extension images**: Database image releases now build and validate the PostgreSQL 18 DocumentDB extension package from a pinned upstream source tag on native amd64 and arm64 runners, then use it directly to build the signed multi-architecture extension image.

## [0.3.0] - 2026-07-15

Expand Down
2 changes: 1 addition & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ After the PR is approved and merged:

Database images follow an **independent release cycle** from the operator:

1. Run **"RELEASE - Build DocumentDB Candidate Images"** (`build_documentdb_images.yml`) with the released DocumentDB `version`
1. Run **"RELEASE - Build DocumentDB Candidate Images"** (`build_documentdb_images.yml`) with the released DocumentDB `version`. For version 0.116.0 and later, the workflow builds the Debian 13 / PostgreSQL 18 extension package from pinned source before building the images.
2. Run **"RELEASE - Promote DocumentDB Images"** (`release_documentdb_images.yml`) to promote and auto-create a PR that bumps default image versions across the codebase

> **Note:** The deprecated combined workflows (`build_images.yml`, `release_images.yml`) are still available but will be removed in a future release.
Expand Down
17 changes: 9 additions & 8 deletions docs/designs/image-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ All images are published to **GitHub Container Registry (GHCR)** under `ghcr.io/

| Image | GHCR Path | Source | Dockerfile | Purpose |
|-------|-----------|--------|------------|---------|
| **documentdb** | `.../documentdb` | Public `deb13` PostgreSQL 18 package from `documentdb/documentdb` releases | `.github/dockerfiles/Dockerfile_extension` | DocumentDB PostgreSQL extension files for CNPG ImageVolume mode |
| **documentdb** | `.../documentdb` | Debian 13 / PostgreSQL 18 package built from pinned `documentdb/documentdb` source (releases before 0.116 use published assets) | `.github/dockerfiles/Dockerfile_extension` | DocumentDB PostgreSQL extension files for CNPG ImageVolume mode |
| **gateway** | `.../gateway` | Public gateway payload copied from `ghcr.io/documentdb/documentdb/documentdb-local:pg17-<version>` | `.github/dockerfiles/Dockerfile_gateway_public_image` | MongoDB wire-protocol gateway binary (Rust) |

### External Image (Not Built Here)
Expand Down Expand Up @@ -199,25 +199,26 @@ Builds operator and sidecar images from this repo's Go source.

### Database Image Build (`build_documentdb_images.yml`)

Builds documentdb extension and gateway images from public DocumentDB release artifacts.
Builds documentdb extension and gateway images from released DocumentDB source.

| Aspect | Details |
|--------|---------|
| **Trigger** | `workflow_dispatch`, `repository_dispatch` (from upstream) |
| **Images** | documentdb, gateway |
| **Dockerfiles** | `.github/dockerfiles/Dockerfile_extension`, `.github/dockerfiles/Dockerfile_gateway_public_image` |
| **Tag pattern** | `{documentdb_version}-build-{run_id}-{attempt}-{sha}` (candidate) |
| **Build time** | ~5 minutes (public artifact download + image build) |
| **Build time** | ~5 minutes (native package builds + image builds) |
| **Multi-arch** | amd64 + arm64 → multi-arch manifest |
| **Signing** | cosign keyless (OIDC) |
| **Version detection** | Workflow input / repository dispatch payload (defaults to released `0.113.0`) |

The build process:
1. Resolves the released DocumentDB version to package
2. Downloads the public `deb13` PostgreSQL 18 extension package from `documentdb/documentdb` release assets
3. Verifies the public multi-arch `documentdb-local:pg17-<version>` image exists
4. Builds `Dockerfile_extension` using the public extension `.deb` (installs pg_cron, pgvector, postgis alongside)
5. Builds `Dockerfile_gateway_public_image` by copying the gateway binary and runtime files from the public upstream image
1. Resolves the released DocumentDB version and source ref to an immutable commit
2. Builds Debian 13 PostgreSQL 18 extension packages on native amd64 and arm64 runners; releases before 0.116 use their published packages
3. Validates each package's name, version, and architecture
4. Uses each package directly to build `Dockerfile_extension` (installs pg_cron, pgvector, and postgis alongside)
5. Verifies the public multi-arch `documentdb-local:pg17-<version>` image and builds `Dockerfile_gateway_public_image` from its gateway payload
6. Creates and signs the multi-architecture extension and gateway image manifests

### Dockerfile Details

Expand Down
Loading