From 64eee23cf30daf52f954f070b95a485e90b2caf6 Mon Sep 17 00:00:00 2001 From: souravbiswassanto Date: Thu, 17 Sep 2026 10:24:01 +0600 Subject: [PATCH 1/2] restore.sh: take the standby parameter settings from pg_control A PITR restore replays WAL, which makes the recovery instance a standby, so PostgreSQL enforces that max_connections, max_worker_processes, max_locks_per_transaction and max_prepared_transactions are each >= the value recorded in the source's pg_control. On a KubeDB database those are normally raised through the config secret. That secret is mounted at /etc/config in the database pod, but the WAL-restore sidekick gets no such mount, so the role template's include_if_exists '/etc/config/user.conf' resolves to nothing and recovery falls back to the built-in defaults of 100 / 8 / 64 / 0. Against any database whose configuration raises them, replay dies immediately with FATAL: recovery aborted because of insufficient parameter settings DETAIL: max_connections = 100 is a lower setting than on the primary server, where its value was 200. and keeps retrying. Nothing surfaces it: the pods stay Running and the Postgres object sits in Provisioning, so the only evidence is the restorer's log. Read the values out of the restored control file instead. pg_controldata is already in the image and reports exactly what the check compares against, so this needs no knowledge of how the source was configured and cannot drag in settings the restore image cannot honour - notably shared_preload_libraries, which this script deliberately pins to a minimal list because the restore image does not carry the enterprise extensions. That is also why mounting the config secret here is the wrong fix. max_wal_senders is left alone unless the source ran with more than the 90 this script already pins. The block is appended after the role template so nothing downstream can lower the values again, and it starts with an explicit newline because the template does not end with one - without it the first setting is silently absorbed into the template's trailing comment line. Verified on a KubeDB cluster: a source with max_connections 200 and max_worker_processes 24, restored to a chosen timestamp, previously needed the values hand-appended inside the restorer pod to get past the parameter check. With this change the same restore completes unattended and lands exactly on the target - 3600 of 4200 rows, every row after the target correctly absent, latest row 25 seconds before it, and the instance promoted out of recovery. Signed-off-by: souravbiswassanto --- scripts/restore.sh | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/scripts/restore.sh b/scripts/restore.sh index 8361229..c66a2b0 100755 --- a/scripts/restore.sh +++ b/scripts/restore.sh @@ -99,6 +99,45 @@ if [[ "${TDE_ENABLED:-false}" == "true" ]]; then echo "pg_tde.cipher = '${TDE_CIPHER:-aes_128}'" >>/tmp/postgresql.conf fi cat /run_scripts/role/postgresql.conf >>/tmp/postgresql.conf +# The role template does not end in a newline, so anything appended after it would be +# swallowed by its last comment line. Start a fresh line before writing more. +printf '\n' >>/tmp/postgresql.conf + +# Replaying WAL makes this a standby, so PostgreSQL enforces that these settings are +# each >= the value recorded in the source's pg_control; otherwise recovery stops with +# FATAL: recovery aborted because of insufficient parameter settings +# On a KubeDB database these are normally raised through the config secret, which is +# mounted at /etc/config in the DB pod but NOT here -- the role template's +# `include_if_exists = '/etc/config/user.conf'` therefore resolves to nothing and we +# would fall back to the built-in defaults (100 / 8 / 64 / 0). Read the real values out +# of the restored control file instead: pg_controldata reports exactly what the check +# compares against, so this needs no knowledge of how the source was configured. +# Appended after the role template so nothing downstream can lower them again. +if command -v pg_controldata >/dev/null 2>&1; then + control_setting() { + pg_controldata "$PGDATA" 2>/dev/null | + sed -n "s/^$1 setting: *\([0-9]\+\)$/\1/p" + } + for pair in \ + "max_connections:max_connections" \ + "max_worker_processes:max_worker_processes" \ + "max_prepared_xacts:max_prepared_transactions" \ + "max_locks_per_xact:max_locks_per_transaction"; do + control_field="${pair%%:*}" + guc="${pair##*:}" + value="$(control_setting "$control_field")" + if [[ -n "$value" ]]; then + echo "$guc = $value" >>/tmp/postgresql.conf + fi + done + # max_wal_senders is already pinned to 90 above; only raise it further if the + # source ran with more, never lower it. + senders="$(control_setting max_wal_senders)" + if [[ -n "$senders" ]] && [[ "$senders" -gt 90 ]]; then + echo "max_wal_senders = $senders" >>/tmp/postgresql.conf + fi +fi + mv /tmp/postgresql.conf "$PGDATA/postgresql.conf" echo "max_replication_slots = 90" >>/tmp/postgresql.conf # setup pg_hba.conf for initial start. this one is just for initialization From 19343bf04da0fc1ce13ba986dcb99ea027f5672a Mon Sep 17 00:00:00 2001 From: souravbiswassanto Date: Fri, 18 Sep 2026 17:27:11 +0600 Subject: [PATCH 2/2] restore.sh: restate why pg_control is read, now that the config secret is mounted kubedb/postgres#937 projects the database's configSecret into the wal-restore sidekick, so the role template's include_if_exists = '/etc/config/user.conf' now resolves. The comment here claimed it never does, which stops being true the moment that lands. The block still has a job, and it is a narrower one: the config secret carries the CURRENT configuration, while replay is bound by what the source was running when the backup was taken. Restoring an older backup after the secret has been lowered still aborts, and nothing surfaces it. Reword to say that instead. Signed-off-by: souravbiswassanto --- scripts/restore.sh | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/scripts/restore.sh b/scripts/restore.sh index c66a2b0..ffd81aa 100755 --- a/scripts/restore.sh +++ b/scripts/restore.sh @@ -106,13 +106,16 @@ printf '\n' >>/tmp/postgresql.conf # Replaying WAL makes this a standby, so PostgreSQL enforces that these settings are # each >= the value recorded in the source's pg_control; otherwise recovery stops with # FATAL: recovery aborted because of insufficient parameter settings -# On a KubeDB database these are normally raised through the config secret, which is -# mounted at /etc/config in the DB pod but NOT here -- the role template's -# `include_if_exists = '/etc/config/user.conf'` therefore resolves to nothing and we -# would fall back to the built-in defaults (100 / 8 / 64 / 0). Read the real values out -# of the restored control file instead: pg_controldata reports exactly what the check -# compares against, so this needs no knowledge of how the source was configured. -# Appended after the role template so nothing downstream can lower them again. +# On a KubeDB database these are normally raised through the config secret, which the +# role template pulls in with `include_if_exists = '/etc/config/user.conf'`. That is +# the right value in the common case, but it is the CURRENT configuration, not the one +# the source was running under when the backup was taken -- so restoring an older +# backup after the secret has been lowered still aborts replay, and the operator has +# no way to know it will. +# pg_control does know: it records exactly what the check compares against. Read the +# values from there, appended after the role template so they take precedence during +# recovery. This only affects the recovery instance; the restored database starts +# again from its own configuration afterwards. if command -v pg_controldata >/dev/null 2>&1; then control_setting() { pg_controldata "$PGDATA" 2>/dev/null |