From 09e93ddff42136072d259eb80809d96be32a83e8 Mon Sep 17 00:00:00 2001 From: Lukas Frank Date: Mon, 24 Aug 2026 10:39:20 +0200 Subject: [PATCH 1/2] Added docs for machine eviction Signed-off-by: Lukas Frank --- .../architecture/machine-pool-eviction.md | 75 +++++++++++++++++++ docs/iaas/architecture/machine-pool-health.md | 4 - 2 files changed, 75 insertions(+), 4 deletions(-) create mode 100644 docs/iaas/architecture/machine-pool-eviction.md diff --git a/docs/iaas/architecture/machine-pool-eviction.md b/docs/iaas/architecture/machine-pool-eviction.md new file mode 100644 index 0000000..32280aa --- /dev/null +++ b/docs/iaas/architecture/machine-pool-eviction.md @@ -0,0 +1,75 @@ +# Machine Eviction + +Machine eviction lets an operator gracefully remove all `Machine`s from a `MachinePool`. This feature can be used for +example before a backing server goes into maintenance. Without it, the host is shut down while the +`Machine` objects in the API still appear to be running, and there is no way to signal that the VMs +on the pool should be shut down in an ordered fashion first. + +Eviction is modeled after the Kubernetes taint eviction pattern: setting a `NoExecute` taint on a +`MachinePool` triggers deletion of every bound `Machine` that does not tolerate the taint. The +provider then handles graceful VM shutdown via its existing finalizer before the object is fully +removed. + +The mechanism is specified in +[IEP-21: Machine Eviction](https://github.com/ironcore-dev/enhancements/blob/main/ieps/21-machine-eviction.md) +and involves two parties: +1. **The taint eviction controller** in the IronCore control plane, which watches `MachinePool` + taint changes and issues `DELETE` on bound `Machine`s that do not tolerate a `NoExecute` taint. +2. **The `machinepoollet`**, which observes the resulting `deletionTimestamp`, drives the provider + to shut the VM down, and then releases the object by removing its finalizer. + +## Taint Effects + +Eviction is driven by the taint effect set on a `MachinePool`: + +- `NoSchedule` — prevents new `Machine`s from being scheduled onto the pool. Existing machines are + left untouched. +- `NoExecute` — additionally signals that machines already bound to the pool must be deleted unless + they tolerate the taint. + +A `Machine` **tolerates** a taint when its `spec.tolerations` contains an entry matching the taint's +`key`, `value`, and `effect`. Tolerating machines are retained; all others are evicted. + +The two effects compose into a drain pattern: first cordon the pool with `NoSchedule` so no new +work lands on it, then evict the running machines with `NoExecute`. + +Tainting a `MachinePool` for eviction looks like this: + +```yaml +apiVersion: compute.ironcore.dev/v1alpha1 +kind: MachinePool +metadata: + name: my-machinepool +spec: + taints: + - key: maintenance + value: "true" + effect: NoExecute +``` + +## Eviction Flow + +Every `Machine` carries a finalizer set by the `machinepoollet`: + +```yaml +metadata: + finalizers: + - machinepoollet.ironcore.dev/machine +``` + +This finalizer ensures the object is not removed from the API until the poollet has confirmed the +VM is shut down. Deleting a `Machine` therefore blocks on graceful shutdown rather than dropping the +object immediately. + +When a `NoExecute` taint is added to a `MachinePool`, eviction proceeds as follows: + +1. The `NoExecute` taint is added to the `MachinePool`. +2. The taint eviction controller issues `DELETE` on every bound `Machine` that has no matching + toleration. +3. The `Machine` receives a `deletionTimestamp`; the finalizer blocks its removal. +4. The `machinepoollet` reconciles the `Machine`: + 1. Calls the provider to shut down and delete the VM. + 2. Removes its finalizer. +5. The API server removes the `Machine` object once the finalizer is gone. + +Machines that tolerate the taint skip steps 2–5 and remain running on the pool. diff --git a/docs/iaas/architecture/machine-pool-health.md b/docs/iaas/architecture/machine-pool-health.md index 9d819eb..8e16e8e 100644 --- a/docs/iaas/architecture/machine-pool-health.md +++ b/docs/iaas/architecture/machine-pool-health.md @@ -116,10 +116,6 @@ lifecycle controller detects the change and the poollet resumes publishing `Read This mirrors the Kubernetes node lifecycle controller behavior and provides the foundation for subsequent features like pool rolling and workload eviction, which rely on accurate pool health information. -::: warning -The IronCore scheduler currently does **not** consider the `Ready` condition when placing `Machine`s on a `MachinePool`. -Workloads can therefore still be assigned to an unhealthy pool, where they remain unprocessed until the `machinepoollet` recovers. -::: ## Configuration and Defaults From 9997c92bd5698f4d29de18fce2005448026d0f46 Mon Sep 17 00:00:00 2001 From: Lukas Frank Date: Tue, 25 Aug 2026 11:17:41 +0200 Subject: [PATCH 2/2] PR Review Signed-off-by: Lukas Frank --- docs/.vitepress/config.mts | 1 + .../{machine-pool-eviction.md => machine-eviction.md} | 0 2 files changed, 1 insertion(+) rename docs/iaas/architecture/{machine-pool-eviction.md => machine-eviction.md} (100%) diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index 4d03a52..803afd0 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -98,6 +98,7 @@ export default withMermaid({ collapsed: true, items: [ { text: 'MachinePool Health', link: '/iaas/architecture/machine-pool-health' }, + { text: 'Machine Eviction', link: '/iaas/architecture/machine-eviction' }, ], }, { text: 'Networking', link: '/iaas/architecture/networking' }, diff --git a/docs/iaas/architecture/machine-pool-eviction.md b/docs/iaas/architecture/machine-eviction.md similarity index 100% rename from docs/iaas/architecture/machine-pool-eviction.md rename to docs/iaas/architecture/machine-eviction.md