Add aws-iam-authenticator so the runner can drive a remote cluster - #323
Open
jasonopslevel wants to merge 1 commit into
Open
Add aws-iam-authenticator so the runner can drive a remote cluster#323jasonopslevel wants to merge 1 commit into
jasonopslevel wants to merge 1 commit into
Conversation
The runner already resolves its client from a kubeconfig when one is present, so targeting another cluster was always possible in principle. The image just had no way to authenticate to EKS: it is alpine plus the Go binary, with no credential helper. Adds aws-iam-authenticator, pinned by version and per-arch SHA-256. Chosen over the AWS CLI, which has no musl build, and over the AWS SDK in Go, which would put AWS specifics into a binary that otherwise has none. This way EKS support is a property of the image and the runner keeps speaking plain kubeconfig. Motivation is the agent sandbox platform: the control plane and the workloads it schedules are different trust levels, and separate clusters is a stronger boundary than separate namespaces. It is also a prerequisite for dispatching across cells, where a controller confined to its own cluster cannot address more than one. Verified: both arches build, the binary reports 0.7.20 and exposes `token -i`, and a tampered checksum fails the build rather than silently installing something else. docs/cross-cluster.md covers the kubeconfig shape, the two IAM pieces, and the gotcha that an unmapped identity returns 401 rather than 403.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The runner already resolves its client from a kubeconfig when
$KUBECONFIGisset (
clientcmd.NewNonInteractiveDeferredLoadingClientConfig), falling back toin-cluster config otherwise. So driving a cluster it doesn't run in was always
possible in principle — the image just had no way to authenticate to EKS. It's
alpineplus the Go binary, with no credential helper, so a kubeconfigexecplugin has nothing to exec.
Motivation is the agent sandbox platform. The control plane and the workloads it
schedules are different trust levels: the runner holds an OpsLevel API token and
pod-create rights, job pods run tenant-supplied work. Separate clusters is a
stronger boundary than separate namespaces. It's also a prerequisite for
dispatching across cells later — a controller confined to its own cluster can't
address more than one.
What
Adds
aws-iam-authenticator0.7.20, pinned by version and per-architectureSHA-256 from the release's published checksums.
Chosen over the alternatives deliberately:
image.
go.modcurrently has noAWS dependency at all. That would make a deliberately cloud-agnostic binary
AWS-aware.
This way EKS support is a property of the image, and the runner keeps speaking
plain kubeconfig — the same image works against any cluster whose kubeconfig you
supply.
Verified
amd64andarm64authenticator stages build{"Version":"0.7.20"}and exposestoken -isomething else — checked by mutating the SHA and confirming
sha256sum: WARNING: 1 of 1 computed checksums did NOT matchNotes for reviewers
The kubeconfig carries no credentials — a ConfigMap is fine, no secret needed.
AWS credentials come from Pod Identity or IRSA on the ServiceAccount in the
cluster the runner runs in.
aws-iam-authenticator tokenpresigns an STSrequest locally and makes no AWS API call, so the role needs no policies.
docs/cross-cluster.mdcovers the kubeconfig shape, the two IAM pieces, and thegotcha that an unmapped identity gets 401, not 403 under
authentication_mode = "API"— which reads as a credentials problem and sendsyou looking at the wrong layer.