diff --git a/docs/component.md b/docs/component.md index b4fba04f..bd249eef 100644 --- a/docs/component.md +++ b/docs/component.md @@ -236,6 +236,9 @@ message: `comp.Reconcile(ctx, recCtx)` runs the following steps on every call. They match the authoritative order in the `Reconcile` GoDoc. +Every declared [data cell](#declared-data) is cleared before step 1 runs, so no value extracted during a previous +reconcile can be observed during this one. + 1. **Feature gate check.** If a feature gate is set and disabled, all managed resources are deleted and the condition is set to `True/Disabled`. No further processing occurs. A gate evaluation error sets `FeatureGateError`. 2. **Prerequisite check.** If prerequisites are registered and the initialization barrier is still active, all @@ -246,8 +249,8 @@ message: remaining steps are skipped. Guards are not evaluated during suspension. 4. **Resource reconciliation.** All non-delete resources are processed sequentially in registration order, managed or read-only alike. For each resource: its guard (if any) is evaluated and a blocked guard stops that resource and all - later ones; the resource is applied (managed) or fetched (read-only); its data extractors run immediately, making - extracted data available to subsequent resources' guards and mutations. + later ones; the resource is applied (managed) or fetched (read-only); its declared data extractions run immediately, + making the extracted values available to subsequent resources' data guards and mutations. 5. **Status aggregation.** The converging status of every processed resource is collected, including any blocked-guard result. 6. **Condition update.** A new component condition is derived from the aggregate resource status, the previous @@ -257,7 +260,8 @@ message: ```mermaid flowchart TD - Start([Reconcile]) --> Gate{Feature gate set?} + Start([Reconcile]) --> Reset[Clear declared data cells] + Reset --> Gate{Feature gate set?} Gate -->|disabled| DelAll[Delete all resources] --> Disabled([True / Disabled]) Gate -->|enabled or unset| Prereq{Barrier active
and prereqs set?} Prereq -->|unmet| NotMet([False / PrerequisiteNotMet]) @@ -270,7 +274,7 @@ flowchart TD DelMarked --> End([Return; controller calls FlushStatus]) ``` -A read-only resource registered before a managed one can extract data that feeds the managed resource's guard or +A read-only resource registered before a managed one can extract data that feeds the managed resource's data guard or mutations within the same reconcile cycle. Read-only resources that implement `ObservationRecorder` have the fetched object recorded back onto them so later inspection sees live cluster state; resources built from `generic.BaseResource` do this automatically. Managed resources are applied with Server-Side Apply and receive a controller owner reference, @@ -289,6 +293,25 @@ in a cluster-free render. `Preview` therefore returns the full desired set, incl skip behind a blocked guard, which keeps the snapshot deterministic and focused on baseline construction, mutation wiring, and registration order. +No extraction runs during a preview either, so every [data cell](#declared-data) is unset. A mutation that calls `Get` +degrades quietly and simply omits the enriched field. A mutation that calls `Require` returns an error wrapping +`concepts.ErrDataNotExtracted`, which fails the whole preview. Tests that render such a resource must seed the cell +first: + +```go +comp, dbHost, err := BuildComponent(owner) +if err != nil { + return err +} +dbHost.Set("postgres.default.svc") // stand in for the value a real reconcile would extract + +objs, err := comp.Preview() +``` + +Return the cells from your component assembly function so tests can reach them, as +[`examples/extraction-and-guards`](https://github.com/sourcehawk/operator-component-framework/tree/main/examples/extraction-and-guards) +does. + Each managed resource must implement [`concepts.Previewable`](primitives.md#lifecycle-interfaces) (`Preview()`). All built-in primitives satisfy it through `generic.BaseResource`. A custom resource must implement it to be previewable; without it, `Component.Preview` returns an error for that resource. `Preview` is the natural input for whole-component @@ -321,6 +344,9 @@ implements the same interface so version-matrix golden generation can introspect [`concepts.MutationInspector`](primitives.md#lifecycle-interfaces) for the contract and the [Testing](testing.md) guide for how it drives version-matrix goldens. +The data-flow counterpart is `concepts.DataInspector` (`DataTopology()`), which reports the declared flow of every data +cell through the component without running any extraction. See [Inspecting the topology](#inspecting-the-topology). + ### Cluster-scoped resources When a component manages cluster-scoped resources (such as `ClusterRole` or `PersistentVolume`) and the owner CRD is @@ -449,6 +475,13 @@ Resources with `DeleteOnSuspend` enabled are **not** created if already absent; suspended, which avoids a create-then-delete loop on every reconcile while the component stays suspended. Resources that are not `Suspendable` are left in place. +Guards are not evaluated during suspension, but [declared extractions](#declared-data) still run for each managed +resource in registration order, so a mutation that calls `Require()` on a cell an earlier managed resource produces +still works while the component is suspended. Cells produced by read-only resources (which are not fetched during +suspension) or by resources with `DeleteOnSuspend` (which are skipped once absent) stay absent for as long as the +component is suspended. A mutation that depends on one of those must use `Get()` rather than `Require()` if the +component can ever be suspended. + ## ReconcileContext `ReconcileContext` carries all dependencies for a reconciliation pass. Pass it from your controller on each call: @@ -515,63 +548,170 @@ them in a single write. Persisting after each component would race the component [Keep Controllers Thin](guidelines.md#keep-controllers-thin) and [One Component Per Logical Condition](guidelines.md#one-component-per-logical-condition). +## Declared Data + +Resources inside one component pass observed values to each other through **data cells**. A cell is created in the +component assembly function, written by a declared extraction on an earlier resource, and read by later resources' +guards and mutations during the same reconcile. + +```go +dbHost := concepts.NewData[string]("db-host") +``` + +`concepts.Data[T]` is named, typed, and presence-aware, which is what separates "not extracted yet" from "extracted as +the empty string". + +| Method | Returns | +| ---------------------- | ---------------------------------------------------------------------------------- | +| `Name() string` | The diagnostic name, used in guard reasons, validation errors, and topology output | +| `IsSet() bool` | Whether the cell currently holds a value | +| `Get() (T, bool)` | The value and its presence; the zero value of `T` when unset | +| `Require() (T, error)` | The value, or an error wrapping `concepts.ErrDataNotExtracted` and naming the cell | + +There is deliberately no panicking accessor: reconciler code must degrade to conditions and requeues, never crash the +manager. Cell identity is the pointer, not the name. + +`Set` and `Clear` are exported because the extraction runner and the reconcile-start reset live in other packages. +Calling them from resource code bypasses topology validation and is unsupported. Seeding a cell in a test before +[previewing](#previewing-desired-state) is the one intended manual use of `Set`. + +### Declaring a write + +Every primitive package exports an `ExtractInto` function that records "this resource produces this cell". It is a +package-level function rather than a builder method because a Go method cannot introduce the value type parameter. + +```go +configmap.ExtractInto(cmBuilder, dbHost, func(cm corev1.ConfigMap) (string, error) { + return cm.Data["db-host"], nil +}) +``` + +The function runs immediately after the resource is applied (managed) or fetched (read-only), and the framework stores +its result in the cell and marks it present. If it returns an error, the reconcile fails with that error. Extracting +several values from one object means several `ExtractInto` calls, one per cell. + +Custom resource wrappers expose the same shape by delegating to `generic.ExtractInto`; see the +[custom resource guide](custom-resource.md#5-implement-the-builder). + +### Declaring a read + +Two builder methods record "this resource reads this cell", and both accept any number of cells: + +- `WithDataGuard(cells...)` blocks the resource until every listed cell is set. See [Guards](#guards). +- `WithOptionalData(cells...)` does not gate. Use it when the resource proceeds either way and a mutation enriches the + object only when the value is there. + +Both modes are validated and both show up in the topology. The +[Guidelines](guidelines.md#use-data-extraction-and-guards-for-intra-component-dependencies) page has the table of +consumption modes and when to pick each. + +### Reset at the start of each reconcile + +`Reconcile` clears every declared cell before it does anything else. Cells created in the assembly function are already +scoped to one reconcile; the reset is a hardening so that a cell which somehow outlives its assembly function still +cannot leak a value from one pass into the next. Sharing a cell across components is unsupported, because both the reset +and the validation below are per component. + +### Build-time validation + +`Build()` walks the resources in registration order and rejects a component whose data flow cannot work. + +Every cell a resource reads must have a producer registered **strictly earlier**. A producer never satisfies its own +read, since extraction runs after mutations on the same resource: + +```text +resource "v1/Secret/default/db-credentials" reads data "db-host" but no earlier resource produces it +``` + +No two distinct cells may share a name. Pointer identity is what the checks run on; the name check exists so diagnostics +and topology output stay unambiguous: + +```text +resource "v1/ConfigMap/default/app-config" in component "database" declares data "db-host", but a distinct cell already uses that name; data names must be unique within a component +``` + +Multiple resources may produce the same cell. That is allowed, and at runtime the last registered producer's extraction +wins, because each one overwrites the cell as it runs. + +Only resources that actually reconcile participate. Declarations on resources registered with `Delete()`, +`DeleteWhen()`, or `OrphanWhen()` never run an extraction and are not considered. A resource whose `GatedBy` gate is +disabled is moved to the delete set at registration, so if it was the only producer of a cell, `Build()` fails with the +no-earlier-producer error; that is intentional, and it surfaces the broken data flow at build time rather than leaving a +reader permanently blocked at runtime. + +### Inspecting the topology + +The built component satisfies `concepts.DataInspector`. `DataTopology()` returns one `concepts.DataEdge` per declared +cell, in first-producer registration order, without running any extraction: + +```go +for _, edge := range comp.DataTopology() { + fmt.Printf("%s: produced by %v, guarded by %v, optional for %v\n", + edge.Data, edge.Producers, edge.Guarded, edge.Optional) +} +``` + +`Producers`, `Guarded`, and `Optional` hold resource identities in registration order. This is the data-flow counterpart +of [`concepts.MutationInspector`](primitives.md#lifecycle-interfaces): nothing in the reconcile path calls it, and tests +can assert a component's declared data flow the same way they assert its registered mutations. + ## Guards -Guards let resources within a component express runtime dependencies on each other. A guard is a precondition function -registered on a resource and evaluated before the resource is applied. If the guard returns `Blocked`, the resource and -all resources registered after it are skipped for that reconcile cycle. +Guards let resources within a component express runtime dependencies on each other. A guard is a precondition evaluated +before the resource is applied. If it reports `Blocked`, the resource and all resources registered after it are skipped +for that reconcile cycle. -Combined with per-resource data extraction, guards enable indirect dependency graphs: resource A is applied first, its -data extractor populates a shared variable, and resource B's guard checks that variable before allowing B to proceed. +There are two forms. A **data guard**, declared with `WithDataGuard(cells...)`, blocks until every listed +[data cell](#declared-data) holds a value; the framework writes both the guard and its reason. A **custom guard**, +registered with `WithGuard`, runs arbitrary logic against the resource object. A resource may use both: data guards are +evaluated first, and the custom guard is consulted only once every guarded cell is set. -### Registering a guard +### Blocking on declared data -Guards are registered on the resource builder with `WithGuard`. The guard receives a copy of the resource object and -returns a `concepts.GuardStatusWithReason`. The following example shows the full pattern: a first resource extracts a -value after being applied, and a second resource guards against running before that value is available. +Reach for `WithDataGuard` whenever the precondition is "an earlier resource produced this value". The example below is +the full pattern: a config source declares an extraction, and a consumer declares a guard and a mutation that read it. ```go -func buildBackendComponent(owner *v1alpha1.WebApp, endpoint *string) (*component.Component, error) { - // First resource: a config source. After it is applied, the data extractor - // reads a value from the live object into *endpoint. - configRes, err := static.NewBuilder(newBackendConfig(owner)). - WithDataExtractor(func(obj uns.Unstructured) error { - *endpoint = obj.Object["data"].(map[string]any)["endpoint"].(string) - return nil - }). - Build() +func buildBackendComponent(owner *v1alpha1.WebApp) (*component.Component, error) { + endpoint := concepts.NewData[string]("backend-endpoint") + + // First resource: a config source. Once it is applied, the declared + // extraction reads a value out of the live object and into the cell. + configBuilder := static.NewBuilder(newBackendConfig(owner)) + static.ExtractInto(configBuilder, endpoint, func(obj uns.Unstructured) (string, error) { + value, _, err := uns.NestedString(obj.Object, "data", "endpoint") + return value, err + }) + configRes, err := configBuilder.Build() if err != nil { return nil, err } - // Second resource: a consumer that needs the extracted endpoint. Its guard - // blocks until *endpoint is populated earlier in this same reconcile cycle; - // the mutation then injects the value at Mutate() time. - consumerRes, err := static.NewBuilder(newBackendConsumer(owner)). - WithGuard(func(_ uns.Unstructured) (concepts.GuardStatusWithReason, error) { - if *endpoint == "" { - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusBlocked, - Reason: "waiting for backend endpoint", - }, nil + // Second resource: a consumer that needs the endpoint. The data guard blocks + // it until the cell is set earlier in this same reconcile cycle; the mutation + // then injects the value at Mutate() time. + consumerBuilder := static.NewBuilder(newBackendConsumer(owner)) + consumerBuilder.WithDataGuard(endpoint) + consumerBuilder.WithMutation(unstruct.Mutation{ + Name: "set-endpoint", + Mutate: func(m *unstruct.Mutator) error { + value, err := endpoint.Require() + if err != nil { + return err } - return concepts.GuardStatusWithReason{Status: concepts.GuardStatusUnblocked}, nil - }). - WithMutation(unstruct.Mutation{ - Name: "set-endpoint", - Mutate: func(m *unstruct.Mutator) error { - m.EditContent(func(e *editors.UnstructuredContentEditor) error { - return e.SetNestedString(*endpoint, "spec", "endpoint") - }) - return nil - }, - }). - Build() + m.EditContent(func(e *editors.UnstructuredContentEditor) error { + return e.SetNestedString(value, "spec", "endpoint") + }) + return nil + }, + }) + consumerRes, err := consumerBuilder.Build() if err != nil { return nil, err } - // Registration order matters: the config source must be registered before the consumer. + // Registration order matters: the config source must be registered before the + // consumer, and Build() rejects the component if it is not. return component.NewComponentBuilder(). WithName("backend"). WithConditionType("BackendReady"). @@ -581,11 +721,36 @@ func buildBackendComponent(owner *v1alpha1.WebApp, endpoint *string) (*component } ``` -The guard receives the resource's object but need not use it. Guards that only check external state (closure variables -populated by prior extractors) can ignore the parameter. +The reason comes from the cells, so a blocked consumer reports `waiting for data "backend-endpoint"` without anyone +writing that string, and it cannot drift when the dependency changes. Guarding on several cells names every missing one: +`waiting for data "backend-endpoint", "api-token"`. + +### Registering a custom guard + +Use `WithGuard` for preconditions that are not "a value exists". The guard receives a copy of the resource object and +returns a `concepts.GuardStatusWithReason`. + +```go +res, err := deployment.NewBuilder(base). + WithGuard(func(_ appsv1.Deployment) (concepts.GuardStatusWithReason, error) { + if !owner.Spec.LicenseAccepted { + return concepts.GuardStatusWithReason{ + Status: concepts.GuardStatusBlocked, + Reason: "waiting for the license terms to be accepted", + }, nil + } + return concepts.GuardStatusWithReason{Status: concepts.GuardStatusUnblocked}, nil + }). + Build() +``` + +The guard receives the resource's object but need not use it, as above. Passing nil to `WithGuard` clears any previously +registered custom guard; it does not affect declared data guards. ### Guard behavior +- Data guards are evaluated before the custom guard. If any guarded cell is unset, the resource is `Blocked` with the + generated reason and the custom guard is never called. - Guards are evaluated in registration order, before each resource is applied. - When a guard returns `Blocked`, the blocked resource contributes a `Blocked` status to the component condition regardless of its participation mode, and all resources after it are skipped entirely. This override exists because a @@ -601,7 +766,7 @@ A blocked guard produces a condition like: type: BackendReady status: "False" reason: Blocked -message: "waiting for backend endpoint" +message: 'waiting for data "backend-endpoint"' ``` The `Blocked` status is not sticky. It is self-reinforcing only because the guard re-evaluates on every reconcile; when diff --git a/docs/custom-resource.md b/docs/custom-resource.md index f7f70e7e..80e97a91 100644 --- a/docs/custom-resource.md +++ b/docs/custom-resource.md @@ -2,7 +2,7 @@ This guide is for operator authors who need to manage a Kubernetes object that the [built-in primitives](primitives.md) do not cover. The built-in set handles the common kinds (Deployments, StatefulSets, ConfigMaps, Services, and more) and -is highly customizable through status handlers, suspension logic, mutations, and data extractors. Reach for a custom +is highly customizable through status handlers, suspension logic, mutations, and declared data. Reach for a custom resource only when the kind you manage has no matching primitive: - A **custom CRD** defined by your project or a third-party operator. @@ -68,8 +68,8 @@ interfaces. For the full description of each interface and the runtime string va | **Integration** | `generic.IntegrationResource` | `Operational`, `Graceful`, `Suspendable`, `Guardable`, `DataExtractable` | External-dependency objects (services, ingresses) | In addition to the category-specific interfaces, every generic resource also satisfies -[`concepts.Previewable`](primitives.md#lifecycle-interfaces) and `concepts.MutationInspector`, and your wrapper exposes -both. They are covered in [Step 6](#6-implement-the-resource). +[`concepts.Previewable`](primitives.md#lifecycle-interfaces), `concepts.MutationInspector`, `concepts.DataProducer`, and +`concepts.DataConsumer`, and your wrapper exposes all four. They are covered in [Step 6](#6-implement-the-resource). The rest of the guide uses Workload as the primary example. The pattern is identical for the other categories, with fewer handlers to implement. @@ -457,9 +457,20 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a data extractor to run after the resource is processed. -func (b *Builder) WithDataExtractor(extractor func(examplev1.MessageQueue) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the resource reads the given data cells and must not +// be applied until every one of them is set. The framework generates the guard and +// its reason; component Build validates that a producer for each cell is registered +// earlier. Data guards are evaluated before any custom guard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the resource reads the given data cells without +// gating on them. Component Build still validates that a producer is registered +// earlier, and the dependency stays visible to introspection. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -487,16 +498,40 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this MessageQueue produces the value of cell. fn +// computes the value from a copy of the reconciled MessageQueue; the framework +// stores it in the cell and marks it present, immediately after the object is +// applied or fetched. This is a package-level function because a Go method +// cannot introduce the extra type parameter V. +func ExtractInto[V any]( + b *Builder, cell *concepts.Data[V], fn func(examplev1.MessageQueue) (V, error), +) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} ``` The builder exposes `WithCustomSuspendStatus`, `WithCustomSuspendMutation`, and `WithCustomSuspendDeletionDecision` the same way if callers need to override suspension behavior after construction; they are omitted above for brevity. +Callers then use the package-level form, mirroring every built-in primitive: + +```go +replicas := concepts.NewData[int32]("queue-replicas") + +builder := messagequeue.NewBuilder(mq) +messagequeue.ExtractInto(builder, replicas, func(q examplev1.MessageQueue) (int32, error) { + return q.Status.ReadyReplicas, nil +}) +``` + !!! note "Builder conventions" - - **`generic.WrapGuard` and `generic.WrapExtractor`** convert value-receiver callbacks (`func(T)`) into the - pointer-receiver form (`func(*T)`) the generic layer expects, so your public API can take the kind by value. The - built-in builders use both. + - **`generic.WrapGuard` and `generic.WrapExtraction`** convert value-receiver callbacks (`func(T)` and + `func(T) (V, error)`) into the pointer-receiver form (`func(*T)` and `func(*T) (V, error)`) the generic layer + expects, so your public API can take the kind by value. The built-in builders use both. + - **Reach the embedded base for `ExtractInto`.** `generic.ExtractInto` takes a `*generic.BaseBuilder`, so pass + `&b.base.BaseBuilder`. Every category builder embeds it. - **Register defaults in the constructor.** Set the handlers your CRD has meaningful semantics for, then let callers override them per resource. - **Return `*Builder` from every method** for fluent chaining. @@ -531,6 +566,8 @@ import ( // - concepts.Suspendable (DeleteOnSuspend, Suspend, SuspensionStatus) // - concepts.Guardable (GuardStatus) // - concepts.DataExtractable (ExtractData) +// - concepts.DataProducer (ProducedData) +// - concepts.DataConsumer (ConsumedData) // - concepts.ObservationRecorder (RecordObservation) // - concepts.Previewable (Preview) // - concepts.MutationInspector (RegisteredMutations, FiringSet) @@ -578,6 +615,16 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this resource declares extractions into. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the resource's declared data reads, blocking and optional alike. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) } @@ -600,6 +647,8 @@ func (r *Resource) FiringSet() ([]string, error) { // Compile-time guarantee that the wrapper exposes the inspection surface. var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) ``` !!! warning "Do not omit `Preview`" @@ -611,18 +660,24 @@ var _ concepts.MutationInspector = (*Resource)(nil) them, but [version-matrix golden generation](testing.md) uses them to introspect which mutations a resource registers and which fire at a given version. Delegate both to the base, as shown. -Forward `RecordObservation` whenever the resource may be registered read-only with a data extractor. The framework feeds -the fetched cluster object back to the resource before extraction runs; without it, the extractor would see the inert -base passed to the builder rather than live cluster state. +Forward `ProducedData` and `ConsumedData` whenever the resource can take part in a component's data flow, which is +always if your builder exposes `ExtractInto`, `WithDataGuard`, or `WithOptionalData`. They satisfy +`concepts.DataProducer` and `concepts.DataConsumer`. Without them the component sees no declarations, so +[build-time topology validation](component.md#build-time-validation) silently passes, `DataTopology()` omits the +resource, and its cells are never cleared at the start of a reconcile. + +Forward `RecordObservation` whenever the resource may be registered read-only and declares an extraction. The framework +feeds the fetched cluster object back to the resource before extraction runs; without it, the extraction would see the +inert base passed to the builder rather than live cluster state. Which methods to include depends on the category: -| Category | Methods to include | -| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Workload | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `GraceStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | -| Static | `Identity`, `Object`, `Mutate`, `GuardStatus`, `ExtractData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | -| Task | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | -| Integration | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `GraceStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | +| Category | Methods to include | +| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Workload | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `GraceStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | +| Static | `Identity`, `Object`, `Mutate`, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | +| Task | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | +| Integration | `Identity`, `Object`, `Mutate`, `ConvergingStatus`, `GraceStatus`, `DeleteOnSuspend`, `Suspend`, `SuspensionStatus`, `GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, `FiringSet` | For task and integration resources, `ConvergingStatus` returns `concepts.CompletionStatusWithReason` and `concepts.OperationalStatusWithReason` respectively, matching the generic base method signature. @@ -763,10 +818,10 @@ implications. ### Static resources Static resources have the simplest implementation. They do not participate in convergence, grace, or suspension -reporting. The builder uses `generic.NewStaticBuilder`, which supports `WithMutation`, `WithGuard`, and -`WithDataExtractor`. The resource wrapper needs only `Identity`, `Object`, `Mutate`, `GuardStatus`, `ExtractData`, -`RecordObservation`, `Preview`, `RegisteredMutations`, and `FiringSet`. `pkg/primitives/configmap` is a complete -reference. +reporting. The builder uses `generic.NewStaticBuilder`, which supports `WithMutation`, `WithGuard`, `WithDataGuard`, and +`WithOptionalData`, plus a package-level `ExtractInto`. The resource wrapper needs only `Identity`, `Object`, `Mutate`, +`GuardStatus`, `ExtractData`, `ProducedData`, `ConsumedData`, `RecordObservation`, `Preview`, `RegisteredMutations`, and +`FiringSet`. `pkg/primitives/configmap` is a complete reference. ### Task resources @@ -851,13 +906,13 @@ logic. ## Reference -| Package | Contains | -| ------------------------ | -------------------------------------------------------------- | -| `pkg/generic` | Generic resource types, builders, `WrapGuard`, `WrapExtractor` | -| `pkg/feature` | `Mutation`, `Gate`, `VersionGate`, `NewVersionGate` | -| `pkg/component/concepts` | Lifecycle interfaces and status type constants | -| `pkg/component` | Component builder, resource registration, reconciliation | -| `pkg/primitives/*` | Built-in implementations to use as references | +| Package | Contains | +| ------------------------ | ------------------------------------------------------------------------------ | +| `pkg/generic` | Generic resource types, builders, `ExtractInto`, `WrapGuard`, `WrapExtraction` | +| `pkg/feature` | `Mutation`, `Gate`, `VersionGate`, `NewVersionGate` | +| `pkg/component/concepts` | Lifecycle interfaces, status type constants, `NewData`, `DataCell` | +| `pkg/component` | Component builder, resource registration, reconciliation | +| `pkg/primitives/*` | Built-in implementations to use as references | For a complete, runnable wrapper of a third-party CRD (using the unstructured static builder rather than a typed struct), see `examples/custom-resource`. diff --git a/docs/guidelines.md b/docs/guidelines.md index a5f13427..ea19755e 100644 --- a/docs/guidelines.md +++ b/docs/guidelines.md @@ -59,9 +59,9 @@ A mutation must be a pure function of the owner spec and other inputs available resource's live cluster state to decide what to write. This is not only a style preference. Within a single resource, the framework applies mutations **before** data -extraction runs, so a closure variable populated by a data extractor on the same builder still holds its zero value when -that resource's mutations execute. Data extraction passes observed state from an **earlier** resource to a **later** -resource, not back into a resource's own mutations. +extraction runs, so a cell written by an extraction declared on the same builder is still unset when that resource's +mutations execute. Declared data passes observed state from an **earlier** resource to a **later** resource, not back +into a resource's own mutations. A mutation that produces the same desired object for the same spec, regardless of what currently exists in the cluster, aligns with Server-Side Apply's declarative model and keeps reconciliation predictable. If you find yourself wanting to @@ -202,9 +202,9 @@ status even while controller-runtime backs off. ## Resource Registration Order Is Execution Order -Resources reconcile in the exact order they are registered with `WithResource`. This is deliberate: guards and data -extractors depend on it, and reading the calls top to bottom tells you the order with no implicit dependency graph to -reconstruct. +Resources reconcile in the exact order they are registered with `WithResource`. This is deliberate: guards and declared +data extraction depend on it, and reading the calls top to bottom tells you the order with no implicit dependency graph +to reconstruct. Register dependencies before dependents. A common per-component bundle reads as a dependency chain: read-only Secret references first (with [`BlockOnAbsence`](component.md#resource-registration-options) so an absent Secret blocks the @@ -222,8 +222,9 @@ comp, err := component.NewComponentBuilder(). Build() ``` -The flip side is that reordering these calls can silently break data flow between extractors and guards, so document the -dependency where one exists. +Reordering these calls breaks data flow between a resource that extracts a value and a later one that reads it. Where +the flow is declared with [data cells](#use-data-extraction-and-guards-for-intra-component-dependencies), `Build()` +turns that mistake into a build error instead of a resource that waits forever. ## Mutation Ordering and Container-Name Dependencies @@ -319,39 +320,79 @@ the number of supported versions, and each one deletes cleanly when its version ## Use Data Extraction and Guards for Intra-Component Dependencies -When one resource depends on data from another resource in the **same** component, register a data extractor on the -source and a guard on the dependent. Do not assume a resource is ready just because it was registered earlier. +When one resource depends on data from another resource in the **same** component, declare the flow rather than passing +a variable between two closures. A `concepts.Data[T]` cell is a named, typed, presence-aware container: the producer +declares an extraction into it, the consumer declares its read, and `Build()` verifies the two are registered in an +order that can actually work. ```go -var roleARN string +func buildDatabaseComponent(app *v1alpha1.WebApp) (*component.Component, error) { + dbHost := concepts.NewData[string]("db-host") + + cmBuilder := configmap.NewBuilder(dbConfig(app)) + configmap.ExtractInto(cmBuilder, dbHost, func(cm corev1.ConfigMap) (string, error) { + return cm.Data["db-host"], nil + }) + cmRes, err := cmBuilder.Build() + if err != nil { + return nil, err + } -roleRes, _ := static.NewBuilder(cloudRole(app)). - WithDataExtractor(func(obj uns.Unstructured) error { - roleARN, _, _ = unstructured.NestedString(obj.Object, "status", "arn") - return nil - }). - Build() + secretBuilder := secret.NewBuilder(dbCredentials(app)) + secretBuilder.WithDataGuard(dbHost) + secretBuilder.WithMutation(secret.Mutation{ + Name: "db-host-entry", + Mutate: func(m *secret.Mutator) error { + host, err := dbHost.Require() + if err != nil { + return err + } + m.SetStringData("db-host", host) + return nil + }, + }) + secretRes, err := secretBuilder.Build() + if err != nil { + return nil, err + } -bucketRes, _ := static.NewBuilder(cloudBucket(app)). - WithGuard(func(_ uns.Unstructured) (concepts.GuardStatusWithReason, error) { - if roleARN == "" { - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusBlocked, - Reason: "waiting for cloud role ARN", - }, nil - } - return concepts.GuardStatusWithReason{Status: concepts.GuardStatusUnblocked}, nil - }). - Build() + // The producer must be registered before the consumer; Build() enforces it. + return component.NewComponentBuilder(). + WithName("database"). + WithConditionType("DatabaseReady"). + WithResource(cmRes). + WithResource(secretRes). + Build() +} ``` -A blocked guard surfaces as a `Blocked` condition reason, so users can see why a resource has not been created yet. The -shared variable is scoped to one reconcile, which prevents state leaking between reconciles. +Create cells inside the component assembly function so they stay scoped to a single reconcile. The component clears +every declared cell before it reconciles anything, so a cell that somehow outlives its assembly function still cannot +carry a value into the next pass. Sharing one cell across components is unsupported: validation and reset are per +component. + +`WithDataGuard` generates both the guard and its reason, so the message a user reads (`waiting for data "db-host"`) +cannot drift from the real dependency. A blocked data guard surfaces as the same `Blocked` condition reason any guard +produces. Keep `WithGuard` for preconditions that are not "a value exists", such as a status phase reaching a specific +value. + +Three consumption modes cover the useful cases: + +| Mode | Declaration | Accessor | Behavior when absent | +| -------------------------- | ------------------ | --------- | ------------------------------------------------- | +| Block until present | `WithDataGuard` | `Require` | Resource waits, the condition explains why | +| Proceed, enrich when ready | `WithOptionalData` | `Get` | Mutation skips; the field appears on a later pass | +| Proceed, fail loudly | `WithOptionalData` | `Require` | Mutation errors, the component reports a failure | + +`WithOptionalData` never gates. Declare it anyway: the build-time check then still verifies that some earlier resource +produces the cell, because an optional read with no producer can never be satisfied and is almost always a mistake, and +the dependency stays visible to introspection. -Prefer **stable** values for guard conditions. A guard re-evaluates every reconcile, so a value that can transiently -disappear (a replica count, a field cleared during a rolling update) will re-block a resource that is already running. -Good targets appear once and persist: a status field written by a controller, a provisioned IP, a generated credential -reference. +Prefer **stable** values. A guard re-evaluates every reconcile, so a value that can transiently disappear (a replica +count, a field cleared during a rolling update) will re-block a resource that is already running. Good targets appear +once and persist: a status field written by a controller, a provisioned IP, a generated credential reference. This +applies doubly to optional enrichment, which has no guard to hold the resource back: a source value that comes and goes +makes the enriched field flap, rewriting the consuming resource on every swing. ## Use Prerequisites for Cross-Component Dependencies @@ -440,19 +481,23 @@ func extraEnv(app *v1alpha1.WebApp) deployment.Mutation { Because `EnsureEnvVars` replaces existing entries by name, registering this mutation after the operator's own env mutations lets a user value shadow an operator-emitted one without you enumerating every overridable field. -A related use of a final mutation is **secret-rotation restart**: each read-only Secret has a data extractor that hashes -its contents into a shared map, and a final mutation stamps that map onto the pod template as annotations through -`EditPodTemplateMetadata`. A Secret rotation changes a hash, which changes the pod template, which triggers a rolling -restart. Keep the map empty during preview so golden snapshots stay stable. +A related use of a final mutation is **secret-rotation restart**: each read-only Secret declares an extraction that +hashes its contents into a cell, the workload declares those cells with `WithOptionalData`, and a final mutation stamps +the hashes it can read onto the pod template as annotations through `EditPodTemplateMetadata`. A Secret rotation changes +a hash, which changes the pod template, which triggers a rolling restart. Optional reads keep the workload moving when a +Secret has not been fetched yet, and cells are unset in a cluster-free preview, so golden snapshots stay stable without +any special casing. ```go -func checksumAnnotations(hashes map[string]string) deployment.Mutation { +func checksumAnnotations(hashes map[string]*concepts.Data[string]) deployment.Mutation { return deployment.Mutation{ Name: "ChecksumAnnotations", Mutate: func(m *deployment.Mutator) error { m.EditPodTemplateMetadata(func(e *editors.ObjectMetaEditor) error { - for k, v := range hashes { - e.EnsureAnnotation("checksum/"+k, v) + for name, cell := range hashes { + if hash, ok := cell.Get(); ok { + e.EnsureAnnotation("checksum/"+name, hash) + } } return nil }) diff --git a/docs/primitives.md b/docs/primitives.md index 6443a554..d8dd29d5 100644 --- a/docs/primitives.md +++ b/docs/primitives.md @@ -88,7 +88,7 @@ A primitive participates in status aggregation by implementing one or more lifec | `Completable` | `Completed`, `TaskRunning`, `TaskPending`, `TaskFailing` | Jobs and task primitives | | `Operational` | `Operational`, `OperationPending`, `OperationFailing` | Services, Ingresses, CronJobs | | `Guardable` | `Blocked` | Resources with runtime preconditions | -| `DataExtractable` | _(no status, side-effecting)_ | Resources that expose post-sync data | +| `DataExtractable` | _(no status, side-effecting)_ | Resources that publish post-sync values to cells | !!! warning "`Guardable` reports only `Blocked`" @@ -360,8 +360,8 @@ those. ## Usage Examples The example below builds a frontend `Deployment` for a hypothetical `WebApp` operator, adds a version-gated sidecar -mutation, targets multiple containers, guards on a value extracted from an earlier resource, and registers the result -with a component. +mutation, targets multiple containers, guards on a value declared and extracted by an earlier resource, and registers +the result with a component. === "Building and registering a primitive" @@ -398,6 +398,10 @@ with a component. }, } + // apiEndpoint is created by the component assembly function and written by + // an earlier resource's declared extraction. + apiEndpoint := concepts.NewData[string]("api-endpoint") + res, err := deployment.NewBuilder(base). // 2. A mutation: add a sidecar, gated on a version constraint, and // configure it. The sidecar is added then edited in one pass. @@ -427,17 +431,10 @@ with a component. return nil }, }). - // 4. A guard: do not apply until a precondition (here, a value - // extracted from an earlier resource) is satisfied. - WithGuard(func(_ appsv1.Deployment) (concepts.GuardStatusWithReason, error) { - if apiEndpoint == "" { - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusBlocked, - Reason: "waiting for backend endpoint", - }, nil - } - return concepts.GuardStatusWithReason{Status: concepts.GuardStatusUnblocked}, nil - }). + // 4. A data guard: do not apply until the earlier resource has extracted + // the endpoint. The framework generates the blocked reason from the + // cell name, here `waiting for data "api-endpoint"`. + WithDataGuard(apiEndpoint). Build() if err != nil { return nil, err @@ -462,9 +459,9 @@ with a component. !!! note "Guards versus prerequisites" - A [guard](component.md#guards) handles a dependency **within** one component: an earlier resource extracts data after - it is applied, and a later resource's guard checks that data before proceeding. For a dependency **between** - components (the frontend cannot start until the backend is ready), use + A [guard](component.md#guards) handles a dependency **within** one component: an earlier resource extracts a value + into a [data cell](component.md#declared-data) after it is applied, and a later resource blocks on that cell before + proceeding. For a dependency **between** components (the frontend cannot start until the backend is ready), use [prerequisites](component.md#prerequisites) on the component builder instead. See [Guards](component.md#guards) for the full behavioral contract. diff --git a/docs/primitives/clusterrole.md b/docs/primitives/clusterrole.md index 19657583..cd7a24eb 100644 --- a/docs/primitives/clusterrole.md +++ b/docs/primitives/clusterrole.md @@ -192,17 +192,23 @@ Pass `nil` to clear the aggregation rule. Within a single feature, the last `Set ## Data Extraction -`WithDataExtractor` runs a callback after successful reconciliation with a value copy of the reconciled ClusterRole: +`clusterrole.ExtractInto` declares that this ClusterRole produces the value of a data cell. The function receives a +value copy of the reconciled ClusterRole and runs immediately after each sync cycle: ```go -resource, err := clusterrole.NewBuilder(base). - WithDataExtractor(func(cr rbacv1.ClusterRole) error { - sharedState.ClusterRoleName = cr.Name - return nil - }). - Build() +roleName := concepts.NewData[string]("viewer-cluster-role") + +builder := clusterrole.NewBuilder(base) +clusterrole.ExtractInto(builder, roleName, func(cr rbacv1.ClusterRole) (string, error) { + return cr.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(roleName)` or read it +opportunistically with `WithOptionalData(roleName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/clusterrolebinding.md b/docs/primitives/clusterrolebinding.md index bc549d36..900d440b 100644 --- a/docs/primitives/clusterrolebinding.md +++ b/docs/primitives/clusterrolebinding.md @@ -174,18 +174,24 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -`WithDataExtractor` runs a callback after successful reconciliation with a value copy of the reconciled -ClusterRoleBinding. Use it to surface binding metadata to other resources: +`clusterrolebinding.ExtractInto` declares that this ClusterRoleBinding produces the value of a data cell, which is how +you surface binding metadata to other resources. The function receives a value copy of the reconciled ClusterRoleBinding +and runs immediately after each sync cycle: ```go -resource, err := clusterrolebinding.NewBuilder(base). - WithDataExtractor(func(crb rbacv1.ClusterRoleBinding) error { - sharedState.ClusterRoleBindingName = crb.Name - return nil - }). - Build() +bindingName := concepts.NewData[string]("viewer-cluster-role-binding") + +builder := clusterrolebinding.NewBuilder(base) +clusterrolebinding.ExtractInto(builder, bindingName, func(crb rbacv1.ClusterRoleBinding) (string, error) { + return crb.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(bindingName)` or read it +opportunistically with `WithOptionalData(bindingName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/ingress.md b/docs/primitives/ingress.md index 6ccd685a..b878d5d3 100644 --- a/docs/primitives/ingress.md +++ b/docs/primitives/ingress.md @@ -11,7 +11,7 @@ metadata. | **Operational** | Reports `OperationPending` until the ingress controller assigns an address, then `Operational` | | **Graceful** | Reports `Degraded` until a load balancer IP or hostname is assigned, then `Healthy` | | **Suspendable** | No-op by default. Ingress is left in place; backend returns 502/503 when the backing service is down | -| **DataExtractable** | Reads assigned load balancer addresses after each sync cycle via `WithDataExtractor` | +| **DataExtractable** | Reads assigned load balancer addresses after each sync cycle via `ExtractInto` | | **Mutation pipeline** | Typed editors for metadata and Ingress spec (rules, TLS, class name, default backend) | See [Lifecycle Interfaces](../primitives.md#lifecycle-interfaces) for the full set of status values each interface diff --git a/docs/primitives/networkpolicy.md b/docs/primitives/networkpolicy.md index b49dfa9e..62128187 100644 --- a/docs/primitives/networkpolicy.md +++ b/docs/primitives/networkpolicy.md @@ -10,7 +10,7 @@ resource, providing a structured mutation API for managing pod selectors, ingres | **Static lifecycle** | No health tracking, grace periods, or suspension. The resource is reconciled to desired state | | **Mutation pipeline** | Typed editors for NetworkPolicy spec and object metadata, with a `Raw()` escape hatch | | **Append semantics** | Ingress and egress rules have no unique key; `AppendIngressRule`/`AppendEgressRule` append unconditionally | -| **DataExtractable** | Reads values back from the reconciled NetworkPolicy after each sync cycle via `WithDataExtractor` | +| **DataExtractable** | Reads values back from the reconciled NetworkPolicy after each sync cycle via `ExtractInto` | See [Lifecycle Interfaces](../primitives.md#lifecycle-interfaces) for the full set of status values each interface reports. @@ -169,20 +169,24 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -Use `WithDataExtractor` to read values from the reconciled NetworkPolicy after each sync cycle. This is useful when -downstream resources need to observe the final applied policy (for example, its resource version or assigned labels): +`networkpolicy.ExtractInto` declares that this NetworkPolicy produces the value of a data cell. This is useful when +downstream resources need to observe the final applied policy (for example, its resource version or assigned labels). +The function receives a value copy of the reconciled NetworkPolicy after each sync cycle: ```go -var policyName string +policyName := concepts.NewData[string]("frontend-network-policy") -resource, err := networkpolicy.NewBuilder(base). - WithDataExtractor(func(np networkingv1.NetworkPolicy) error { - policyName = np.Name - return nil - }). - Build() +builder := networkpolicy.NewBuilder(base) +networkpolicy.ExtractInto(builder, policyName, func(np networkingv1.NetworkPolicy) (string, error) { + return np.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(policyName)` or read it +opportunistically with `WithOptionalData(policyName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/pdb.md b/docs/primitives/pdb.md index ceab4477..6653eaf8 100644 --- a/docs/primitives/pdb.md +++ b/docs/primitives/pdb.md @@ -154,18 +154,24 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -Use `WithDataExtractor` to read generated or server-populated fields after each sync cycle. The extractor receives a -value copy of the reconciled PDB: +`pdb.ExtractInto` declares that this PDB produces the value of a data cell, which is how you read generated or +server-populated fields. The function receives a value copy of the reconciled PDB after each sync cycle: ```go -pdb.NewBuilder(base). - WithDataExtractor(func(p policyv1.PodDisruptionBudget) error { - // p.Status.ExpectedPods is populated by the Kubernetes PDB controller - myComponent.ExpectedPods = p.Status.ExpectedPods - return nil - }) +expectedPods := concepts.NewData[int32]("expected-pods") + +builder := pdb.NewBuilder(base) +pdb.ExtractInto(builder, expectedPods, func(p policyv1.PodDisruptionBudget) (int32, error) { + // Status.ExpectedPods is populated by the Kubernetes PDB controller. + return p.Status.ExpectedPods, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(expectedPods)` or read it +opportunistically with `WithOptionalData(expectedPods)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go @@ -237,5 +243,5 @@ protects. If a mutation renames pods or changes their labels, update the PDB sel **Register mutations in dependency order.** If mutation B relies on state set by mutation A, register A first. **Use data extraction to read `Status` fields.** Fields like `Status.ExpectedPods`, `Status.CurrentHealthy`, and -`Status.DisruptionsAllowed` are populated by the Kubernetes PDB controller after reconciliation. Access them through -`WithDataExtractor` rather than inspecting the baseline object. +`Status.DisruptionsAllowed` are populated by the Kubernetes PDB controller after reconciliation. Declare an extraction +into a data cell rather than inspecting the baseline object. diff --git a/docs/primitives/pvc.md b/docs/primitives/pvc.md index 777ce034..c8298096 100644 --- a/docs/primitives/pvc.md +++ b/docs/primitives/pvc.md @@ -225,20 +225,22 @@ func ExpandedStorageMutation(version string) pvc.Mutation { } } -var boundVolumeName string +boundVolume := concepts.NewData[string]("bound-volume") -resource, err := pvc.NewBuilder(base). +builder := pvc.NewBuilder(base). WithMutation(StorageRequestMutation(owner.Spec.Version)). - WithMutation(ExpandedStorageMutation(owner.Spec.Version)). - WithDataExtractor(func(p corev1.PersistentVolumeClaim) error { - boundVolumeName = p.Spec.VolumeName - return nil - }). - Build() + WithMutation(ExpandedStorageMutation(owner.Spec.Version)) + +pvc.ExtractInto(builder, boundVolume, func(p corev1.PersistentVolumeClaim) (string, error) { + return p.Spec.VolumeName, nil +}) + +resource, err := builder.Build() ``` On versions 2.0.0 and above, `ExpandedStorageMutation` fires and sets the storage request to 50Gi. On earlier versions, -only the base 10Gi request is applied. After each reconcile cycle, the data extractor captures the bound volume name. +only the base 10Gi request is applied. After each reconcile cycle, the declared extraction captures the bound volume +name into the `bound-volume` cell. ## Guidance @@ -249,8 +251,8 @@ invalid requests. **Prefer `WithCustomSuspendDeletionDecision` over deleting PVCs manually.** If you need PVCs to be cleaned up during suspension, register a deletion decision handler rather than deleting them in a mutation. -**Use `WithDataExtractor` to read bound volume information.** The bound volume name and actual allocated capacity are -server-assigned. Read them with a data extractor after reconciliation rather than caching them in mutation logic. +**Use `ExtractInto` to read bound volume information.** The bound volume name and actual allocated capacity are +server-assigned. Declare an extraction into a data cell rather than caching them in mutation logic. **Use string status values in conditions.** The operational status values that appear in conditions are the runtime strings `"Operational"`, `"OperationPending"`, and `"OperationFailing"`, not the Go constant identifiers. diff --git a/docs/primitives/role.md b/docs/primitives/role.md index 29664c65..575d1e67 100644 --- a/docs/primitives/role.md +++ b/docs/primitives/role.md @@ -154,18 +154,24 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -`WithDataExtractor` runs a callback after successful reconciliation with a value copy of the reconciled Role. Use it to -surface the applied rules or metadata to other resources: +`role.ExtractInto` declares that this Role produces the value of a data cell, which is how you surface the applied rules +or metadata to other resources. The function receives a value copy of the reconciled Role and runs immediately after +each sync cycle: ```go -resource, err := role.NewBuilder(base). - WithDataExtractor(func(r rbacv1.Role) error { - sharedState.RoleName = r.Name - return nil - }). - Build() +roleName := concepts.NewData[string]("app-role") + +builder := role.NewBuilder(base) +role.ExtractInto(builder, roleName, func(r rbacv1.Role) (string, error) { + return r.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(roleName)` or read it +opportunistically with `WithOptionalData(roleName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/rolebinding.md b/docs/primitives/rolebinding.md index 0017bfa3..cc99af66 100644 --- a/docs/primitives/rolebinding.md +++ b/docs/primitives/rolebinding.md @@ -157,18 +157,24 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -`WithDataExtractor` runs a callback after successful reconciliation with a value copy of the reconciled RoleBinding. Use -it to surface binding metadata to other resources: +`rolebinding.ExtractInto` declares that this RoleBinding produces the value of a data cell, which is how you surface +binding metadata to other resources. The function receives a value copy of the reconciled RoleBinding and runs +immediately after each sync cycle: ```go -resource, err := rolebinding.NewBuilder(base). - WithDataExtractor(func(rb rbacv1.RoleBinding) error { - sharedState.RoleBindingName = rb.Name - return nil - }). - Build() +bindingName := concepts.NewData[string]("app-role-binding") + +builder := rolebinding.NewBuilder(base) +rolebinding.ExtractInto(builder, bindingName, func(rb rbacv1.RoleBinding) (string, error) { + return rb.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(bindingName)` or read it +opportunistically with `WithOptionalData(bindingName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/service.md b/docs/primitives/service.md index 609efd26..78b1a795 100644 --- a/docs/primitives/service.md +++ b/docs/primitives/service.md @@ -154,20 +154,23 @@ m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { ## Data Extraction -Use `WithDataExtractor` to read values from the reconciled Service after each sync cycle, such as the assigned ClusterIP -or LoadBalancer ingress: +`service.ExtractInto` declares that this Service produces the value of a data cell, such as the assigned ClusterIP or +LoadBalancer ingress. The function receives a value copy of the reconciled Service after each sync cycle: ```go -var assignedIP string +clusterIP := concepts.NewData[string]("backend-cluster-ip") -resource, err := service.NewBuilder(base). - WithDataExtractor(func(svc corev1.Service) error { - assignedIP = svc.Spec.ClusterIP - return nil - }). - Build() +builder := service.NewBuilder(base) +service.ExtractInto(builder, clusterIP, func(svc corev1.Service) (string, error) { + return svc.Spec.ClusterIP, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(clusterIP)` or read it +opportunistically with `WithOptionalData(clusterIP)`. See [Declared Data](../component.md#declared-data). + ## Operational Status The Service primitive implements `concepts.Operational`. The default handler reports: @@ -278,16 +281,17 @@ func MetricsPortMutation(version string, enabled bool) service.Mutation { } } -var assignedIP string +clusterIP := concepts.NewData[string]("backend-cluster-ip") -resource, err := service.NewBuilder(base). +builder := service.NewBuilder(base). WithMutation(BaseServiceMutation(owner.Spec.Version)). - WithMutation(MetricsPortMutation(owner.Spec.Version, owner.Spec.EnableMetrics)). - WithDataExtractor(func(svc corev1.Service) error { - assignedIP = svc.Spec.ClusterIP - return nil - }). - Build() + WithMutation(MetricsPortMutation(owner.Spec.Version, owner.Spec.EnableMetrics)) + +service.ExtractInto(builder, clusterIP, func(svc corev1.Service) (string, error) { + return svc.Spec.ClusterIP, nil +}) + +resource, err := builder.Build() ``` When `EnableMetrics` is true, the Service exposes both the HTTP and metrics ports. When false, only HTTP is configured. @@ -305,5 +309,5 @@ repeated calls with the same name produce the same result. **Leave Services in place during suspension.** The no-op default is correct for most Services. Only override `WithCustomSuspendDeletionDecision` when your use case requires explicitly removing the Service during suspension. -**Use `WithDataExtractor` for assigned addresses.** ClusterIP and LoadBalancer ingress are server-assigned. Read them -with a data extractor after reconciliation rather than caching them in mutation logic. +**Use `ExtractInto` for assigned addresses.** ClusterIP and LoadBalancer ingress are server-assigned. Declare an +extraction into a data cell rather than caching them in mutation logic. diff --git a/docs/primitives/serviceaccount.md b/docs/primitives/serviceaccount.md index 7d936ee3..992d5f27 100644 --- a/docs/primitives/serviceaccount.md +++ b/docs/primitives/serviceaccount.md @@ -121,18 +121,24 @@ The pointed-to value is snapshotted at registration time, so later caller-side c ## Data Extraction -`WithDataExtractor` runs a callback after successful reconciliation with a value copy of the reconciled ServiceAccount. -Use it to surface generated fields to other resources: +`serviceaccount.ExtractInto` declares that this ServiceAccount produces the value of a data cell, which is how you +surface generated fields to other resources. The function receives a value copy of the reconciled ServiceAccount and +runs immediately after each sync cycle: ```go -resource, err := serviceaccount.NewBuilder(base). - WithDataExtractor(func(sa corev1.ServiceAccount) error { - sharedState.ServiceAccountName = sa.Name - return nil - }). - Build() +saName := concepts.NewData[string]("app-service-account") + +builder := serviceaccount.NewBuilder(base) +serviceaccount.ExtractInto(builder, saName, func(sa corev1.ServiceAccount) (string, error) { + return sa.Name, nil +}) + +resource, err := builder.Build() ``` +Resources registered later in the same component block on the cell with `WithDataGuard(saName)` or read it +opportunistically with `WithOptionalData(saName)`. See [Declared Data](../component.md#declared-data). + ## Full Example ```go diff --git a/docs/primitives/unstructured.md b/docs/primitives/unstructured.md index 93368655..a5ea96f8 100644 --- a/docs/primitives/unstructured.md +++ b/docs/primitives/unstructured.md @@ -237,19 +237,24 @@ was called. ## Data Extraction -All four variants support data extraction. The extractor receives a value copy of the reconciled object after each sync -cycle: +All four variants support declared extraction through a package-level `ExtractInto`, which records that the resource +produces the value of a data cell. The function receives a value copy of the reconciled object after each sync cycle: ```go -builder.WithDataExtractor(func(obj uns.Unstructured) error { - ip, found, _ := uns.NestedString(obj.Object, "status", "atProvider", "ipAddress") - if found { - myComponent.ResourceIP = ip - } - return nil +providerIP := concepts.NewData[string]("provider-ip") + +builder := static.NewBuilder(obj) +static.ExtractInto(builder, providerIP, func(obj uns.Unstructured) (string, error) { + ip, _, err := uns.NestedString(obj.Object, "status", "atProvider", "ipAddress") + return ip, err }) ``` +An absent field yields the zero value and the cell is still marked present, which a data guard treats as satisfied. +Return an error from the function instead when a missing field should fail the reconcile and leave the cell unset. +Resources registered later in the same component block on the cell with `WithDataGuard(providerIP)` or read it +opportunistically with `WithOptionalData(providerIP)`. See [Declared Data](../component.md#declared-data). + ## Suspension Handlers The non-static variants support custom suspension behavior. All three handlers default to safe no-ops when omitted. @@ -287,7 +292,9 @@ obj.SetGroupVersionKind(schema.GroupVersionKind{ obj.SetName("app-db") obj.SetNamespace(owner.Namespace) -resource, err := integration.NewBuilder(obj). +dbEndpoint := concepts.NewData[string]("db-endpoint") + +builder := integration.NewBuilder(obj). WithMutation(unstruct.Mutation{ Name: "connection-config", Feature: feature.NewVersionGate(owner.Spec.Version, nil), @@ -311,13 +318,14 @@ resource, err := integration.NewBuilder(obj). default: return concepts.OperationalStatusWithReason{Status: concepts.OperationalStatusFailing, Reason: phase}, nil } - }). - WithDataExtractor(func(o uns.Unstructured) error { - endpoint, _, _ := uns.NestedString(o.Object, "status", "endpoint") - myComponent.DBEndpoint = endpoint - return nil - }). - Build() + }) + +integration.ExtractInto(builder, dbEndpoint, func(o uns.Unstructured) (string, error) { + endpoint, _, err := uns.NestedString(o.Object, "status", "endpoint") + return endpoint, err +}) + +resource, err := builder.Build() ``` ## Guidance diff --git a/e2e/component/multi_resource_test.go b/e2e/component/multi_resource_test.go index a3ad5a7d..e5be3e09 100644 --- a/e2e/component/multi_resource_test.go +++ b/e2e/component/multi_resource_test.go @@ -282,54 +282,42 @@ var _ = Describe("Multi-Resource Component", func() { }) It("should extract data from resource A, unblock resource B guard, and inject into B via mutation", func() { - // This validates the full extractor -> guard -> mutation flow end-to-end: - // Resource A's extractor populates a shared variable. Resource B's guard - // checks that variable to unblock. Resource B's mutation reads the variable - // and injects it into the ConfigMap's data at Mutate() time. - var extractedARN atomic.Value - + // This validates the full declared extraction -> guard -> mutation flow + // end-to-end: resource A's extraction writes a declared data cell, + // resource B's data guard blocks until the cell is set, and resource + // B's mutation reads the cell and injects it into the ConfigMap's data + // at Mutate() time. clusterReconciler.RegisterComponent(name, func(owner *framework.ClusterTestApp) (*component.Component, error) { - // First resource: extracts the ARN after apply - cmRes, err := configmap.NewBuilder(newConfigMap(ns, "provider-role", map[string]string{ + arn := concepts.NewData[string]("provider-role-arn") + + // First resource: declares the extraction that writes the arn cell + cmBuilder := configmap.NewBuilder(newConfigMap(ns, "provider-role", map[string]string{ "arn": "arn:aws:iam::123456789:role/test", - })). - WithDataExtractor(func(cm corev1.ConfigMap) error { - if v, ok := cm.Data["arn"]; ok { - extractedARN.Store(v) - } - return nil - }). - Build() + })) + configmap.ExtractInto(cmBuilder, arn, func(cm corev1.ConfigMap) (string, error) { + return cm.Data["arn"], nil + }) + cmRes, err := cmBuilder.Build() if err != nil { return nil, err } - // Second resource: guard checks the extracted value, mutation injects it + // Second resource: data guard blocks on the cell, mutation injects it bucketRes, err := configmap.NewBuilder(newConfigMap(ns, "provider-bucket", map[string]string{ "name": "my-bucket", })). - WithGuard(func(_ corev1.ConfigMap) (concepts.GuardStatusWithReason, error) { - v := extractedARN.Load() - if v == nil || v.(string) == "" { - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusBlocked, - Reason: "waiting for provider role ARN", - }, nil - } - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusUnblocked, - }, nil - }). + WithDataGuard(arn). WithMutation(configmap.Mutation{ Name: "inject-role-arn", Mutate: func(m *configmap.Mutator) error { - v := extractedARN.Load() - if v != nil { - m.EditData(func(e *editors.ConfigMapDataEditor) error { - e.Set("role-arn", v.(string)) - return nil - }) + v, err := arn.Require() + if err != nil { + return err } + m.EditData(func(e *editors.ConfigMapDataEditor) error { + e.Set("role-arn", v) + return nil + }) return nil }, }). diff --git a/examples/custom-resource/README.md b/examples/custom-resource/README.md index e3c64289..cf3feeb7 100644 --- a/examples/custom-resource/README.md +++ b/examples/custom-resource/README.md @@ -10,7 +10,7 @@ the **unstructured static builder**. - **Content mutations**: `EditContent` with `UnstructuredContentEditor` sets nested spec fields (`issuerRef`, `dnsNames`) using structured helpers rather than raw map manipulation. - **Metadata mutations**: `EditObjectMetadata` works the same way as on typed primitives. -- **Data extraction**: `WithDataExtractor` reads fields from the reconciled unstructured object. +- **Declared extraction**: `static.ExtractInto` reads fields from the reconciled unstructured object into a data cell. ## Use case diff --git a/examples/custom-resource/resources/certificate.go b/examples/custom-resource/resources/certificate.go index 5a20cb9e..dd3e935f 100644 --- a/examples/custom-resource/resources/certificate.go +++ b/examples/custom-resource/resources/certificate.go @@ -6,6 +6,7 @@ import ( "github.com/sourcehawk/operator-component-framework/examples/custom-resource/app" "github.com/sourcehawk/operator-component-framework/pkg/component" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/sourcehawk/operator-component-framework/pkg/mutation/editors" unstruct "github.com/sourcehawk/operator-component-framework/pkg/primitives/unstructured" "github.com/sourcehawk/operator-component-framework/pkg/primitives/unstructured/static" @@ -61,10 +62,15 @@ func NewCertificateResource(owner *app.ExampleApp) (component.Resource, error) { }, }) - builder.WithDataExtractor(func(obj uns.Unstructured) error { - dnsNames, _, _ := uns.NestedStringSlice(obj.Object, "spec", "dnsNames") - fmt.Printf(" Certificate DNS names: %v\n", dnsNames) - return nil + // A real consumer would receive this cell from the assembly function, the + // way the extraction-and-guards example wires a shared cell across + // resource factories. Here nothing downstream reads it, so it is declared + // locally and only its extracted value is printed. + dnsNames := concepts.NewData[[]string]("certificate-dns-names") + static.ExtractInto(builder, dnsNames, func(obj uns.Unstructured) ([]string, error) { + names, _, _ := uns.NestedStringSlice(obj.Object, "spec", "dnsNames") + fmt.Printf(" Certificate DNS names: %v\n", names) + return names, nil }) return builder.Build() diff --git a/examples/extraction-and-guards/README.md b/examples/extraction-and-guards/README.md index a5763734..882534ab 100644 --- a/examples/extraction-and-guards/README.md +++ b/examples/extraction-and-guards/README.md @@ -1,23 +1,37 @@ # Data Extraction and Guards -This example demonstrates how to use **data extraction** from one resource to feed a **guard** on a subsequent resource -within the same component. +This example demonstrates the declared data API: a **cell** carries a value from one resource's **extraction** to a +later resource's **data guard** and **mutation**, within the same component. ## What it shows -- **Data extraction**: The ConfigMap resource registers a `WithDataExtractor` that captures the `db-host` value into a - shared pointer after reconciliation. -- **Guard**: The Secret resource registers a `WithGuard` that checks whether the extracted `db-host` is non-empty. If it - is empty, the guard returns `Blocked` and the Secret (and any resources registered after it) are skipped. -- **Registration order matters**: The ConfigMap is registered before the Secret. Guards can only read data extracted by - preceding resources. +- **Cells**: `concepts.NewData[string]("db-host")` creates a named, typed cell. Cells are created inside the component + assembly function, once per reconcile, and passed to the resource factories that need them. +- **Declared extraction**: The ConfigMap resource calls `configmap.ExtractInto(builder, dbHost, fn)`. After the + ConfigMap is reconciled, `fn` runs against the reconciled object and its return value is written into the cell. +- **Data guard**: The Secret resource calls `builder.WithDataGuard(dbHost)`. The component blocks the Secret (and + anything registered after it) until `dbHost` has been set, with a generated reason explaining which cell it is waiting + on. +- **Require in a mutation**: The Secret also registers a mutation that calls `dbHost.Require()` to read the value and + copy it into the Secret's data, so the credentials and the endpoint they connect to travel together. +- **Registration order matters**: The ConfigMap is registered before the Secret. `Build()` validates that every guarded + or read cell has a producer registered strictly earlier, and rejects the component otherwise. +- **Topology introspection**: `Component.DataTopology()` returns the declared data flow, one edge per cell, without + running any extraction. `main.go` prints it before reconciling. ## Reconciliation steps -1. Normal reconciliation: the ConfigMap is created, `db-host` is extracted, the guard unblocks, and the Secret is - created. +1. Normal reconciliation: the ConfigMap is created, its extraction writes `dbHost`, the Secret's guard unblocks, and the + Secret is created with the `db-host` entry copied in. 2. Steady-state: both resources reconcile normally. +## Testing cluster-free previews + +A mutation that calls `Require()` needs the cell to already be set, but a golden-file preview never runs reconciliation, +so no extraction ever executes. The resource- and component-level tests seed the cell directly +(`dbHost.Set("postgres.default.svc")`) before asserting the golden file, simulating the value a real reconcile would +have extracted. `BuildComponent` returns the cell for exactly this reason. + ## Running ```bash diff --git a/examples/extraction-and-guards/app/component_test.go b/examples/extraction-and-guards/app/component_test.go index 78126da2..9bc624d5 100644 --- a/examples/extraction-and-guards/app/component_test.go +++ b/examples/extraction-and-guards/app/component_test.go @@ -7,6 +7,7 @@ import ( "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/app" "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/resources" sharedapp "github.com/sourcehawk/operator-component-framework/examples/shared/app" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/sourcehawk/operator-component-framework/pkg/testing/golden" "github.com/stretchr/testify/require" "k8s.io/apimachinery/pkg/runtime" @@ -38,22 +39,34 @@ func testOwner() *sharedapp.ExampleApp { return owner } -// TestBuildComponent goldens the whole component the controller reconciles. The -// point of this example is data extraction feeding a guard: the ConfigMap is -// registered before the Secret, and BuildComponent owns the shared dbHost pointer -// that wires the extractor to the guard. The multi-document golden pins the -// rendered desired state of both resources, in the order the component applies -// them. The controller and this test build the component the same way, so the -// reconciled component and the snapshot stay in lockstep. +// TestBuildComponent goldens the whole component the controller reconciles. +// The point of this example is declared data extraction feeding a data guard: +// the ConfigMap is registered before the Secret, and BuildComponent owns the +// shared db-host cell that wires the extraction to the guard and the +// mutation. The topology assertion pins the declared data flow itself, ahead +// of seeding the cell to simulate a preceding ConfigMap reconciliation, so +// the golden preview renders the Secret's db-host entry. The multi-document +// golden pins the rendered desired state of both resources, in the order the +// component applies them. The controller and this test build the component +// the same way, so the reconciled component and the snapshot stay in +// lockstep. func TestBuildComponent(t *testing.T) { controller := &app.Controller{ NewConfigMapResource: resources.NewConfigMapResource, NewSecretResource: resources.NewSecretResource, } - comp, err := controller.BuildComponent(testOwner()) + comp, dbHost, err := controller.BuildComponent(testOwner()) require.NoError(t, err) + require.Equal(t, []concepts.DataEdge{{ + Data: "db-host", + Producers: []string{"v1/ConfigMap/default/my-app-db-config"}, + Guarded: []string{"v1/Secret/default/my-app-db-credentials"}, + }}, comp.DataTopology()) + + dbHost.Set("postgres.default.svc") + golden.AssertComponentYAML(t, "testdata/component.yaml", comp, golden.WithScheme(scheme), golden.Update(*update)) } diff --git a/examples/extraction-and-guards/app/controller.go b/examples/extraction-and-guards/app/controller.go index 13f16c9a..1c42f6ec 100644 --- a/examples/extraction-and-guards/app/controller.go +++ b/examples/extraction-and-guards/app/controller.go @@ -5,26 +5,28 @@ import ( "context" "github.com/sourcehawk/operator-component-framework/pkg/component" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "k8s.io/apimachinery/pkg/runtime" "k8s.io/client-go/tools/record" "sigs.k8s.io/controller-runtime/pkg/client" ) // Controller reconciles an ExampleApp by managing a ConfigMap and a Secret -// within a single component. The ConfigMap exposes data via extraction, and -// the Secret is guarded until that data is available. +// within a single component. The ConfigMap exposes data via a declared +// extraction, and the Secret is guarded until that data is available. type Controller struct { client.Client Scheme *runtime.Scheme Recorder record.EventRecorder Metrics component.Recorder - // NewConfigMapResource builds the ConfigMap and wires the data extractor. - // The extractor writes to dbHost so the Secret guard can read it. - NewConfigMapResource func(owner *ExampleApp, dbHost *string) (component.Resource, error) + // NewConfigMapResource builds the ConfigMap and declares the extraction + // that writes the dbHost cell. + NewConfigMapResource func(owner *ExampleApp, dbHost *concepts.Data[string]) (component.Resource, error) - // NewSecretResource builds the Secret with a guard that reads dbHost. - NewSecretResource func(owner *ExampleApp, dbHost *string) (component.Resource, error) + // NewSecretResource builds the Secret with a data guard and a mutation + // that read the dbHost cell. + NewSecretResource func(owner *ExampleApp, dbHost *concepts.Data[string]) (component.Resource, error) } // Reconcile builds and reconciles a component where the ConfigMap is registered @@ -44,7 +46,7 @@ func (r *Controller) Reconcile(ctx context.Context, owner *ExampleApp) (err erro } }() - comp, err := r.BuildComponent(owner) + comp, _, err := r.BuildComponent(owner) if err != nil { return err } @@ -52,29 +54,32 @@ func (r *Controller) Reconcile(ctx context.Context, owner *ExampleApp) (err erro return comp.Reconcile(ctx, recCtx) } -// BuildComponent assembles the database component: a ConfigMap registered before -// a Secret, both wired to a shared dbHost pointer. The ConfigMap extractor writes -// the pointer and the Secret guard reads it, so registration order matters. The -// controller and tests share this assembly so the reconciled component and the -// golden snapshot stay in lockstep. -func (r *Controller) BuildComponent(owner *ExampleApp) (*component.Component, error) { - // Shared state: the ConfigMap extractor writes here, the Secret guard reads it. - var dbHost string +// BuildComponent assembles the database component: a ConfigMap registered +// before a Secret, both wired to a shared data cell. The ConfigMap's declared +// extraction writes the cell; the Secret's data guard and mutation read it. +// Build() verifies the ordering. The cell is returned so tests can seed it +// when rendering cluster-free previews and assert the declared topology. +func (r *Controller) BuildComponent(owner *ExampleApp) (*component.Component, *concepts.Data[string], error) { + dbHost := concepts.NewData[string]("db-host") - cmResource, err := r.NewConfigMapResource(owner, &dbHost) + cmResource, err := r.NewConfigMapResource(owner, dbHost) if err != nil { - return nil, err + return nil, nil, err } - secretResource, err := r.NewSecretResource(owner, &dbHost) + secretResource, err := r.NewSecretResource(owner, dbHost) if err != nil { - return nil, err + return nil, nil, err } - return component.NewComponentBuilder(). + comp, err := component.NewComponentBuilder(). WithName("database"). WithConditionType("DatabaseReady"). WithResource(cmResource). WithResource(secretResource). Build() + if err != nil { + return nil, nil, err + } + return comp, dbHost, nil } diff --git a/examples/extraction-and-guards/app/testdata/component.yaml b/examples/extraction-and-guards/app/testdata/component.yaml index 4a694ea9..c9749748 100644 --- a/examples/extraction-and-guards/app/testdata/component.yaml +++ b/examples/extraction-and-guards/app/testdata/component.yaml @@ -11,6 +11,7 @@ metadata: --- apiVersion: v1 data: + db-host: cG9zdGdyZXMuZGVmYXVsdC5zdmM= password: Y2hhbmdlbWU= username: YXBwLXVzZXI= kind: Secret diff --git a/examples/extraction-and-guards/main.go b/examples/extraction-and-guards/main.go index 5caa5a03..c96ebe32 100644 --- a/examples/extraction-and-guards/main.go +++ b/examples/extraction-and-guards/main.go @@ -1,8 +1,9 @@ -// Package main demonstrates data extraction and guard-based resource ordering. +// Package main demonstrates declared data extraction and guard-based resource +// ordering. // -// A single component manages a ConfigMap and a Secret. The ConfigMap's data -// extractor captures a value that the Secret's guard checks before allowing -// reconciliation to proceed. +// A single component manages a ConfigMap and a Secret. The ConfigMap declares +// an extraction into a shared data cell, and the Secret declares a data guard +// on that cell, blocking reconciliation until the ConfigMap has produced it. package main import ( @@ -56,9 +57,19 @@ func main() { NewSecretResource: resources.NewSecretResource, } - // Step 1: Normal reconciliation. The ConfigMap is created first, its data - // extractor captures db-host, and the Secret guard unblocks. - fmt.Println("--- Step 1: Normal reconciliation ---") + comp, _, err := controller.BuildComponent(owner) + if err != nil { + exit("failed to build component: %v", err) + } + fmt.Println("--- Declared data topology ---") + for _, edge := range comp.DataTopology() { + fmt.Printf(" data %q: producers=%v guarded=%v optional=%v\n", edge.Data, edge.Producers, edge.Guarded, edge.Optional) + } + + // Step 1: Normal reconciliation. The ConfigMap is created first, its + // declared extraction captures db-host, and the Secret's data guard + // unblocks. + fmt.Println("\n--- Step 1: Normal reconciliation ---") if err := controller.Reconcile(ctx, owner); err != nil { exit("reconciliation failed: %v", err) } diff --git a/examples/extraction-and-guards/resources/configmap.go b/examples/extraction-and-guards/resources/configmap.go index 89fa9693..f238754d 100644 --- a/examples/extraction-and-guards/resources/configmap.go +++ b/examples/extraction-and-guards/resources/configmap.go @@ -6,6 +6,7 @@ import ( "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/app" "github.com/sourcehawk/operator-component-framework/pkg/component" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/sourcehawk/operator-component-framework/pkg/primitives/configmap" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -27,16 +28,16 @@ func BaseConfigMap(owner *app.ExampleApp) *corev1.ConfigMap { } } -// NewConfigMapResource constructs a ConfigMap for database config. After -// reconciliation, the data extractor captures the db-host value into the -// provided pointer so downstream resources can use it. -func NewConfigMapResource(owner *app.ExampleApp, dbHost *string) (component.Resource, error) { +// NewConfigMapResource constructs a ConfigMap for database config. The +// declared extraction captures the db-host value into the provided cell so +// downstream resources can guard on it and read it. +func NewConfigMapResource(owner *app.ExampleApp, dbHost *concepts.Data[string]) (component.Resource, error) { builder := configmap.NewBuilder(BaseConfigMap(owner)) - builder.WithDataExtractor(func(cm corev1.ConfigMap) error { - *dbHost = cm.Data["db-host"] - fmt.Printf(" Extracted db-host: %q\n", *dbHost) - return nil + configmap.ExtractInto(builder, dbHost, func(cm corev1.ConfigMap) (string, error) { + host := cm.Data["db-host"] + fmt.Printf(" Extracted db-host: %q\n", host) + return host, nil }) return builder.Build() diff --git a/examples/extraction-and-guards/resources/configmap_test.go b/examples/extraction-and-guards/resources/configmap_test.go index fa59f3e4..120c2609 100644 --- a/examples/extraction-and-guards/resources/configmap_test.go +++ b/examples/extraction-and-guards/resources/configmap_test.go @@ -6,6 +6,7 @@ import ( "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/resources" sharedapp "github.com/sourcehawk/operator-component-framework/examples/shared/app" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/sourcehawk/operator-component-framework/pkg/testing/golden" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" @@ -24,16 +25,16 @@ func testOwner() *sharedapp.ExampleApp { } // TestConfigMapShape pins the database config ConfigMap as built by its factory. -// The factory registers a data extractor but no mutations, so the golden file -// captures the full desired state. If the base object changes (e.g. new keys -// added or defaults changed), the golden file catches it. +// The factory registers a declared extraction but no mutations, so the golden +// file captures the full desired state. If the base object changes (e.g. new +// keys added or defaults changed), the golden file catches it. func TestConfigMapShape(t *testing.T) { scheme := runtime.NewScheme() require.NoError(t, corev1.AddToScheme(scheme)) owner := testOwner() - var dbHost string - res, err := resources.NewConfigMapResource(owner, &dbHost) + dbHost := concepts.NewData[string]("db-host") + res, err := resources.NewConfigMapResource(owner, dbHost) require.NoError(t, err) golden.AssertYAML(t, "testdata/configmap.yaml", res.(golden.Previewer), diff --git a/examples/extraction-and-guards/resources/secret.go b/examples/extraction-and-guards/resources/secret.go index d3984975..203a438c 100644 --- a/examples/extraction-and-guards/resources/secret.go +++ b/examples/extraction-and-guards/resources/secret.go @@ -1,8 +1,6 @@ package resources import ( - "fmt" - "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/app" "github.com/sourcehawk/operator-component-framework/pkg/component" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -26,25 +24,24 @@ func BaseSecret(owner *app.ExampleApp) *corev1.Secret { } } -// NewSecretResource constructs a Secret for database credentials. A guard -// blocks this resource until the db-host value has been extracted from the -// preceding ConfigMap. -func NewSecretResource(owner *app.ExampleApp, dbHost *string) (component.Resource, error) { +// NewSecretResource constructs a Secret for database credentials. A declared +// data guard blocks it until the db-host cell has been extracted from the +// preceding ConfigMap, and a mutation copies the extracted host into the +// Secret so the credentials and endpoint travel together. +func NewSecretResource(owner *app.ExampleApp, dbHost *concepts.Data[string]) (component.Resource, error) { builder := secret.NewBuilder(BaseSecret(owner)) - builder.WithGuard(func(_ corev1.Secret) (concepts.GuardStatusWithReason, error) { - if *dbHost == "" { - fmt.Println(" Guard: blocked, waiting for db-host from ConfigMap") - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusBlocked, - Reason: "waiting for db-host to be extracted from ConfigMap", - }, nil - } - - fmt.Printf(" Guard: unblocked, db-host is %q\n", *dbHost) - return concepts.GuardStatusWithReason{ - Status: concepts.GuardStatusUnblocked, - }, nil + builder.WithDataGuard(dbHost) + builder.WithMutation(secret.Mutation{ + Name: "db-host-entry", + Mutate: func(m *secret.Mutator) error { + host, err := dbHost.Require() + if err != nil { + return err + } + m.SetStringData("db-host", host) + return nil + }, }) return builder.Build() diff --git a/examples/extraction-and-guards/resources/secret_test.go b/examples/extraction-and-guards/resources/secret_test.go index 6fc151a8..7bcd027c 100644 --- a/examples/extraction-and-guards/resources/secret_test.go +++ b/examples/extraction-and-guards/resources/secret_test.go @@ -4,23 +4,26 @@ import ( "testing" "github.com/sourcehawk/operator-component-framework/examples/extraction-and-guards/resources" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/sourcehawk/operator-component-framework/pkg/testing/golden" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/runtime" ) -// TestSecretShape pins the database credentials Secret as built by its factory. -// The factory registers a guard but no mutations, so the golden file captures -// the full desired state. The guard is not exercised here; this test only -// verifies the resource's desired state before reconciliation. +// TestSecretShape pins the database credentials Secret as built by its +// factory. The factory registers a data guard and a mutation that reads the +// extracted value, so the golden file captures the full desired state +// including the db-host entry the mutation writes. The seeded cell simulates +// the value a preceding ConfigMap extraction would have produced. func TestSecretShape(t *testing.T) { scheme := runtime.NewScheme() require.NoError(t, corev1.AddToScheme(scheme)) owner := testOwner() - var dbHost string - res, err := resources.NewSecretResource(owner, &dbHost) + dbHost := concepts.NewData[string]("db-host") + dbHost.Set("postgres.default.svc") + res, err := resources.NewSecretResource(owner, dbHost) require.NoError(t, err) golden.AssertYAML(t, "testdata/secret.yaml", res.(golden.Previewer), diff --git a/examples/extraction-and-guards/resources/testdata/secret.yaml b/examples/extraction-and-guards/resources/testdata/secret.yaml index 1f8900ae..c8c5f4c9 100644 --- a/examples/extraction-and-guards/resources/testdata/secret.yaml +++ b/examples/extraction-and-guards/resources/testdata/secret.yaml @@ -1,5 +1,6 @@ apiVersion: v1 data: + db-host: cG9zdGdyZXMuZGVmYXVsdC5zdmM= password: Y2hhbmdlbWU= username: YXBwLXVzZXI= kind: Secret diff --git a/pkg/component/builder.go b/pkg/component/builder.go index 164269f3..d31ef371 100644 --- a/pkg/component/builder.go +++ b/pkg/component/builder.go @@ -60,6 +60,10 @@ func (b *Builder) Build() (*Component, error) { )) } + cells, dataErrs := validateDataTopology(b.component.name, b.component.reconcileResources) + b.buildErrors = append(b.buildErrors, dataErrs...) + b.component.dataCells = cells + if len(b.buildErrors) > 0 { return nil, errors.Join(b.buildErrors...) } diff --git a/pkg/component/component.go b/pkg/component/component.go index 0d2f8327..8b42a7ef 100644 --- a/pkg/component/component.go +++ b/pkg/component/component.go @@ -112,6 +112,11 @@ type Component struct { // component reconciles for the first time. Once the component passes through // to normal reconciliation, prerequisites are never re-evaluated. prerequisites []Prerequisite + + // dataCells holds every declared data cell, in first-producer registration + // order, collected at Build time. Reconcile clears them all at the start of + // each pass so no extracted value leaks between reconciles. + dataCells []concepts.DataCell } // reconcileEntry pairs a resource with its configuration options. @@ -254,7 +259,10 @@ func (c *Component) Resource(identity string) (Resource, bool) { // once per reconciliation, typically via defer so that conditions set on error // paths are still written. // -// Reconciliation follows these steps: +// Before any step runs, every declared data cell on the component is cleared, +// so no value extracted during a previous reconcile leaks into this one. +// +// Reconciliation then follows these steps: // // 1. Feature gate check: If a feature gate is set and disabled, all resources // managed by the component are deleted and the condition is set to @@ -269,9 +277,15 @@ func (c *Component) Resource(identity string) (Resource, bool) { // is permanently cleared and prerequisites are never re-evaluated. // // 3. Suspension check: If the component is marked as suspended, it performs -// suspension of all managed (non-read-only) resources. Guards are not evaluated. -// The status is updated to reflect suspension progress (PendingSuspension, -// Suspending, or Suspended), and then deletion resources are processed. +// suspension of all managed (non-read-only) resources. Guards are not evaluated, +// but declared data extraction runs for each managed resource in registration +// order, so mutations that require a cell produced by an earlier managed +// resource still succeed while suspended. Cells produced by read-only resources +// (which are not fetched during suspension) or by resources deleted on suspend +// remain absent; readers of those must use Get, not Require, if the component +// can be suspended. The status is updated to reflect suspension progress +// (PendingSuspension, Suspending, or Suspended), and then deletion resources +// are processed. // // 4. Resource reconciliation: All non-delete resources are processed sequentially // in registration order, regardless of whether they are managed or read-only. @@ -279,7 +293,7 @@ func (c *Component) Resource(identity string) (Resource, bool) { // - Its guard (if any) is evaluated. A blocked guard stops processing of that // resource and all subsequent resources. // - The resource is either applied (managed) or fetched (read-only). -// - Its data extractors run immediately, making extracted data available to +// - Its declared data extractions run immediately, making extracted data available to // subsequent resources' guards and mutations. // // 5. Status Aggregation: Collects converging status from all processed resources @@ -300,6 +314,12 @@ func (c *Component) Reconcile(ctx context.Context, rec ReconcileContext) error { ) ctx = log.IntoContext(ctx, logger) + // Reset declared data cells before anything else runs so no extracted + // value leaks from a previous reconcile into this one. + for _, cell := range c.dataCells { + cell.Clear() + } + mapper := rec.Client.RESTMapper() if mapper == nil { return fail( diff --git a/pkg/component/concepts/data.go b/pkg/component/concepts/data.go new file mode 100644 index 00000000..389e0ee7 --- /dev/null +++ b/pkg/component/concepts/data.go @@ -0,0 +1,90 @@ +package concepts + +import ( + "errors" + "fmt" +) + +// ErrDataNotExtracted is returned (wrapped) by Data.Require when the cell has +// not been set during the current reconcile. Callers can match it with +// errors.Is to distinguish "not extracted yet" from other failures. +var ErrDataNotExtracted = errors.New("data not extracted") + +// DataCell is the non-generic view of a *Data[T] cell. It lets untyped code +// (builders, the component, introspection) hold heterogeneous cells without +// knowing their value type. Every *Data[T] satisfies it. +type DataCell interface { + // Name returns the diagnostic name of the cell. Cell identity is the + // pointer; the name exists for validation messages and introspection. + Name() string + // IsSet reports whether the cell currently holds an extracted value. + IsSet() bool + // Clear resets the cell's value and presence. It is called by the owning + // component at the start of each reconcile; calling it from user code is + // unsupported. + Clear() +} + +// Data is a named, typed, presence-aware cell for intra-component data flow. +// A cell is written by a declared extraction (ExtractInto on a builder) and +// read by later resources' guards and mutations within the same reconcile. +// +// Create cells inside the component assembly function so they stay scoped to +// a single reconcile. As a hardening, the owning component clears every +// declared cell at the start of each reconcile, so accidental reuse of a +// long-lived cell cannot leak state between reconciles. Sharing a cell across +// components is unsupported: validation and reset are per component. +// +// The presence flag separates "not extracted" from "extracted as the zero +// value". There is deliberately no panicking accessor: reconciler code must +// degrade to conditions and requeues, never crash the manager. +type Data[T any] struct { + name string + value T + set bool +} + +// NewData creates a new, unset data cell with the given diagnostic name. +// Within one component, no two distinct cells may share a name; the component +// builder rejects the collision at Build time. +func NewData[T any](name string) *Data[T] { + return &Data[T]{name: name} +} + +// Name returns the diagnostic name of the cell. +func (d *Data[T]) Name() string { return d.name } + +// IsSet reports whether the cell currently holds an extracted value. +func (d *Data[T]) IsSet() bool { return d.set } + +// Get returns the cell's value and whether it has been set. When the cell is +// unset, the value is the zero value of T. +func (d *Data[T]) Get() (T, bool) { return d.value, d.set } + +// Require returns the cell's value, or the zero value of T and an error +// wrapping ErrDataNotExtracted (naming the cell) when the cell is unset. +// Mutations propagate the error through their normal error path. +func (d *Data[T]) Require() (T, error) { + if !d.set { + var zero T + return zero, fmt.Errorf("data %q: %w", d.name, ErrDataNotExtracted) + } + return d.value, nil +} + +// Set stores a value in the cell and marks it present. Set is called by +// declared extractions (ExtractInto). The one supported manual use is a test +// seeding a cell before rendering a cluster-free preview; any other manual +// call bypasses topology validation and is unsupported. +func (d *Data[T]) Set(value T) { + d.value = value + d.set = true +} + +// Clear resets the cell to unset and the zero value of T. Clear is called by +// the owning component at the start of each reconcile. +func (d *Data[T]) Clear() { + var zero T + d.value = zero + d.set = false +} diff --git a/pkg/component/concepts/data_inspector.go b/pkg/component/concepts/data_inspector.go new file mode 100644 index 00000000..2249b9d4 --- /dev/null +++ b/pkg/component/concepts/data_inspector.go @@ -0,0 +1,54 @@ +package concepts + +// DataConsumption records one declared read of a data cell by a resource. +type DataConsumption struct { + // Cell is the cell being read. + Cell DataCell + // Optional reports the read mode: false means the resource blocks until + // the cell is set (WithDataGuard); true means the resource proceeds and + // reads opportunistically (WithOptionalData). + Optional bool +} + +// DataProducer is implemented by resources that declare data extractions. +// The component builder uses it to validate that every consumed cell has a +// producer registered strictly earlier, and the component uses it to know +// which cells to clear at the start of each reconcile. +type DataProducer interface { + // ProducedData returns the cells this resource extracts into, deduplicated, + // in declaration order. + ProducedData() []DataCell +} + +// DataConsumer is implemented by resources that declare data reads, either +// blocking (WithDataGuard) or optional (WithOptionalData). +type DataConsumer interface { + // ConsumedData returns the declared reads in declaration order. + ConsumedData() []DataConsumption +} + +// DataEdge describes the declared flow of one data cell through a component: +// which resources write it and which resources read it. +type DataEdge struct { + // Data is the cell name. + Data string + // Producers lists the resource identities declaring a write, in + // registration order. + Producers []string + // Guarded lists the resource identities blocking on the cell, in + // registration order. + Guarded []string + // Optional lists the resource identities optionally reading the cell, in + // registration order. + Optional []string +} + +// DataInspector surfaces, read-only, the declared data topology of a built +// component. It is the data-flow counterpart of MutationInspector: an inert +// capability that nothing in the reconcile path calls, so importing it costs +// nothing at runtime. +type DataInspector interface { + // DataTopology returns one edge per declared cell, in first-producer + // registration order. + DataTopology() []DataEdge +} diff --git a/pkg/component/concepts/data_test.go b/pkg/component/concepts/data_test.go new file mode 100644 index 00000000..90106c44 --- /dev/null +++ b/pkg/component/concepts/data_test.go @@ -0,0 +1,71 @@ +package concepts + +import ( + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestDataStartsUnset(t *testing.T) { + d := NewData[string]("db-host") + + assert.Equal(t, "db-host", d.Name()) + assert.False(t, d.IsSet()) + + v, ok := d.Get() + assert.False(t, ok) + assert.Empty(t, v) +} + +func TestDataRequireWhenUnset(t *testing.T) { + d := NewData[string]("db-host") + + v, err := d.Require() + require.Error(t, err) + assert.True(t, errors.Is(err, ErrDataNotExtracted)) + assert.Contains(t, err.Error(), `"db-host"`) + assert.Empty(t, v) +} + +func TestDataSetMarksPresent(t *testing.T) { + d := NewData[string]("db-host") + d.Set("postgres.default.svc") + + assert.True(t, d.IsSet()) + + v, ok := d.Get() + assert.True(t, ok) + assert.Equal(t, "postgres.default.svc", v) + + rv, err := d.Require() + require.NoError(t, err) + assert.Equal(t, "postgres.default.svc", rv) +} + +func TestDataSetZeroValueIsPresent(t *testing.T) { + d := NewData[string]("maybe-empty") + d.Set("") + + assert.True(t, d.IsSet()) + v, ok := d.Get() + assert.True(t, ok) + assert.Empty(t, v) +} + +func TestDataClearResetsValueAndPresence(t *testing.T) { + d := NewData[int]("replicas") + d.Set(3) + d.Clear() + + assert.False(t, d.IsSet()) + v, ok := d.Get() + assert.False(t, ok) + assert.Zero(t, v) +} + +func TestDataSatisfiesDataCell(t *testing.T) { + var cell DataCell = NewData[string]("x") + assert.Equal(t, "x", cell.Name()) +} diff --git a/pkg/component/concepts/extractable.go b/pkg/component/concepts/extractable.go index baa88582..5902a89a 100644 --- a/pkg/component/concepts/extractable.go +++ b/pkg/component/concepts/extractable.go @@ -1,22 +1,16 @@ package concepts -// DataExtractable defines the contract for resources that need to expose internal data -// after they have been created, updated, or fetched from the cluster. +// DataExtractable is the runtime hook through which the component triggers a +// resource's declared data extractions (see ExtractInto on the builders). +// Extraction runs immediately after each resource is applied or fetched during +// reconciliation, so data extracted from one resource is available to +// subsequent resources' guards and mutations within the same cycle, and always +// before the final component condition is calculated. // -// Implement this interface when a resource contains information (like generated credentials, -// endpoint URLs, or status fields) that needs to be pulled back into the operator's -// memory for use by other components or for updating the parent CRD's status. -// -// Data extraction is intended to be an observational/read-only operation on the resource. -// -// Extraction is triggered immediately after each resource is applied or fetched during -// reconciliation, regardless of whether the resource is managed or read-only. This allows -// data extracted from one resource to be available to subsequent resources' guards and -// mutations within the same reconciliation cycle. Extraction always occurs before the -// final component condition is calculated. +// All built-in primitives satisfy this through generic.BaseResource. User code +// does not call ExtractData; declare extractions on the builder instead. type DataExtractable interface { - // ExtractData performs the data extraction from the resource's underlying Kubernetes object. - // The implementation should store the extracted data in its own fields or shared state - // where it can be accessed by the caller. + // ExtractData runs the resource's declared data extractions against its + // reconciled Kubernetes object, storing each computed value in its cell. ExtractData() error } diff --git a/pkg/component/create.go b/pkg/component/create.go index 11b1969a..6e007dc9 100644 --- a/pkg/component/create.go +++ b/pkg/component/create.go @@ -125,6 +125,9 @@ func applyResource( // often have implicit dependencies (e.g., a Deployment depending on a ConfigMap). // 3. Status Collection: For each resource that implements a lifecycle concept interface, // its converging status is collected after the Apply operation. +// 4. Data Extraction: Each resource's declared data extractions run immediately after it +// is applied, so a later resource's mutations can read what an earlier one produced. +// Guards are not evaluated on this path; the caller uses reconcileResources for that. // // Server-Side Apply behavior: // - The resource's desired state is built via Object() + Mutate(), then patched into the @@ -152,6 +155,15 @@ func applyResources( if result != nil { results = append(results, *result) } + + // Per-resource data extraction: run immediately after the apply so that + // extracted data is available to subsequent resources' mutations. This + // path is used during suspension, where a consumer's content mutations + // still run and may Require a cell an earlier managed producer fills. + // extractResourceData already wraps failures with the resource identity. + if err := extractResourceData([]Resource{entry.Resource}); err != nil { + return nil, err + } } return results, nil @@ -232,10 +244,9 @@ func reconcileResources( // Per-resource data extraction: run immediately after processing so that // extracted data is available to subsequent resources' guards and mutations. + // extractResourceData already wraps failures with the resource identity. if err := extractResourceData([]Resource{resource}); err != nil { - return nil, fmt.Errorf( - "failed to extract data from resource %s: %w", resource.Identity(), err, - ) + return nil, err } } diff --git a/pkg/component/data.go b/pkg/component/data.go new file mode 100644 index 00000000..e91f061f --- /dev/null +++ b/pkg/component/data.go @@ -0,0 +1,150 @@ +package component + +import ( + "fmt" + "reflect" + + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" +) + +// isNilCell reports whether the cell is a nil interface or an interface holding +// a typed-nil value such as (*concepts.Data[string])(nil). Both forms panic on +// the first method call, so validation rejects them with a build error rather +// than letting the panic escape. It mirrors isNilResource in builder.go. +func isNilCell(cell concepts.DataCell) bool { + if cell == nil { + return true + } + v := reflect.ValueOf(cell) + switch v.Kind() { + case reflect.Ptr, reflect.Map, reflect.Slice, reflect.Func, reflect.Chan, reflect.Interface: + return v.IsNil() + default: + return false + } +} + +// validateDataTopology walks resources in registration order and validates the +// component's declared data flow: +// +// 1. Every cell a resource reads (guarded or optional) has at least one +// producer registered strictly earlier. +// 2. No two distinct cells within the component share a name. Pointer +// identity is what the checks run on; the name collision check exists so +// diagnostics and introspection stay unambiguous. +// +// It returns the declared cells in first-producer registration order (the set +// the component clears at the start of each reconcile) and all violations +// found. Only reconcile resources participate: delete and orphan resources +// never run extraction, so data declared on them is not considered. +func validateDataTopology(componentName string, entries []reconcileEntry) ([]concepts.DataCell, []error) { + var errs []error + produced := make(map[concepts.DataCell]struct{}) + names := make(map[string]concepts.DataCell) + var cells []concepts.DataCell + + checkName := func(identity string, cell concepts.DataCell) { + existing, ok := names[cell.Name()] + if !ok { + names[cell.Name()] = cell + return + } + if existing != cell { + errs = append(errs, fmt.Errorf( + "resource %q in component %q declares data %q, but a distinct cell already uses that name; data names must be unique within a component", + identity, componentName, cell.Name(), + )) + } + } + + for _, entry := range entries { + identity := entry.Resource.Identity() + + // Reads are checked before this resource's own writes so that a + // producer can never satisfy its own read: the producer must be + // registered strictly earlier. + if consumer, ok := entry.Resource.(concepts.DataConsumer); ok { + for _, consumption := range consumer.ConsumedData() { + if isNilCell(consumption.Cell) { + errs = append(errs, fmt.Errorf( + "resource %q in component %q declares a nil data cell read", identity, componentName, + )) + continue + } + checkName(identity, consumption.Cell) + if _, ok := produced[consumption.Cell]; !ok { + errs = append(errs, fmt.Errorf( + "resource %q reads data %q but no earlier resource produces it", + identity, consumption.Cell.Name(), + )) + } + } + } + + if producer, ok := entry.Resource.(concepts.DataProducer); ok { + for _, cell := range producer.ProducedData() { + if isNilCell(cell) { + errs = append(errs, fmt.Errorf( + "resource %q in component %q declares a nil data cell write", identity, componentName, + )) + continue + } + checkName(identity, cell) + if _, dup := produced[cell]; !dup { + produced[cell] = struct{}{} + cells = append(cells, cell) + } + } + } + } + + return cells, errs +} + +// DataTopology returns one edge per declared data cell, in first-producer +// registration order. Within an edge, producers and readers are listed in +// registration order. It satisfies concepts.DataInspector, giving tests and +// tooling the same read-only view of data flow that MutationInspector gives +// for mutations. Nothing in the reconcile path calls it. +func (c *Component) DataTopology() []concepts.DataEdge { + edges := make(map[concepts.DataCell]*concepts.DataEdge) + var order []concepts.DataCell + + for _, entry := range c.reconcileResources { + identity := entry.Resource.Identity() + + if producer, ok := entry.Resource.(concepts.DataProducer); ok { + for _, cell := range producer.ProducedData() { + edge, ok := edges[cell] + if !ok { + edge = &concepts.DataEdge{Data: cell.Name()} + edges[cell] = edge + order = append(order, cell) + } + edge.Producers = append(edge.Producers, identity) + } + } + + if consumer, ok := entry.Resource.(concepts.DataConsumer); ok { + for _, consumption := range consumer.ConsumedData() { + edge, ok := edges[consumption.Cell] + if !ok { + // Build validation guarantees every read has an earlier + // producer, so this only guards a hand-built Component. + continue + } + if consumption.Optional { + edge.Optional = append(edge.Optional, identity) + } else { + edge.Guarded = append(edge.Guarded, identity) + } + } + } + } + + out := make([]concepts.DataEdge, 0, len(order)) + for _, cell := range order { + out = append(out, *edges[cell]) + } + return out +} diff --git a/pkg/component/data_reconcile_test.go b/pkg/component/data_reconcile_test.go new file mode 100644 index 00000000..7d516d6b --- /dev/null +++ b/pkg/component/data_reconcile_test.go @@ -0,0 +1,287 @@ +package component + +import ( + "context" + + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" + "github.com/sourcehawk/operator-component-framework/pkg/generic" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +// noopMutator satisfies generic.FeatureMutator for test resources that need +// no mutation behavior. +type noopMutator struct{} + +func (*noopMutator) Apply() error { return nil } +func (*noopMutator) NextFeature() {} + +// fakeCellProducer is a managed Resource producing one string cell. Its +// extraction records whether the cell was already set when extraction ran, +// which is how the reconcile-start reset is observed. +type fakeCellProducer struct { + obj *corev1.ConfigMap + cell *concepts.Data[string] + setAtExtract []bool +} + +func (f *fakeCellProducer) Identity() string { + return "v1/ConfigMap/" + f.obj.Namespace + "/" + f.obj.Name +} +func (f *fakeCellProducer) Object() (client.Object, error) { return f.obj.DeepCopy(), nil } +func (f *fakeCellProducer) Mutate(client.Object) error { return nil } +func (f *fakeCellProducer) ExtractData() error { + f.setAtExtract = append(f.setAtExtract, f.cell.IsSet()) + f.cell.Set(f.obj.Data["db-host"]) + return nil +} +func (f *fakeCellProducer) ProducedData() []concepts.DataCell { + return []concepts.DataCell{f.cell} +} + +// suspendableCellProducer is a fakeCellProducer that also satisfies +// concepts.Suspendable with the no-op behavior a static managed resource has, +// so it takes part in the suspension path. +type suspendableCellProducer struct { + *fakeCellProducer +} + +func (*suspendableCellProducer) DeleteOnSuspend() bool { return false } +func (*suspendableCellProducer) Suspend() error { return nil } +func (*suspendableCellProducer) SuspensionStatus() (concepts.SuspensionStatusWithReason, error) { + return concepts.SuspensionStatusWithReason{ + Status: concepts.SuspensionStatusSuspended, + Reason: "static resource is always suspended", + }, nil +} + +// silentCellProducer declares production of a cell but has no extraction, so +// the cell stays unset. It stands in for a producer whose extraction has not +// run yet (for example an absent read-only source). +type silentCellProducer struct { + obj *corev1.ConfigMap + cell *concepts.Data[string] +} + +func (f *silentCellProducer) Identity() string { + return "v1/ConfigMap/" + f.obj.Namespace + "/" + f.obj.Name +} +func (f *silentCellProducer) Object() (client.Object, error) { return f.obj.DeepCopy(), nil } +func (f *silentCellProducer) Mutate(client.Object) error { return nil } +func (f *silentCellProducer) ProducedData() []concepts.DataCell { + return []concepts.DataCell{f.cell} +} + +func newGuardedConsumer(ns string, cell *concepts.Data[string], optional bool) Resource { + cm := &corev1.ConfigMap{} + cm.Name = "consumer" + cm.Namespace = ns + b := generic.NewStaticBuilder[*corev1.ConfigMap, *noopMutator]( + cm, + func(c *corev1.ConfigMap) string { return "v1/ConfigMap/" + c.Namespace + "/" + c.Name }, + func(*corev1.ConfigMap) *noopMutator { return &noopMutator{} }, + ) + if optional { + b.WithOptionalData(cell) + } else { + b.WithDataGuard(cell) + } + res, err := b.Build() + Expect(err).NotTo(HaveOccurred()) + return res +} + +// cellMutator is a generic.FeatureMutator over a ConfigMap. It keeps the +// object being mutated so a test mutation can write extracted data into it. +type cellMutator struct { + cm *corev1.ConfigMap +} + +func (*cellMutator) Apply() error { return nil } +func (*cellMutator) NextFeature() {} + +// newRequiringConsumer builds a managed ConfigMap that declares a data guard on +// cell and whose content mutation copies the cell's required value into the +// object. The mutation fails unless the cell was extracted earlier in the pass. +func newRequiringConsumer(ns string, cell *concepts.Data[string]) Resource { + cm := &corev1.ConfigMap{} + cm.Name = "consumer" + cm.Namespace = ns + b := generic.NewStaticBuilder[*corev1.ConfigMap, *cellMutator]( + cm, + func(c *corev1.ConfigMap) string { return "v1/ConfigMap/" + c.Namespace + "/" + c.Name }, + func(c *corev1.ConfigMap) *cellMutator { return &cellMutator{cm: c} }, + ) + b.WithDataGuard(cell) + b.WithMutation(generic.Mutation[*cellMutator]{ + Name: "copy-db-host", + Mutate: func(m *cellMutator) error { + value, err := cell.Require() + if err != nil { + return err + } + if m.cm.Data == nil { + m.cm.Data = map[string]string{} + } + m.cm.Data["db-host"] = value + return nil + }, + }) + res, err := b.Build() + Expect(err).NotTo(HaveOccurred()) + return res +} + +var _ = Describe("Declared data reconciliation", func() { + var ( + ctx = context.Background() + namespace string + owner *MockOperatorCRD + recCtx ReconcileContext + ) + + BeforeEach(func() { + namespace = createNamespace(ctx, "data-reconcile-test-") + owner = &MockOperatorCRD{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-owner", + Namespace: namespace, + }, + } + Expect(k8sClient.Create(ctx, owner)).To(Succeed()) + + recCtx = newTestReconcileContext(owner) + }) + + AfterEach(func() { + Expect(k8sClient.Delete(ctx, owner)).To(Succeed()) + }) + + It("clears declared cells at the start of each reconcile", func() { + cell := concepts.NewData[string]("db-host") + producer := &fakeCellProducer{ + obj: &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "producer", Namespace: namespace}, + Data: map[string]string{"db-host": "postgres"}, + }, + cell: cell, + } + + comp, err := NewComponentBuilder(). + WithName("data-reconcile-test"). + WithConditionType("DataReady"). + WithResource(producer). + Build() + Expect(err).NotTo(HaveOccurred()) + + // First reconcile: the cell starts unset, so extraction observes false. + Expect(comp.Reconcile(ctx, recCtx)).To(Succeed()) + Expect(producer.setAtExtract).To(Equal([]bool{false})) + Expect(cell.IsSet()).To(BeTrue()) + + // Second reconcile: without the reconcile-start reset, the cell would + // still be set from the previous pass. The second recorded false proves + // Reconcile cleared it before extraction ran. + Expect(comp.Reconcile(ctx, recCtx)).To(Succeed()) + Expect(producer.setAtExtract).To(Equal([]bool{false, false})) + }) + + It("blocks a guarded consumer with the generated reason and surfaces it on the condition", func() { + cell := concepts.NewData[string]("db-host") + producer := &silentCellProducer{ + obj: &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "producer", Namespace: namespace}, + }, + cell: cell, + } + consumer := newGuardedConsumer(namespace, cell, false) + + comp, err := NewComponentBuilder(). + WithName("data-reconcile-test"). + WithConditionType("DataReady"). + WithResource(producer). + WithResource(consumer). + Build() + Expect(err).NotTo(HaveOccurred()) + + Expect(comp.Reconcile(ctx, recCtx)).To(Succeed()) + + cond := comp.GetCondition(owner) + Expect(cond.Reason).To(Equal(string(GuardBlocked))) + Expect(cond.Message).To(ContainSubstring(`waiting for data "db-host"`)) + + // The guarded consumer must never have been created in the cluster. + var fetched corev1.ConfigMap + err = k8sClient.Get(ctx, client.ObjectKey{Name: "consumer", Namespace: namespace}, &fetched) + Expect(err).To(HaveOccurred()) + Expect(client.IgnoreNotFound(err)).To(Succeed()) + }) + + It("runs declared extractions while suspended so a requiring consumer still applies", func() { + cell := concepts.NewData[string]("db-host") + producer := &suspendableCellProducer{ + fakeCellProducer: &fakeCellProducer{ + obj: &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "producer", Namespace: namespace}, + Data: map[string]string{"db-host": "postgres"}, + }, + cell: cell, + }, + } + consumer := newRequiringConsumer(namespace, cell) + + comp, err := NewComponentBuilder(). + WithName("data-reconcile-test"). + WithConditionType("DataReady"). + WithResource(producer). + WithResource(consumer). + Suspend(true). + Build() + Expect(err).NotTo(HaveOccurred()) + + Expect(comp.Reconcile(ctx, recCtx)).To(Succeed()) + Expect(cell.IsSet()).To(BeTrue()) + + // The consumer's Require-based mutation could only succeed because the + // managed producer's extraction ran on the suspension path. + var fetched corev1.ConfigMap + Expect(k8sClient.Get(ctx, client.ObjectKey{Name: "consumer", Namespace: namespace}, &fetched)).To(Succeed()) + Expect(fetched.Data).To(HaveKeyWithValue("db-host", "postgres")) + + cond := comp.GetCondition(owner) + Expect(cond.Status).To(Equal(metav1.ConditionTrue)) + Expect(cond.Reason).To(Equal(string(Suspended))) + }) + + It("applies an optional consumer even when the cell is unset", func() { + cell := concepts.NewData[string]("db-host") + producer := &silentCellProducer{ + obj: &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "producer", Namespace: namespace}, + }, + cell: cell, + } + consumer := newGuardedConsumer(namespace, cell, true) + + comp, err := NewComponentBuilder(). + WithName("data-reconcile-test"). + WithConditionType("DataReady"). + WithResource(producer). + WithResource(consumer). + Build() + Expect(err).NotTo(HaveOccurred()) + + Expect(comp.Reconcile(ctx, recCtx)).To(Succeed()) + + // The optional consumer must have been created despite the unset cell. + var fetched corev1.ConfigMap + Expect(k8sClient.Get(ctx, client.ObjectKey{Name: "consumer", Namespace: namespace}, &fetched)).To(Succeed()) + + cond := comp.GetCondition(owner) + Expect(cond.Reason).NotTo(Equal(string(GuardBlocked))) + }) +}) diff --git a/pkg/component/data_test.go b/pkg/component/data_test.go new file mode 100644 index 00000000..556ee7e1 --- /dev/null +++ b/pkg/component/data_test.go @@ -0,0 +1,198 @@ +package component + +import ( + "testing" + + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// fakeDataResource is a minimal Resource with declared data produced and +// consumed. Build-time validation never touches Object or Mutate. +type fakeDataResource struct { + identity string + produced []concepts.DataCell + consumed []concepts.DataConsumption +} + +func (f *fakeDataResource) Identity() string { return f.identity } +func (f *fakeDataResource) Object() (client.Object, error) { return nil, nil } +func (f *fakeDataResource) Mutate(client.Object) error { return nil } +func (f *fakeDataResource) ProducedData() []concepts.DataCell { return f.produced } +func (f *fakeDataResource) ConsumedData() []concepts.DataConsumption { return f.consumed } + +func newDataComponentBuilder() *Builder { + return NewComponentBuilder().WithName("data-test").WithConditionType("DataReady") +} + +func TestBuildRejectsGuardedReadWithNoProducer(t *testing.T) { + cell := concepts.NewData[string]("db-host") + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: cell}}, + } + + _, err := newDataComponentBuilder().WithResource(consumer).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), `resource "v1/Secret/default/creds" reads data "db-host" but no earlier resource produces it`) +} + +func TestBuildRejectsOptionalReadWithNoProducer(t *testing.T) { + cell := concepts.NewData[string]("db-host") + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: cell, Optional: true}}, + } + + _, err := newDataComponentBuilder().WithResource(consumer).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), `reads data "db-host" but no earlier resource produces it`) +} + +func TestBuildRejectsProducerRegisteredAfterConsumer(t *testing.T) { + cell := concepts.NewData[string]("db-host") + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: cell}}, + } + producer := &fakeDataResource{ + identity: "v1/ConfigMap/default/config", + produced: []concepts.DataCell{cell}, + } + + _, err := newDataComponentBuilder().WithResource(consumer).WithResource(producer).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "no earlier resource produces it") +} + +func TestBuildRejectsTypedNilConsumedCell(t *testing.T) { + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: (*concepts.Data[string])(nil)}}, + } + + _, err := newDataComponentBuilder().WithResource(consumer).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "declares a nil data cell read") +} + +func TestBuildRejectsTypedNilProducedCell(t *testing.T) { + producer := &fakeDataResource{ + identity: "v1/ConfigMap/default/config", + produced: []concepts.DataCell{(*concepts.Data[string])(nil)}, + } + + _, err := newDataComponentBuilder().WithResource(producer).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "declares a nil data cell write") +} + +func TestBuildRejectsDistinctCellsSharingAName(t *testing.T) { + a := concepts.NewData[string]("db-host") + b := concepts.NewData[int]("db-host") + producerA := &fakeDataResource{identity: "v1/ConfigMap/default/a", produced: []concepts.DataCell{a}} + producerB := &fakeDataResource{identity: "v1/ConfigMap/default/b", produced: []concepts.DataCell{b}} + + _, err := newDataComponentBuilder().WithResource(producerA).WithResource(producerB).Build() + require.Error(t, err) + assert.Contains(t, err.Error(), `"db-host"`) + assert.Contains(t, err.Error(), "distinct") +} + +func TestBuildAllowsMultipleProducers(t *testing.T) { + cell := concepts.NewData[string]("db-host") + first := &fakeDataResource{identity: "v1/ConfigMap/default/a", produced: []concepts.DataCell{cell}} + second := &fakeDataResource{identity: "v1/ConfigMap/default/b", produced: []concepts.DataCell{cell}} + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: cell}}, + } + + comp, err := newDataComponentBuilder(). + WithResource(first).WithResource(second).WithResource(consumer).Build() + require.NoError(t, err) + require.NotNil(t, comp) +} + +func TestBuildAcceptsValidTopologyAndCollectsCells(t *testing.T) { + host := concepts.NewData[string]("db-host") + port := concepts.NewData[string]("db-port") + producer := &fakeDataResource{identity: "v1/ConfigMap/default/config", produced: []concepts.DataCell{host, port}} + consumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: host}, {Cell: port, Optional: true}}, + } + + comp, err := newDataComponentBuilder().WithResource(producer).WithResource(consumer).Build() + require.NoError(t, err) + require.Len(t, comp.dataCells, 2) + assert.Same(t, host, comp.dataCells[0].(*concepts.Data[string])) + assert.Same(t, port, comp.dataCells[1].(*concepts.Data[string])) +} + +func TestBuildIgnoresResourcesWithoutDataDeclarations(t *testing.T) { + plain := &fakeDataResource{identity: "v1/ConfigMap/default/plain"} + + comp, err := newDataComponentBuilder().WithResource(plain).Build() + require.NoError(t, err) + assert.Empty(t, comp.dataCells) +} + +func TestDataTopologyEdgesAndOrdering(t *testing.T) { + host := concepts.NewData[string]("db-host") + port := concepts.NewData[string]("db-port") + + configProducer := &fakeDataResource{ + identity: "v1/ConfigMap/default/config", + produced: []concepts.DataCell{host, port}, + } + secondHostProducer := &fakeDataResource{ + identity: "v1/ConfigMap/default/override", + produced: []concepts.DataCell{host}, + } + guardedConsumer := &fakeDataResource{ + identity: "v1/Secret/default/creds", + consumed: []concepts.DataConsumption{{Cell: host}}, + } + optionalConsumer := &fakeDataResource{ + identity: "v1/ConfigMap/default/enricher", + consumed: []concepts.DataConsumption{{Cell: host, Optional: true}, {Cell: port, Optional: true}}, + } + + comp, err := newDataComponentBuilder(). + WithResource(configProducer). + WithResource(secondHostProducer). + WithResource(guardedConsumer). + WithResource(optionalConsumer). + Build() + require.NoError(t, err) + + topology := comp.DataTopology() + require.Equal(t, []concepts.DataEdge{ + { + Data: "db-host", + Producers: []string{"v1/ConfigMap/default/config", "v1/ConfigMap/default/override"}, + Guarded: []string{"v1/Secret/default/creds"}, + Optional: []string{"v1/ConfigMap/default/enricher"}, + }, + { + Data: "db-port", + Producers: []string{"v1/ConfigMap/default/config"}, + Optional: []string{"v1/ConfigMap/default/enricher"}, + }, + }, topology) +} + +func TestDataTopologyEmptyComponent(t *testing.T) { + comp, err := newDataComponentBuilder(). + WithResource(&fakeDataResource{identity: "v1/ConfigMap/default/plain"}). + Build() + require.NoError(t, err) + assert.Empty(t, comp.DataTopology()) +} + +func TestComponentSatisfiesDataInspector(t *testing.T) { + var _ concepts.DataInspector = (*Component)(nil) +} diff --git a/pkg/component/resource_options.go b/pkg/component/resource_options.go index 2d42ec20..3e114f9b 100644 --- a/pkg/component/resource_options.go +++ b/pkg/component/resource_options.go @@ -67,7 +67,7 @@ type resourceOptions struct { BlockOnAbsence bool // IgnoreIfAbsent applies to read-only resources. When true, a NotFound response // when reading the resource is silently ignored: the entry is skipped, no - // condition or observation is recorded, the data extractor is not invoked, and + // condition or observation is recorded, no declared data extraction is run, and // reconciliation of subsequent resources continues. Last-known state is // preserved across an absence. Mutually exclusive with BlockOnAbsence. IgnoreIfAbsent bool diff --git a/pkg/generic/builder_base.go b/pkg/generic/builder_base.go index ca27a25e..c2f4134c 100644 --- a/pkg/generic/builder_base.go +++ b/pkg/generic/builder_base.go @@ -72,16 +72,36 @@ func (b *BaseBuilder[T, M]) WithGuard(handler func(T) (concepts.GuardStatusWithR b.BaseRes.GuardHandler = handler } -// WithDataExtractor registers a typed data extractor to run immediately after the -// resource has been processed during reconciliation. +// WithDataGuard declares that the resource reads the given cells and must not +// be applied until every one of them is set. The framework generates the guard +// and its reason (for example: waiting for data "db-host"), so the reason can +// never drift from the actual dependency. A blocked data guard surfaces as the +// same Blocked condition reason custom guards produce. // -// For managed resources, the extractor receives the object as it stands after feature -// mutations have been applied. For read-only resources, it receives the object as it -// was just fetched from the cluster. Extractors must be idempotent because they run on -// every reconcile pass. -func (b *BaseBuilder[T, M]) WithDataExtractor(extractor func(T) error) { - if extractor != nil { - b.BaseRes.DataExtractors = append(b.BaseRes.DataExtractors, extractor) +// Data guards are evaluated before any custom guard registered with WithGuard; +// both may be combined. Component Build validates that a producer for each +// cell is registered strictly earlier in the component. +func (b *BaseBuilder[T, M]) WithDataGuard(cells ...concepts.DataCell) { + for _, cell := range cells { + b.BaseRes.DataConsumptions = append( + b.BaseRes.DataConsumptions, + concepts.DataConsumption{Cell: cell, Optional: false}, + ) + } +} + +// WithOptionalData declares that the resource reads the given cells without +// gating on them. The declaration exists so component Build still verifies a +// producer is registered earlier (an optional read with no producer is +// permanently absent, which is dead code and almost certainly a bug) and so +// the dependency stays visible to introspection. Consumers in this mode use +// Get and skip quietly when the cell is absent. +func (b *BaseBuilder[T, M]) WithOptionalData(cells ...concepts.DataCell) { + for _, cell := range cells { + b.BaseRes.DataConsumptions = append( + b.BaseRes.DataConsumptions, + concepts.DataConsumption{Cell: cell, Optional: true}, + ) } } @@ -136,6 +156,29 @@ func (b *BaseBuilder[T, M]) ValidateBase() error { return errors.New("mutator factory cannot be nil") } + // Declared data extractions must reference a real cell and a real + // extraction function. A typed-nil cell or nil fn passed to ExtractInto is + // recorded and rejected here so the failure surfaces at build time with a + // clear message instead of panicking mid-reconcile. + for _, extraction := range b.BaseRes.DataExtractions { + if isNil(extraction.Cell) { + return errors.New("declared data extraction requires a non-nil cell") + } + if extraction.Extract == nil { + return fmt.Errorf( + "declared data extraction into %q requires a non-nil extraction function", + extraction.Cell.Name(), + ) + } + } + + // Declared data reads must reference a real cell for the same reason. + for _, consumption := range b.BaseRes.DataConsumptions { + if isNil(consumption.Cell) { + return errors.New("declared data read (WithDataGuard or WithOptionalData) requires a non-nil cell") + } + } + // Mutation names must be unique within a resource. A name is the identifier // that gating and error reporting refer to, so two mutations sharing one is // ambiguous: it silently masks a mis-targeted or dead mutation behind its diff --git a/pkg/generic/builder_integration.go b/pkg/generic/builder_integration.go index 9cd51d23..5b62a72a 100644 --- a/pkg/generic/builder_integration.go +++ b/pkg/generic/builder_integration.go @@ -59,11 +59,17 @@ func (b *IntegrationBuilder[T, M]) WithGuard( return b } -// WithDataExtractor registers a typed data extractor to run after successful reconciliation. -func (b *IntegrationBuilder[T, M]) WithDataExtractor( - extractor func(T) error, -) *IntegrationBuilder[T, M] { - b.BaseBuilder.WithDataExtractor(extractor) +// WithDataGuard declares blocking data reads for the integration resource. +// See BaseBuilder.WithDataGuard. +func (b *IntegrationBuilder[T, M]) WithDataGuard(cells ...concepts.DataCell) *IntegrationBuilder[T, M] { + b.BaseBuilder.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares non-blocking data reads for the integration +// resource. See BaseBuilder.WithOptionalData. +func (b *IntegrationBuilder[T, M]) WithOptionalData(cells ...concepts.DataCell) *IntegrationBuilder[T, M] { + b.BaseBuilder.WithOptionalData(cells...) return b } diff --git a/pkg/generic/builder_static.go b/pkg/generic/builder_static.go index 3d4f3a79..a71082e6 100644 --- a/pkg/generic/builder_static.go +++ b/pkg/generic/builder_static.go @@ -50,12 +50,17 @@ func (b *StaticBuilder[T, M]) WithGuard( return b } -// WithDataExtractor registers a typed data extractor to run after successful -// reconciliation. -func (b *StaticBuilder[T, M]) WithDataExtractor( - extractor func(T) error, -) *StaticBuilder[T, M] { - b.BaseBuilder.WithDataExtractor(extractor) +// WithDataGuard declares blocking data reads for the static resource. See +// BaseBuilder.WithDataGuard. +func (b *StaticBuilder[T, M]) WithDataGuard(cells ...concepts.DataCell) *StaticBuilder[T, M] { + b.BaseBuilder.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares non-blocking data reads for the static resource. +// See BaseBuilder.WithOptionalData. +func (b *StaticBuilder[T, M]) WithOptionalData(cells ...concepts.DataCell) *StaticBuilder[T, M] { + b.BaseBuilder.WithOptionalData(cells...) return b } diff --git a/pkg/generic/builder_static_test.go b/pkg/generic/builder_static_test.go index beb84746..76ac1bab 100644 --- a/pkg/generic/builder_static_test.go +++ b/pkg/generic/builder_static_test.go @@ -26,14 +26,6 @@ func TestStaticBuilder(t *testing.T) { assert.Equal(t, obj, res.DesiredObject) }) - t.Run("with data extractor", func(t *testing.T) { - extractor := func(_ *corev1.ConfigMap) error { return nil } - builder := NewStaticBuilder(obj, identityFunc, newMutator). - WithDataExtractor(extractor) - res, _ := builder.Build() - assert.Len(t, res.DataExtractors, 1) - }) - t.Run("with mutation", func(t *testing.T) { mut := Mutation[*mockMutator]{ Name: "test-mutation", diff --git a/pkg/generic/builder_task.go b/pkg/generic/builder_task.go index 1d2453a5..2d21acbd 100644 --- a/pkg/generic/builder_task.go +++ b/pkg/generic/builder_task.go @@ -52,11 +52,17 @@ func (b *TaskBuilder[T, M]) WithGuard( return b } -// WithDataExtractor registers a typed data extractor to run after successful reconciliation. -func (b *TaskBuilder[T, M]) WithDataExtractor( - extractor func(T) error, -) *TaskBuilder[T, M] { - b.BaseBuilder.WithDataExtractor(extractor) +// WithDataGuard declares blocking data reads for the task resource. See +// BaseBuilder.WithDataGuard. +func (b *TaskBuilder[T, M]) WithDataGuard(cells ...concepts.DataCell) *TaskBuilder[T, M] { + b.BaseBuilder.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares non-blocking data reads for the task resource. +// See BaseBuilder.WithOptionalData. +func (b *TaskBuilder[T, M]) WithOptionalData(cells ...concepts.DataCell) *TaskBuilder[T, M] { + b.BaseBuilder.WithOptionalData(cells...) return b } diff --git a/pkg/generic/builder_workload.go b/pkg/generic/builder_workload.go index ff90a569..ba696249 100644 --- a/pkg/generic/builder_workload.go +++ b/pkg/generic/builder_workload.go @@ -62,11 +62,17 @@ func (b *WorkloadBuilder[T, M]) WithGuard( return b } -// WithDataExtractor registers a typed data extractor to run after successful reconciliation. -func (b *WorkloadBuilder[T, M]) WithDataExtractor( - extractor func(T) error, -) *WorkloadBuilder[T, M] { - b.BaseBuilder.WithDataExtractor(extractor) +// WithDataGuard declares blocking data reads for the workload resource. See +// BaseBuilder.WithDataGuard. +func (b *WorkloadBuilder[T, M]) WithDataGuard(cells ...concepts.DataCell) *WorkloadBuilder[T, M] { + b.BaseBuilder.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares non-blocking data reads for the workload +// resource. See BaseBuilder.WithOptionalData. +func (b *WorkloadBuilder[T, M]) WithOptionalData(cells ...concepts.DataCell) *WorkloadBuilder[T, M] { + b.BaseBuilder.WithOptionalData(cells...) return b } diff --git a/pkg/generic/callback_helpers.go b/pkg/generic/callback_helpers.go index edd4d694..cea69d97 100644 --- a/pkg/generic/callback_helpers.go +++ b/pkg/generic/callback_helpers.go @@ -13,15 +13,3 @@ func WrapGuard[E any](guard func(E) (concepts.GuardStatusWithReason, error)) fun return guard(*ptr) } } - -// WrapExtractor converts a value-receiver data extractor callback into a -// pointer-receiver callback suitable for the generic builder layer. -// If the input function is nil, nil is returned. -func WrapExtractor[E any](extractor func(E) error) func(*E) error { - if extractor == nil { - return nil - } - return func(ptr *E) error { - return extractor(*ptr) - } -} diff --git a/pkg/generic/data.go b/pkg/generic/data.go new file mode 100644 index 00000000..86a27947 --- /dev/null +++ b/pkg/generic/data.go @@ -0,0 +1,62 @@ +package generic + +import ( + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// DataExtraction records one declared write of a data cell by a resource: the +// destination cell plus the function that computes and stores its value. It is +// recorded by ExtractInto; constructing it manually is unsupported. +type DataExtraction[T client.Object] struct { + // Cell is the destination cell, held through the non-generic DataCell view. + Cell concepts.DataCell + // Extract computes the value from the reconciled object and stores it in + // the cell. A nil Extract is rejected at Build time. + Extract func(T) error +} + +// ExtractInto declares that the resource built by b produces the value of +// cell. fn computes the value from the reconciled object; the framework stores +// it in the cell and marks it present. The extraction runs immediately after +// the resource is applied or fetched, before subsequent resources reconcile. +// +// This is a package-level function rather than a builder method because Go +// methods cannot introduce the extra type parameter V. +// +// Extracting several values from one object means several ExtractInto calls, +// one per cell. Multiple resources may produce the same cell; the last +// registered producer's extraction wins at runtime. +// +// A nil cell or nil fn is rejected when the builder's Build method runs. +func ExtractInto[T client.Object, M FeatureMutator, V any]( + b *BaseBuilder[T, M], cell *concepts.Data[V], fn func(T) (V, error), +) { + extraction := DataExtraction[T]{} + if cell != nil { + extraction.Cell = cell + } + if fn != nil { + extraction.Extract = func(obj T) error { + v, err := fn(obj) + if err != nil { + return err + } + cell.Set(v) + return nil + } + } + b.BaseRes.DataExtractions = append(b.BaseRes.DataExtractions, extraction) +} + +// WrapExtraction converts a value-receiver extraction callback into a +// pointer-receiver callback suitable for the generic layer's ExtractInto. +// If the input function is nil, nil is returned. +func WrapExtraction[E any, V any](fn func(E) (V, error)) func(*E) (V, error) { + if fn == nil { + return nil + } + return func(ptr *E) (V, error) { + return fn(*ptr) + } +} diff --git a/pkg/generic/data_test.go b/pkg/generic/data_test.go new file mode 100644 index 00000000..747ee310 --- /dev/null +++ b/pkg/generic/data_test.go @@ -0,0 +1,220 @@ +package generic + +import ( + "errors" + "testing" + + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func newDataTestBuilder() *StaticBuilder[*corev1.ConfigMap, *mockMutator] { + cm := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "cm", Namespace: "default"}, + Data: map[string]string{"db-host": "postgres.default.svc"}, + } + return NewStaticBuilder[*corev1.ConfigMap, *mockMutator]( + cm, + func(c *corev1.ConfigMap) string { return "v1/ConfigMap/" + c.Namespace + "/" + c.Name }, + func(*corev1.ConfigMap) *mockMutator { return &mockMutator{} }, + ) +} + +func TestExtractIntoSetsCellOnExtractData(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + ExtractInto(&b.BaseBuilder, cell, func(cm *corev1.ConfigMap) (string, error) { + return cm.Data["db-host"], nil + }) + + res, err := b.Build() + require.NoError(t, err) + assert.False(t, cell.IsSet()) + + require.NoError(t, res.ExtractData()) + + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "postgres.default.svc", v) +} + +func TestExtractIntoErrorLeavesCellUnsetAndNamesCell(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + ExtractInto(&b.BaseBuilder, cell, func(*corev1.ConfigMap) (string, error) { + return "", errors.New("boom") + }) + + res, err := b.Build() + require.NoError(t, err) + + extractErr := res.ExtractData() + require.Error(t, extractErr) + assert.Contains(t, extractErr.Error(), `"db-host"`) + assert.Contains(t, extractErr.Error(), "boom") + assert.False(t, cell.IsSet()) +} + +func TestProducedDataOrderAndDedupe(t *testing.T) { + host := concepts.NewData[string]("db-host") + port := concepts.NewData[string]("db-port") + b := newDataTestBuilder() + ExtractInto(&b.BaseBuilder, host, func(cm *corev1.ConfigMap) (string, error) { return cm.Data["db-host"], nil }) + ExtractInto(&b.BaseBuilder, port, func(cm *corev1.ConfigMap) (string, error) { return cm.Data["db-port"], nil }) + ExtractInto(&b.BaseBuilder, host, func(cm *corev1.ConfigMap) (string, error) { return cm.Data["db-host"], nil }) + + res, err := b.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 2) + assert.Same(t, host, produced[0].(*concepts.Data[string])) + assert.Same(t, port, produced[1].(*concepts.Data[string])) +} + +func TestExtractIntoLastProducerWins(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + ExtractInto(&b.BaseBuilder, cell, func(*corev1.ConfigMap) (string, error) { return "first", nil }) + ExtractInto(&b.BaseBuilder, cell, func(*corev1.ConfigMap) (string, error) { return "second", nil }) + + res, err := b.Build() + require.NoError(t, err) + require.NoError(t, res.ExtractData()) + + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "second", v) +} + +func TestExtractIntoNilCellRejectedAtBuild(t *testing.T) { + b := newDataTestBuilder() + ExtractInto[*corev1.ConfigMap, *mockMutator, string](&b.BaseBuilder, nil, func(*corev1.ConfigMap) (string, error) { + return "", nil + }) + + _, err := b.Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "non-nil cell") +} + +func TestExtractIntoNilFuncRejectedAtBuild(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + ExtractInto[*corev1.ConfigMap, *mockMutator, string](&b.BaseBuilder, cell, nil) + + _, err := b.Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "non-nil extraction function") +} + +func TestWrapExtraction(t *testing.T) { + fn := WrapExtraction(func(cm corev1.ConfigMap) (string, error) { return cm.Data["k"], nil }) + v, err := fn(&corev1.ConfigMap{Data: map[string]string{"k": "v"}}) + require.NoError(t, err) + assert.Equal(t, "v", v) + + assert.Nil(t, WrapExtraction[corev1.ConfigMap, string](nil)) +} + +func TestWithDataGuardBlocksUntilSet(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + b.WithDataGuard(cell) + + res, err := b.Build() + require.NoError(t, err) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + cell.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) +} + +func TestWithDataGuardListsAllMissingCells(t *testing.T) { + host := concepts.NewData[string]("db-host") + port := concepts.NewData[string]("db-port") + b := newDataTestBuilder() + b.WithDataGuard(host, port) + + res, err := b.Build() + require.NoError(t, err) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host", "db-port"`, status.Reason) +} + +func TestWithDataGuardRunsBeforeCustomGuard(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + b.WithDataGuard(cell) + customCalled := false + b.WithGuard(func(*corev1.ConfigMap) (concepts.GuardStatusWithReason, error) { + customCalled = true + return concepts.GuardStatusWithReason{Status: concepts.GuardStatusUnblocked}, nil + }) + + res, err := b.Build() + require.NoError(t, err) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.False(t, customCalled) + + cell.Set("x") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) + assert.True(t, customCalled) +} + +func TestWithOptionalDataNeverBlocks(t *testing.T) { + cell := concepts.NewData[string]("db-host") + b := newDataTestBuilder() + b.WithOptionalData(cell) + + res, err := b.Build() + require.NoError(t, err) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) +} + +func TestConsumedDataDeclarationOrderAndModes(t *testing.T) { + host := concepts.NewData[string]("db-host") + port := concepts.NewData[string]("db-port") + b := newDataTestBuilder() + b.WithDataGuard(host) + b.WithOptionalData(port) + + res, err := b.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Same(t, host, consumed[0].Cell.(*concepts.Data[string])) + assert.False(t, consumed[0].Optional) + assert.Same(t, port, consumed[1].Cell.(*concepts.Data[string])) + assert.True(t, consumed[1].Optional) +} + +func TestDataReadNilCellRejectedAtBuild(t *testing.T) { + b := newDataTestBuilder() + b.WithDataGuard(nil) + + _, err := b.Build() + require.Error(t, err) + assert.Contains(t, err.Error(), "non-nil cell") +} diff --git a/pkg/generic/resource_base.go b/pkg/generic/resource_base.go index 07332755..bcdb37e8 100644 --- a/pkg/generic/resource_base.go +++ b/pkg/generic/resource_base.go @@ -2,6 +2,8 @@ package generic import ( "fmt" + "strconv" + "strings" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "sigs.k8s.io/controller-runtime/pkg/client" @@ -13,7 +15,13 @@ type BaseResource[T client.Object, M FeatureMutator] struct { IdentityFunc func(T) string - DataExtractors []func(T) error + // DataExtractions holds the declared data extractions recorded by + // ExtractInto, run by ExtractData after the resource is applied or fetched. + DataExtractions []DataExtraction[T] + + // DataConsumptions holds the declared data reads recorded by WithDataGuard + // and WithOptionalData, in declaration order. + DataConsumptions []concepts.DataConsumption NewMutator func(T) M Mutations []Mutation[M] @@ -133,29 +141,52 @@ func (r *BaseResource[T, M]) Preview() (client.Object, error) { return r.PreviewObject() } -// ExtractData runs all registered data extractors against a deep copy of the reconciled object. +// ExtractData runs all declared data extractions against a deep copy of the +// reconciled object, storing each computed value in its cell. // -// For managed resources the reconciled object is the desired state produced by Mutate. -// For read-only resources it is the object most recently supplied via RecordObservation, -// which the read flow invokes after fetching from the cluster. +// For managed resources the reconciled object is the desired state produced by +// Mutate. For read-only resources it is the object most recently supplied via +// RecordObservation, which the read flow invokes after fetching from the +// cluster. Extractions run on every reconcile pass. func (r *BaseResource[T, M]) ExtractData() error { copyObj, ok := r.DesiredObject.DeepCopyObject().(T) if !ok { return fmt.Errorf("failed to deep copy object of type %T", r.DesiredObject) } - for _, extractor := range r.DataExtractors { - if extractor == nil { - continue - } - if err := extractor(copyObj); err != nil { - return err + for _, extraction := range r.DataExtractions { + if err := extraction.Extract(copyObj); err != nil { + return fmt.Errorf("extract data %q: %w", extraction.Cell.Name(), err) } } return nil } +// ProducedData returns the cells this resource declares extractions into, +// deduplicated by cell identity, in declaration order. It satisfies +// concepts.DataProducer. +func (r *BaseResource[T, M]) ProducedData() []concepts.DataCell { + seen := make(map[concepts.DataCell]struct{}, len(r.DataExtractions)) + cells := make([]concepts.DataCell, 0, len(r.DataExtractions)) + for _, extraction := range r.DataExtractions { + if _, ok := seen[extraction.Cell]; ok { + continue + } + seen[extraction.Cell] = struct{}{} + cells = append(cells, extraction.Cell) + } + return cells +} + +// ConsumedData returns the resource's declared data reads in declaration +// order. It satisfies concepts.DataConsumer. +func (r *BaseResource[T, M]) ConsumedData() []concepts.DataConsumption { + out := make([]concepts.DataConsumption, len(r.DataConsumptions)) + copy(out, r.DataConsumptions) + return out +} + // RecordObservation stores the supplied object as the resource's most recently observed // cluster state. The framework invokes this on read-only resources immediately after // fetching them, so that subsequent capabilities such as ExtractData observe the live @@ -175,10 +206,31 @@ func (r *BaseResource[T, M]) RecordObservation(observed client.Object) error { return nil } -// GuardStatus evaluates the resource's guard precondition. -// If no guard handler is configured, the resource is unconditionally unblocked. -// The handler receives a deep copy of the desired object to prevent accidental mutations. +// GuardStatus evaluates the resource's guard preconditions. +// +// Declared data guards (WithDataGuard) are evaluated first: if any guarded +// cell is unset, the resource is Blocked with a framework-generated reason +// naming the missing cells. Only when every guarded cell is set is the custom +// guard handler (WithGuard) consulted. If neither is configured, the resource +// is unconditionally unblocked. +// +// The custom handler receives a deep copy of the desired object to prevent +// accidental mutations. func (r *BaseResource[T, M]) GuardStatus() (concepts.GuardStatusWithReason, error) { + var missing []string + for _, consumption := range r.DataConsumptions { + if consumption.Optional || consumption.Cell.IsSet() { + continue + } + missing = append(missing, strconv.Quote(consumption.Cell.Name())) + } + if len(missing) > 0 { + return concepts.GuardStatusWithReason{ + Status: concepts.GuardStatusBlocked, + Reason: "waiting for data " + strings.Join(missing, ", "), + }, nil + } + if r.GuardHandler == nil { return concepts.GuardStatusWithReason{ Status: concepts.GuardStatusUnblocked, diff --git a/pkg/generic/resource_static_test.go b/pkg/generic/resource_static_test.go index 6ec99eab..9fd24475 100644 --- a/pkg/generic/resource_static_test.go +++ b/pkg/generic/resource_static_test.go @@ -66,30 +66,6 @@ func TestStaticResource(t *testing.T) { res.Mutations = nil }) - t.Run("ExtractData", func(t *testing.T) { - extracted := false - res.DataExtractors = []func(*corev1.ConfigMap) error{ - func(cm *corev1.ConfigMap) error { - extracted = true - assert.Equal(t, testVal, cm.Data["foo"]) - return nil - }, - } - err := res.ExtractData() - require.NoError(t, err) - assert.True(t, extracted, "extractor was not called") - }) - - t.Run("ExtractData error", func(t *testing.T) { - res.DataExtractors = []func(*corev1.ConfigMap) error{ - func(_ *corev1.ConfigMap) error { - return errors.New("extract error") - }, - } - err := res.ExtractData() - assert.EqualError(t, err, "extract error") - }) - t.Run("RecordObservation makes the observed object visible to ExtractData", func(t *testing.T) { base := &corev1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ @@ -114,16 +90,21 @@ func TestStaticResource(t *testing.T) { } require.NoError(t, readOnly.RecordObservation(observed)) - var seen string - readOnly.DataExtractors = []func(*corev1.ConfigMap) error{ - func(cm *corev1.ConfigMap) error { - seen = cm.Data["foo"] - return nil + cell := concepts.NewData[string]("foo") + readOnly.DataExtractions = []DataExtraction[*corev1.ConfigMap]{ + { + Cell: cell, + Extract: func(cm *corev1.ConfigMap) error { + cell.Set(cm.Data["foo"]) + return nil + }, }, } require.NoError(t, readOnly.ExtractData()) + seen, ok := cell.Get() + require.True(t, ok) assert.Equal(t, "from-cluster", seen, - "extractor must see the observed cluster object, not the empty desired base") + "the declared extraction must see the observed cluster object, not the empty desired base") }) t.Run("RecordObservation rejects an object of the wrong type", func(t *testing.T) { diff --git a/pkg/primitives/clusterrole/builder.go b/pkg/primitives/clusterrole/builder.go index 884b153a..a3eab214 100644 --- a/pkg/primitives/clusterrole/builder.go +++ b/pkg/primitives/clusterrole/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a ClusterRole Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -62,15 +62,22 @@ func (b *Builder) WithGuard(guard func(rbacv1.ClusterRole) (concepts.GuardStatus return b } -// WithDataExtractor registers a function to read values from the ClusterRole after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled ClusterRole. This is useful -// for surfacing generated or updated fields to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(rbacv1.ClusterRole) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the ClusterRole reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the ClusterRole reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -86,3 +93,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: res}, nil } + +// ExtractInto declares that this ClusterRole produces the value of cell. fn +// computes the value from a copy of the reconciled ClusterRole; the framework +// stores it in the cell and marks it present, immediately after the ClusterRole +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(rbacv1.ClusterRole) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/clusterrole/builder_test.go b/pkg/primitives/clusterrole/builder_test.go index d3c7283a..d61bcf04 100644 --- a/pkg/primitives/clusterrole/builder_test.go +++ b/pkg/primitives/clusterrole/builder_test.go @@ -1,9 +1,9 @@ package clusterrole import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" rbacv1 "k8s.io/api/rbac/v1" @@ -103,49 +103,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - cr := &rbacv1.ClusterRole{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cr"}, - } - called := false - extractor := func(_ rbacv1.ClusterRole) error { - called = true - return nil - } - res, err := NewBuilder(cr). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&rbacv1.ClusterRole{ + ObjectMeta: metav1.ObjectMeta{Name: "cr", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o rbacv1.ClusterRole) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&rbacv1.ClusterRole{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - cr := &rbacv1.ClusterRole{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cr"}, - } - res, err := NewBuilder(cr). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&rbacv1.ClusterRole{ + ObjectMeta: metav1.ObjectMeta{Name: "cr"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - cr := &rbacv1.ClusterRole{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cr"}, - } - res, err := NewBuilder(cr). - WithDataExtractor(func(_ rbacv1.ClusterRole) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&rbacv1.ClusterRole{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/clusterrole/resource.go b/pkg/primitives/clusterrole/resource.go index e5fdc575..117d9553 100644 --- a/pkg/primitives/clusterrole/resource.go +++ b/pkg/primitives/clusterrole/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // ClusterRole resources are static: they do not model convergence health, grace periods, // or suspension. Use a workload or task primitive for resources that require those concepts. @@ -49,7 +49,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled ClusterRole. // // This is called by the framework after successful reconciliation, allowing the @@ -58,9 +58,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this ClusterRole declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the ClusterRole's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -96,3 +109,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/clusterrole/resource_test.go b/pkg/primitives/clusterrole/resource_test.go index 81cab13f..9a3a5946 100644 --- a/pkg/primitives/clusterrole/resource_test.go +++ b/pkg/primitives/clusterrole/resource_test.go @@ -1,7 +1,6 @@ package clusterrole import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -122,32 +121,3 @@ func TestResource_Mutate_FeatureOrdering(t *testing.T) { assert.Equal(t, []string{"secrets"}, got.Rules[1].Resources) assert.Equal(t, []string{"configmaps"}, got.Rules[2].Resources) } - -func TestResource_ExtractData(t *testing.T) { - cr := newValidCR() - - var extracted string - res, err := NewBuilder(cr). - WithDataExtractor(func(c rbacv1.ClusterRole) error { - extracted = c.Name - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "test-cr", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidCR()). - WithDataExtractor(func(_ rbacv1.ClusterRole) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/clusterrolebinding/builder.go b/pkg/primitives/clusterrolebinding/builder.go index 3334e06b..6c3d2b52 100644 --- a/pkg/primitives/clusterrolebinding/builder.go +++ b/pkg/primitives/clusterrolebinding/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a ClusterRoleBinding Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -65,16 +65,22 @@ func (b *Builder) WithGuard(guard func(rbacv1.ClusterRoleBinding) (concepts.Guar return b } -// WithDataExtractor registers a function to read values from the ClusterRoleBinding -// after it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled ClusterRoleBinding. This -// is useful for surfacing generated or updated entries to other components or -// resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(rbacv1.ClusterRoleBinding) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the ClusterRoleBinding reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the ClusterRoleBinding reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -91,3 +97,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this ClusterRoleBinding produces the value of cell. fn +// computes the value from a copy of the reconciled ClusterRoleBinding; the framework +// stores it in the cell and marks it present, immediately after the ClusterRoleBinding +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(rbacv1.ClusterRoleBinding) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/clusterrolebinding/builder_test.go b/pkg/primitives/clusterrolebinding/builder_test.go index 93a17120..ccc9b3c2 100644 --- a/pkg/primitives/clusterrolebinding/builder_test.go +++ b/pkg/primitives/clusterrolebinding/builder_test.go @@ -1,9 +1,9 @@ package clusterrolebinding import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" rbacv1 "k8s.io/api/rbac/v1" @@ -85,49 +85,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - crb := &rbacv1.ClusterRoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-crb"}, - } - called := false - extractor := func(_ rbacv1.ClusterRoleBinding) error { - called = true - return nil - } - res, err := NewBuilder(crb). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&rbacv1.ClusterRoleBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "crb", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o rbacv1.ClusterRoleBinding) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&rbacv1.ClusterRoleBinding{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - crb := &rbacv1.ClusterRoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-crb"}, - } - res, err := NewBuilder(crb). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&rbacv1.ClusterRoleBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "crb"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - crb := &rbacv1.ClusterRoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-crb"}, - } - res, err := NewBuilder(crb). - WithDataExtractor(func(_ rbacv1.ClusterRoleBinding) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&rbacv1.ClusterRoleBinding{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/clusterrolebinding/resource.go b/pkg/primitives/clusterrolebinding/resource.go index b632ab1a..2a26b5ee 100644 --- a/pkg/primitives/clusterrolebinding/resource.go +++ b/pkg/primitives/clusterrolebinding/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // ClusterRoleBinding resources are static: they do not model convergence health, // grace periods, or suspension. Use a workload or task primitive for resources @@ -49,7 +49,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled ClusterRoleBinding. // // This is called by the framework after successful reconciliation, allowing the @@ -58,9 +58,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this ClusterRoleBinding declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the ClusterRoleBinding's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -96,3 +109,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/configmap/builder.go b/pkg/primitives/configmap/builder.go index 5dd4fe75..9c0c756b 100644 --- a/pkg/primitives/configmap/builder.go +++ b/pkg/primitives/configmap/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a ConfigMap Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -61,15 +61,22 @@ func (b *Builder) WithGuard(guard func(corev1.ConfigMap) (concepts.GuardStatusWi return b } -// WithDataExtractor registers a function to read values from the ConfigMap after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled ConfigMap. This is useful -// for surfacing generated or updated entries to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(corev1.ConfigMap) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the ConfigMap reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the ConfigMap reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -85,3 +92,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this ConfigMap produces the value of cell. fn +// computes the value from a copy of the reconciled ConfigMap; the framework +// stores it in the cell and marks it present, immediately after the ConfigMap +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.ConfigMap) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/configmap/builder_test.go b/pkg/primitives/configmap/builder_test.go index e834b11d..83325fa4 100644 --- a/pkg/primitives/configmap/builder_test.go +++ b/pkg/primitives/configmap/builder_test.go @@ -1,9 +1,9 @@ package configmap import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" @@ -74,49 +74,55 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - cm := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cm", Namespace: "test-ns"}, - } - called := false - extractor := func(_ corev1.ConfigMap) error { - called = true - return nil - } - res, err := NewBuilder(cm). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("db-host") + builder := NewBuilder(&corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "cm", Namespace: "default"}, + Data: map[string]string{"db-host": "postgres.default.svc"}, + }) + ExtractInto(builder, cell, func(cm corev1.ConfigMap) (string, error) { + return cm.Data["db-host"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&corev1.ConfigMap{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "db-host", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "postgres.default.svc", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - cm := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cm", Namespace: "test-ns"}, - } - res, err := NewBuilder(cm). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{Name: "cm", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - cm := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{Name: "test-cm", Namespace: "test-ns"}, - } - res, err := NewBuilder(cm). - WithDataExtractor(func(_ corev1.ConfigMap) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&corev1.ConfigMap{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/configmap/resource.go b/pkg/primitives/configmap/resource.go index 2fb3a8c0..1e63f0d0 100644 --- a/pkg/primitives/configmap/resource.go +++ b/pkg/primitives/configmap/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // ConfigMap resources are static: they do not model convergence health, grace periods, // or suspension. Use a workload or task primitive for resources that require those concepts. @@ -47,7 +47,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled ConfigMap. // // This is called by the framework after successful reconciliation, allowing the @@ -56,9 +56,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this ConfigMap declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the ConfigMap's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -94,3 +107,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/configmap/resource_test.go b/pkg/primitives/configmap/resource_test.go index c3e107f0..60161a97 100644 --- a/pkg/primitives/configmap/resource_test.go +++ b/pkg/primitives/configmap/resource_test.go @@ -1,7 +1,6 @@ package configmap import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -109,32 +108,3 @@ func TestResource_Mutate_FeatureOrdering(t *testing.T) { assert.Equal(t, "b", current.Data["order"]) } - -func TestResource_ExtractData(t *testing.T) { - cm := newValidCM() - - var extracted string - res, err := NewBuilder(cm). - WithDataExtractor(func(c corev1.ConfigMap) error { - extracted = c.Data["key"] - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "value", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidCM()). - WithDataExtractor(func(_ corev1.ConfigMap) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/cronjob/builder.go b/pkg/primitives/cronjob/builder.go index c75c4747..e7d05c88 100644 --- a/pkg/primitives/cronjob/builder.go +++ b/pkg/primitives/cronjob/builder.go @@ -12,7 +12,7 @@ import ( // Builder is a configuration helper for creating and customizing a CronJob Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.IntegrationBuilder[*batchv1.CronJob, *Mutator] @@ -121,12 +121,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// CronJob after it has been successfully reconciled. -func (b *Builder) WithDataExtractor( - extractor func(batchv1.CronJob) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the CronJob reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the CronJob reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -144,3 +154,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this CronJob produces the value of cell. fn +// computes the value from a copy of the reconciled CronJob; the framework +// stores it in the cell and marks it present, immediately after the CronJob +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(batchv1.CronJob) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/cronjob/builder_test.go b/pkg/primitives/cronjob/builder_test.go index 598c693c..7d8a63c4 100644 --- a/pkg/primitives/cronjob/builder_test.go +++ b/pkg/primitives/cronjob/builder_test.go @@ -172,42 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - cj := &batchv1.CronJob{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-cronjob", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ batchv1.CronJob) error { - called = true - return nil - } - res, err := NewBuilder(cj). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&batchv1.CronJob{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&batchv1.CronJob{ + ObjectMeta: metav1.ObjectMeta{Name: "cj", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - cj := &batchv1.CronJob{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-cronjob", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(cj). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o batchv1.CronJob) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&batchv1.CronJob{ + ObjectMeta: metav1.ObjectMeta{Name: "cj", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/cronjob/resource.go b/pkg/primitives/cronjob/resource.go index 71ffdb66..889a13b7 100644 --- a/pkg/primitives/cronjob/resource.go +++ b/pkg/primitives/cronjob/resource.go @@ -18,7 +18,7 @@ import ( // - concepts.Suspendable: for controlled suspension via spec.suspend. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. type Resource struct { base *generic.IntegrationResource[*batchv1.CronJob, *Mutator] } @@ -93,9 +93,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this CronJob declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the CronJob's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -131,3 +144,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/cronjob/resource_test.go b/pkg/primitives/cronjob/resource_test.go index d8ee0b8f..0488379d 100644 --- a/pkg/primitives/cronjob/resource_test.go +++ b/pkg/primitives/cronjob/resource_test.go @@ -1,7 +1,6 @@ package cronjob import ( - "errors" "testing" "time" @@ -313,32 +312,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) }) } - -func TestResource_ExtractData(t *testing.T) { - cj := newValidCronJob() - - var extractedImage string - res, err := NewBuilder(cj). - WithDataExtractor(func(c batchv1.CronJob) error { - extractedImage = c.Spec.JobTemplate.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "worker:latest", extractedImage) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidCronJob()). - WithDataExtractor(func(_ batchv1.CronJob) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/daemonset/builder.go b/pkg/primitives/daemonset/builder.go index 09e614ed..f6248534 100644 --- a/pkg/primitives/daemonset/builder.go +++ b/pkg/primitives/daemonset/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a DaemonSet Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.WorkloadBuilder[*appsv1.DaemonSet, *Mutator] @@ -153,16 +153,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// DaemonSet after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like names or assigned -// IPs) and making them available to other components or resources via the -// framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(appsv1.DaemonSet) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the DaemonSet reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the DaemonSet reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -180,3 +186,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this DaemonSet produces the value of cell. fn +// computes the value from a copy of the reconciled DaemonSet; the framework +// stores it in the cell and marks it present, immediately after the DaemonSet +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(appsv1.DaemonSet) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/daemonset/builder_test.go b/pkg/primitives/daemonset/builder_test.go index 05f2ba7d..c95a1c9c 100644 --- a/pkg/primitives/daemonset/builder_test.go +++ b/pkg/primitives/daemonset/builder_test.go @@ -193,42 +193,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.False(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - ds := &appsv1.DaemonSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-ds", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ appsv1.DaemonSet) error { - called = true - return nil - } - res, err := NewBuilder(ds). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&appsv1.DaemonSet{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&appsv1.DaemonSet{ + ObjectMeta: metav1.ObjectMeta{Name: "ds", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - ds := &appsv1.DaemonSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-ds", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(ds). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o appsv1.DaemonSet) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&appsv1.DaemonSet{ + ObjectMeta: metav1.ObjectMeta{Name: "ds", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/daemonset/resource.go b/pkg/primitives/daemonset/resource.go index 007fbcde..90f199bb 100644 --- a/pkg/primitives/daemonset/resource.go +++ b/pkg/primitives/daemonset/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for graceful deactivation via deletion. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a DaemonSet, including initial creation, // updates via feature mutations, and status monitoring. @@ -124,9 +124,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this DaemonSet declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the DaemonSet's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -162,3 +175,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/daemonset/resource_test.go b/pkg/primitives/daemonset/resource_test.go index d317f9ae..b4ab1c56 100644 --- a/pkg/primitives/daemonset/resource_test.go +++ b/pkg/primitives/daemonset/resource_test.go @@ -347,29 +347,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) }) } - -func TestResource_ExtractData(t *testing.T) { - ds := &appsv1.DaemonSet{ - ObjectMeta: metav1.ObjectMeta{Name: "test", Namespace: "default"}, - Spec: appsv1.DaemonSetSpec{ - Template: corev1.PodTemplateSpec{ - Spec: corev1.PodSpec{ - Containers: []corev1.Container{{Name: "web", Image: "nginx:latest"}}, - }, - }, - }, - } - - extractedImage := "" - res, err := NewBuilder(ds). - WithDataExtractor(func(d appsv1.DaemonSet) error { - extractedImage = d.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "nginx:latest", extractedImage) -} diff --git a/pkg/primitives/deployment/builder.go b/pkg/primitives/deployment/builder.go index 2a765c05..c00789b5 100644 --- a/pkg/primitives/deployment/builder.go +++ b/pkg/primitives/deployment/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a Deployment Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.WorkloadBuilder[*appsv1.Deployment, *Mutator] @@ -160,16 +160,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// Deployment after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like names or assigned -// IPs) and making them available to other components or resources via the -// framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(appsv1.Deployment) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Deployment reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Deployment reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -187,3 +193,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Deployment produces the value of cell. fn +// computes the value from a copy of the reconciled Deployment; the framework +// stores it in the cell and marks it present, immediately after the Deployment +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(appsv1.Deployment) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/deployment/builder_test.go b/pkg/primitives/deployment/builder_test.go index b16b8e4a..bcd07293 100644 --- a/pkg/primitives/deployment/builder_test.go +++ b/pkg/primitives/deployment/builder_test.go @@ -193,42 +193,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - deploy := &appsv1.Deployment{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-deploy", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ appsv1.Deployment) error { - called = true - return nil - } - res, err := NewBuilder(deploy). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&appsv1.Deployment{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&appsv1.Deployment{ + ObjectMeta: metav1.ObjectMeta{Name: "deploy", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - deploy := &appsv1.Deployment{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-deploy", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(deploy). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o appsv1.Deployment) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&appsv1.Deployment{ + ObjectMeta: metav1.ObjectMeta{Name: "deploy", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/deployment/resource.go b/pkg/primitives/deployment/resource.go index 0eca3e4c..6c94329c 100644 --- a/pkg/primitives/deployment/resource.go +++ b/pkg/primitives/deployment/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for graceful scale-down or temporary deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a Deployment, including initial creation, // updates via feature mutations, and status monitoring. @@ -134,15 +134,28 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro // assigned IP, or status fields) that might be needed by other resources or // higher-level controllers. // -// Data extractors are provided with a deep copy of the current Deployment to +// Declared data extractions are provided with a deep copy of the current Deployment to // prevent accidental mutations during the extraction process. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Deployment declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Deployment's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -178,3 +191,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/deployment/resource_test.go b/pkg/primitives/deployment/resource_test.go index 6e6d8b5e..32621f9e 100644 --- a/pkg/primitives/deployment/resource_test.go +++ b/pkg/primitives/deployment/resource_test.go @@ -339,29 +339,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) }) } - -func TestResource_ExtractData(t *testing.T) { - deploy := &appsv1.Deployment{ - ObjectMeta: metav1.ObjectMeta{Name: "test", Namespace: "default"}, - Spec: appsv1.DeploymentSpec{ - Template: corev1.PodTemplateSpec{ - Spec: corev1.PodSpec{ - Containers: []corev1.Container{{Name: "web", Image: "nginx:latest"}}, - }, - }, - }, - } - - extractedImage := "" - res, err := NewBuilder(deploy). - WithDataExtractor(func(d appsv1.Deployment) error { - extractedImage = d.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "nginx:latest", extractedImage) -} diff --git a/pkg/primitives/hpa/builder.go b/pkg/primitives/hpa/builder.go index 41a5d7ff..e5242405 100644 --- a/pkg/primitives/hpa/builder.go +++ b/pkg/primitives/hpa/builder.go @@ -12,7 +12,7 @@ import ( // Builder is a configuration helper for creating and customizing an HPA Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.IntegrationBuilder[*autoscalingv2.HorizontalPodAutoscaler, *Mutator] @@ -132,17 +132,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to read values from the HPA after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled HPA. This is useful -// for surfacing generated or updated fields to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor( - extractor func(autoscalingv2.HorizontalPodAutoscaler) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the HPA reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the HPA reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -158,3 +163,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this HPA produces the value of cell. fn +// computes the value from a copy of the reconciled HPA; the framework +// stores it in the cell and marks it present, immediately after the HPA +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(autoscalingv2.HorizontalPodAutoscaler) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/hpa/builder_test.go b/pkg/primitives/hpa/builder_test.go index 3f20e4f4..ec66b880 100644 --- a/pkg/primitives/hpa/builder_test.go +++ b/pkg/primitives/hpa/builder_test.go @@ -172,42 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.False(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - hpa := &autoscalingv2.HorizontalPodAutoscaler{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-hpa", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ autoscalingv2.HorizontalPodAutoscaler) error { - called = true - return nil - } - res, err := NewBuilder(hpa). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&autoscalingv2.HorizontalPodAutoscaler{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&autoscalingv2.HorizontalPodAutoscaler{ + ObjectMeta: metav1.ObjectMeta{Name: "hpa", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - hpa := &autoscalingv2.HorizontalPodAutoscaler{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-hpa", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(hpa). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o autoscalingv2.HorizontalPodAutoscaler) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&autoscalingv2.HorizontalPodAutoscaler{ + ObjectMeta: metav1.ObjectMeta{Name: "hpa", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/hpa/resource.go b/pkg/primitives/hpa/resource.go index abdcfd4a..1d74d365 100644 --- a/pkg/primitives/hpa/resource.go +++ b/pkg/primitives/hpa/resource.go @@ -18,7 +18,7 @@ import ( // prevent it from scaling the target back up during suspension. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. type Resource struct { base *generic.IntegrationResource[*autoscalingv2.HorizontalPodAutoscaler, *Mutator] } @@ -88,7 +88,7 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled HPA. // // This is called by the framework after successful reconciliation, allowing the @@ -97,9 +97,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this HPA declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the HPA's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -135,3 +148,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/ingress/builder.go b/pkg/primitives/ingress/builder.go index a0d90b8a..8e71f5d0 100644 --- a/pkg/primitives/ingress/builder.go +++ b/pkg/primitives/ingress/builder.go @@ -139,16 +139,22 @@ func (b *Builder) WithGuard(guard func(networkingv1.Ingress) (concepts.GuardStat return b } -// WithDataExtractor registers a function to read values from the Ingress after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled Ingress. This is useful -// for surfacing generated or updated entries (such as assigned load balancer -// addresses) to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(networkingv1.Ingress) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Ingress reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Ingress reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -164,3 +170,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Ingress produces the value of cell. fn +// computes the value from a copy of the reconciled Ingress; the framework +// stores it in the cell and marks it present, immediately after the Ingress +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(networkingv1.Ingress) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/ingress/builder_test.go b/pkg/primitives/ingress/builder_test.go index 04afaaef..26ecb892 100644 --- a/pkg/primitives/ingress/builder_test.go +++ b/pkg/primitives/ingress/builder_test.go @@ -172,42 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - ing := &networkingv1.Ingress{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-ing", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ networkingv1.Ingress) error { - called = true - return nil - } - res, err := NewBuilder(ing). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&networkingv1.Ingress{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&networkingv1.Ingress{ + ObjectMeta: metav1.ObjectMeta{Name: "ing", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - ing := &networkingv1.Ingress{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-ing", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(ing). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o networkingv1.Ingress) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&networkingv1.Ingress{ + ObjectMeta: metav1.ObjectMeta{Name: "ing", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/ingress/resource.go b/pkg/primitives/ingress/resource.go index 483bfac4..7a7a16bb 100644 --- a/pkg/primitives/ingress/resource.go +++ b/pkg/primitives/ingress/resource.go @@ -18,7 +18,7 @@ import ( // - concepts.Suspendable: for controlled suspension when the parent component is suspended. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // Ingress resources are integration primitives: they depend on an external ingress // controller to assign load balancer addresses. The default operational status handler @@ -101,7 +101,7 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled Ingress. // // This is called by the framework after successful reconciliation, allowing the @@ -111,9 +111,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Ingress declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Ingress's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -149,3 +162,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/ingress/resource_test.go b/pkg/primitives/ingress/resource_test.go index 203853aa..69d8c6cd 100644 --- a/pkg/primitives/ingress/resource_test.go +++ b/pkg/primitives/ingress/resource_test.go @@ -1,7 +1,6 @@ package ingress import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -214,32 +213,3 @@ func TestResource_SuspensionStatus(t *testing.T) { require.NoError(t, err) assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) } - -func TestResource_ExtractData(t *testing.T) { - ing := newValidIngress() - - var extracted string - res, err := NewBuilder(ing). - WithDataExtractor(func(i networkingv1.Ingress) error { - extracted = *i.Spec.IngressClassName - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "nginx", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidIngress()). - WithDataExtractor(func(_ networkingv1.Ingress) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/job/builder.go b/pkg/primitives/job/builder.go index 54e355b6..02b94a03 100644 --- a/pkg/primitives/job/builder.go +++ b/pkg/primitives/job/builder.go @@ -12,7 +12,7 @@ import ( // Builder is a configuration helper for creating and customizing a Job Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.TaskBuilder[*batchv1.Job, *Mutator] @@ -138,16 +138,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// Job after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like completion status -// or pod names) and making them available to other components or resources via -// the framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(batchv1.Job) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Job reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Job reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -165,3 +171,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Job produces the value of cell. fn +// computes the value from a copy of the reconciled Job; the framework +// stores it in the cell and marks it present, immediately after the Job +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(batchv1.Job) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/job/builder_test.go b/pkg/primitives/job/builder_test.go index d0133340..ada38c18 100644 --- a/pkg/primitives/job/builder_test.go +++ b/pkg/primitives/job/builder_test.go @@ -172,42 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.False(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - job := &batchv1.Job{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-job", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ batchv1.Job) error { - called = true - return nil - } - res, err := NewBuilder(job). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&batchv1.Job{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&batchv1.Job{ + ObjectMeta: metav1.ObjectMeta{Name: "job", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - job := &batchv1.Job{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-job", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(job). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o batchv1.Job) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&batchv1.Job{ + ObjectMeta: metav1.ObjectMeta{Name: "job", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/job/resource.go b/pkg/primitives/job/resource.go index 65177d0b..9f09e513 100644 --- a/pkg/primitives/job/resource.go +++ b/pkg/primitives/job/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for controlled deactivation (suspend or delete). // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a Job, including initial creation, // updates via feature mutations, and completion status monitoring. @@ -118,15 +118,28 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro // or generated names) that might be needed by other resources or higher-level // controllers. // -// Data extractors are provided with a deep copy of the current Job to +// Declared data extractions are provided with a deep copy of the current Job to // prevent accidental mutations during the extraction process. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Job declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Job's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -162,3 +175,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/job/resource_test.go b/pkg/primitives/job/resource_test.go index 9299a2a8..0f10b8f7 100644 --- a/pkg/primitives/job/resource_test.go +++ b/pkg/primitives/job/resource_test.go @@ -1,7 +1,6 @@ package job import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -337,36 +336,6 @@ func TestResource_ConvergingStatus(t *testing.T) { }) } -func TestResource_ExtractData(t *testing.T) { - job := newValidJob() - - extractedImage := "" - res, err := NewBuilder(job). - WithDataExtractor(func(j batchv1.Job) error { - extractedImage = j.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "busybox", extractedImage) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidJob()). - WithDataExtractor(func(_ batchv1.Job) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} - func int32Ptr(i int32) *int32 { return &i } diff --git a/pkg/primitives/networkpolicy/builder.go b/pkg/primitives/networkpolicy/builder.go index 436fd6bc..a82d11e6 100644 --- a/pkg/primitives/networkpolicy/builder.go +++ b/pkg/primitives/networkpolicy/builder.go @@ -12,7 +12,7 @@ import ( // Builder is a configuration helper for creating and customizing a NetworkPolicy // Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -62,15 +62,22 @@ func (b *Builder) WithGuard(guard func(networkingv1.NetworkPolicy) (concepts.Gua return b } -// WithDataExtractor registers a function to read values from the NetworkPolicy -// after it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled NetworkPolicy. This is -// useful for surfacing the applied policy rules to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(networkingv1.NetworkPolicy) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the NetworkPolicy reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the NetworkPolicy reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -86,3 +93,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this NetworkPolicy produces the value of cell. fn +// computes the value from a copy of the reconciled NetworkPolicy; the framework +// stores it in the cell and marks it present, immediately after the NetworkPolicy +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(networkingv1.NetworkPolicy) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/networkpolicy/builder_test.go b/pkg/primitives/networkpolicy/builder_test.go index 68bd89ed..e272be11 100644 --- a/pkg/primitives/networkpolicy/builder_test.go +++ b/pkg/primitives/networkpolicy/builder_test.go @@ -1,9 +1,9 @@ package networkpolicy import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" networkingv1 "k8s.io/api/networking/v1" @@ -74,49 +74,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - np := &networkingv1.NetworkPolicy{ - ObjectMeta: metav1.ObjectMeta{Name: "test-np", Namespace: "test-ns"}, - } - called := false - extractor := func(_ networkingv1.NetworkPolicy) error { - called = true - return nil - } - res, err := NewBuilder(np). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&networkingv1.NetworkPolicy{ + ObjectMeta: metav1.ObjectMeta{Name: "np", Namespace: "default", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o networkingv1.NetworkPolicy) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&networkingv1.NetworkPolicy{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - np := &networkingv1.NetworkPolicy{ - ObjectMeta: metav1.ObjectMeta{Name: "test-np", Namespace: "test-ns"}, - } - res, err := NewBuilder(np). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&networkingv1.NetworkPolicy{ + ObjectMeta: metav1.ObjectMeta{Name: "np", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - np := &networkingv1.NetworkPolicy{ - ObjectMeta: metav1.ObjectMeta{Name: "test-np", Namespace: "test-ns"}, - } - res, err := NewBuilder(np). - WithDataExtractor(func(_ networkingv1.NetworkPolicy) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&networkingv1.NetworkPolicy{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/networkpolicy/resource.go b/pkg/primitives/networkpolicy/resource.go index 01f0f6c3..8c5d49e7 100644 --- a/pkg/primitives/networkpolicy/resource.go +++ b/pkg/primitives/networkpolicy/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // NetworkPolicy resources are static: they do not model convergence health, grace // periods, or suspension. Use a workload or task primitive for resources that @@ -49,7 +49,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled NetworkPolicy. // // This is called by the framework after successful reconciliation, allowing the @@ -58,9 +58,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this NetworkPolicy declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the NetworkPolicy's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -96,3 +109,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/networkpolicy/resource_test.go b/pkg/primitives/networkpolicy/resource_test.go index b5123b31..f17f01bc 100644 --- a/pkg/primitives/networkpolicy/resource_test.go +++ b/pkg/primitives/networkpolicy/resource_test.go @@ -221,30 +221,3 @@ func TestResource_Mutate_MutationOrdering(t *testing.T) { assert.Equal(t, int32(80), got.Spec.Ingress[0].Ports[0].Port.IntVal) assert.Equal(t, int32(443), got.Spec.Ingress[1].Ports[0].Port.IntVal) } - -func TestResource_ExtractData(t *testing.T) { - np := &networkingv1.NetworkPolicy{ - ObjectMeta: metav1.ObjectMeta{Name: "test", Namespace: "default"}, - Spec: networkingv1.NetworkPolicySpec{ - PodSelector: metav1.LabelSelector{ - MatchLabels: map[string]string{"app": "test"}, - }, - PolicyTypes: []networkingv1.PolicyType{ - networkingv1.PolicyTypeIngress, - }, - }, - } - - extractedSelector := "" - res, err := NewBuilder(np). - WithDataExtractor(func(np networkingv1.NetworkPolicy) error { - extractedSelector = np.Spec.PodSelector.MatchLabels["app"] - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "test", extractedSelector) -} diff --git a/pkg/primitives/pdb/builder.go b/pkg/primitives/pdb/builder.go index 5b01a00c..99352c77 100644 --- a/pkg/primitives/pdb/builder.go +++ b/pkg/primitives/pdb/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a PodDisruptionBudget Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -61,15 +61,22 @@ func (b *Builder) WithGuard(guard func(policyv1.PodDisruptionBudget) (concepts.G return b } -// WithDataExtractor registers a function to read values from the PodDisruptionBudget -// after it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled PodDisruptionBudget. This is -// useful for surfacing generated or updated values to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(policyv1.PodDisruptionBudget) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the PodDisruptionBudget reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the PodDisruptionBudget reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -85,3 +92,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this PodDisruptionBudget produces the value of cell. fn +// computes the value from a copy of the reconciled PodDisruptionBudget; the framework +// stores it in the cell and marks it present, immediately after the PodDisruptionBudget +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(policyv1.PodDisruptionBudget) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/pdb/builder_test.go b/pkg/primitives/pdb/builder_test.go index 12d93089..5cdae2f2 100644 --- a/pkg/primitives/pdb/builder_test.go +++ b/pkg/primitives/pdb/builder_test.go @@ -1,9 +1,9 @@ package pdb import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" policyv1 "k8s.io/api/policy/v1" @@ -74,49 +74,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - p := &policyv1.PodDisruptionBudget{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pdb", Namespace: "test-ns"}, - } - called := false - extractor := func(_ policyv1.PodDisruptionBudget) error { - called = true - return nil - } - res, err := NewBuilder(p). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&policyv1.PodDisruptionBudget{ + ObjectMeta: metav1.ObjectMeta{Name: "pdb", Namespace: "default", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o policyv1.PodDisruptionBudget) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&policyv1.PodDisruptionBudget{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - p := &policyv1.PodDisruptionBudget{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pdb", Namespace: "test-ns"}, - } - res, err := NewBuilder(p). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&policyv1.PodDisruptionBudget{ + ObjectMeta: metav1.ObjectMeta{Name: "pdb", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - p := &policyv1.PodDisruptionBudget{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pdb", Namespace: "test-ns"}, - } - res, err := NewBuilder(p). - WithDataExtractor(func(_ policyv1.PodDisruptionBudget) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&policyv1.PodDisruptionBudget{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/pdb/resource.go b/pkg/primitives/pdb/resource.go index f2f6a307..f440223d 100644 --- a/pkg/primitives/pdb/resource.go +++ b/pkg/primitives/pdb/resource.go @@ -15,7 +15,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // PodDisruptionBudget resources are static: they do not model convergence health, // grace periods, or suspension. Use a workload or task primitive for resources @@ -49,7 +49,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled PodDisruptionBudget. // // This is called by the framework after successful reconciliation, allowing the @@ -58,9 +58,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this PodDisruptionBudget declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the PodDisruptionBudget's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -96,3 +109,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/pdb/resource_test.go b/pkg/primitives/pdb/resource_test.go index 82d50f61..8dfc86e5 100644 --- a/pkg/primitives/pdb/resource_test.go +++ b/pkg/primitives/pdb/resource_test.go @@ -1,7 +1,6 @@ package pdb import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -115,27 +114,3 @@ func TestResource_Mutate_FeatureOrdering(t *testing.T) { got := obj.(*policyv1.PodDisruptionBudget) assert.Equal(t, "b", got.Labels["order"]) } -func TestResource_ExtractData(t *testing.T) { - p := newValidPDB() - var extracted int32 - res, err := NewBuilder(p). - WithDataExtractor(func(pdb policyv1.PodDisruptionBudget) error { - extracted = pdb.Spec.MinAvailable.IntVal - return nil - }). - Build() - require.NoError(t, err) - require.NoError(t, res.ExtractData()) - assert.Equal(t, int32(2), extracted) -} -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidPDB()). - WithDataExtractor(func(_ policyv1.PodDisruptionBudget) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/pod/builder.go b/pkg/primitives/pod/builder.go index 97109bee..ad10db56 100644 --- a/pkg/primitives/pod/builder.go +++ b/pkg/primitives/pod/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a Pod Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.WorkloadBuilder[*corev1.Pod, *Mutator] @@ -152,16 +152,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// Pod after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like pod IP or node -// assignment) and making them available to other components or resources via -// the framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(corev1.Pod) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Pod reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Pod reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -179,3 +185,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Pod produces the value of cell. fn +// computes the value from a copy of the reconciled Pod; the framework +// stores it in the cell and marks it present, immediately after the Pod +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.Pod) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/pod/builder_test.go b/pkg/primitives/pod/builder_test.go index eea10ba3..9ee72f1c 100644 --- a/pkg/primitives/pod/builder_test.go +++ b/pkg/primitives/pod/builder_test.go @@ -193,42 +193,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.False(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - pod := &corev1.Pod{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-pod", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ corev1.Pod) error { - called = true - return nil - } - res, err := NewBuilder(pod). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&corev1.Pod{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&corev1.Pod{ + ObjectMeta: metav1.ObjectMeta{Name: "pod", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - pod := &corev1.Pod{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-pod", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(pod). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o corev1.Pod) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.Pod{ + ObjectMeta: metav1.ObjectMeta{Name: "pod", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/pod/resource.go b/pkg/primitives/pod/resource.go index a858f4ed..349ee30d 100644 --- a/pkg/primitives/pod/resource.go +++ b/pkg/primitives/pod/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for deletion-based deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a Pod, including initial creation, // updates via feature mutations, and status monitoring. @@ -118,15 +118,28 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro // or status fields) that might be needed by other resources or higher-level // controllers. // -// Data extractors are provided with a deep copy of the current Pod to +// Declared data extractions are provided with a deep copy of the current Pod to // prevent accidental mutations during the extraction process. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Pod declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Pod's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -162,3 +175,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/pod/resource_test.go b/pkg/primitives/pod/resource_test.go index dde7bbda..56c1a98c 100644 --- a/pkg/primitives/pod/resource_test.go +++ b/pkg/primitives/pod/resource_test.go @@ -1,7 +1,6 @@ package pod import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -344,33 +343,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, "Pod deleted on suspend", status.Reason) }) } - -func TestResource_ExtractData(t *testing.T) { - pod := newValidPod() - - extractedImage := "" - res, err := NewBuilder(pod). - WithDataExtractor(func(p corev1.Pod) error { - extractedImage = p.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "nginx:latest", extractedImage) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidPod()). - WithDataExtractor(func(_ corev1.Pod) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/pv/builder.go b/pkg/primitives/pv/builder.go index 923be306..951a5ec3 100644 --- a/pkg/primitives/pv/builder.go +++ b/pkg/primitives/pv/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a PersistentVolume Resource. // // It provides a fluent API for registering mutations, operational status handlers, -// and data extractors. Build() validates the configuration and returns an +// and declared data extractions. Build() validates the configuration and returns an // initialized Resource ready for use in a reconciliation loop. type Builder struct { base *generic.IntegrationBuilder[*corev1.PersistentVolume, *Mutator] @@ -95,15 +95,22 @@ func (b *Builder) WithGuard(guard func(corev1.PersistentVolume) (concepts.GuardS return b } -// WithDataExtractor registers a function to read values from the PersistentVolume -// after it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled PersistentVolume. This is -// useful for surfacing generated or updated fields to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(corev1.PersistentVolume) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the PersistentVolume reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the PersistentVolume reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -122,3 +129,13 @@ func (b *Builder) Build() (*Resource, error) { return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this PersistentVolume produces the value of cell. fn +// computes the value from a copy of the reconciled PersistentVolume; the framework +// stores it in the cell and marks it present, immediately after the PersistentVolume +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.PersistentVolume) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/pv/builder_test.go b/pkg/primitives/pv/builder_test.go index 01cec72a..d41f5feb 100644 --- a/pkg/primitives/pv/builder_test.go +++ b/pkg/primitives/pv/builder_test.go @@ -1,7 +1,6 @@ package pv import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -93,49 +92,54 @@ func TestBuilder_WithCustomOperationalStatus(t *testing.T) { assert.Equal(t, concepts.OperationalStatusOperational, status.Status) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - pv := &corev1.PersistentVolume{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pv"}, - } - called := false - extractor := func(_ corev1.PersistentVolume) error { - called = true - return nil - } - res, err := NewBuilder(pv). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&corev1.PersistentVolume{ + ObjectMeta: metav1.ObjectMeta{Name: "pv", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o corev1.PersistentVolume) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&corev1.PersistentVolume{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - pv := &corev1.PersistentVolume{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pv"}, - } - res, err := NewBuilder(pv). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.PersistentVolume{ + ObjectMeta: metav1.ObjectMeta{Name: "pv"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - pv := &corev1.PersistentVolume{ - ObjectMeta: metav1.ObjectMeta{Name: "test-pv"}, - } - res, err := NewBuilder(pv). - WithDataExtractor(func(_ corev1.PersistentVolume) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&corev1.PersistentVolume{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/pv/resource.go b/pkg/primitives/pv/resource.go index b45463d2..5877ecbe 100644 --- a/pkg/primitives/pv/resource.go +++ b/pkg/primitives/pv/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Graceful: for assessing health after the grace period expires. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. type Resource struct { base *generic.IntegrationResource[*corev1.PersistentVolume, *Mutator] } @@ -66,7 +66,7 @@ func (r *Resource) GraceStatus() (concepts.GraceStatusWithReason, error) { return r.base.GraceStatus() } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled PersistentVolume. // // This is called by the framework after successful reconciliation, allowing the @@ -75,9 +75,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this PersistentVolume declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the PersistentVolume's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -113,3 +126,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/pv/resource_test.go b/pkg/primitives/pv/resource_test.go index 47d833e5..f5337a89 100644 --- a/pkg/primitives/pv/resource_test.go +++ b/pkg/primitives/pv/resource_test.go @@ -1,7 +1,6 @@ package pv import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -132,31 +131,3 @@ func TestResource_ConvergingStatus(t *testing.T) { // Default handler on a PV with no phase set returns OperationPending. assert.Equal(t, concepts.OperationalStatusPending, status.Status) } - -func TestResource_ExtractData(t *testing.T) { - pv := newValidPV() - var extracted string - res, err := NewBuilder(pv). - WithDataExtractor(func(p corev1.PersistentVolume) error { - extracted = p.Spec.HostPath.Path - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "/data", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidPV()). - WithDataExtractor(func(_ corev1.PersistentVolume) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/pvc/builder.go b/pkg/primitives/pvc/builder.go index b0d9b648..5fbe0a7b 100644 --- a/pkg/primitives/pvc/builder.go +++ b/pkg/primitives/pvc/builder.go @@ -133,16 +133,22 @@ func (b *Builder) WithGuard(guard func(corev1.PersistentVolumeClaim) (concepts.G return b } -// WithDataExtractor registers a function to read values from the PVC after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled PVC. This is useful -// for surfacing the bound volume name, capacity, or other status fields to -// other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(corev1.PersistentVolumeClaim) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the PVC reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the PVC reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -158,3 +164,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this PVC produces the value of cell. fn +// computes the value from a copy of the reconciled PVC; the framework +// stores it in the cell and marks it present, immediately after the PVC +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.PersistentVolumeClaim) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/pvc/builder_test.go b/pkg/primitives/pvc/builder_test.go index 34a990c7..2a932213 100644 --- a/pkg/primitives/pvc/builder_test.go +++ b/pkg/primitives/pvc/builder_test.go @@ -8,7 +8,6 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/api/resource" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -173,49 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - p := &corev1.PersistentVolumeClaim{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-pvc", - Namespace: "test-ns", - }, - Spec: corev1.PersistentVolumeClaimSpec{ - Resources: corev1.VolumeResourceRequirements{ - Requests: corev1.ResourceList{ - corev1.ResourceStorage: resource.MustParse("10Gi"), - }, - }, - }, - } - called := false - extractor := func(_ corev1.PersistentVolumeClaim) error { - called = true - return nil - } - res, err := NewBuilder(p). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&corev1.PersistentVolumeClaim{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&corev1.PersistentVolumeClaim{ + ObjectMeta: metav1.ObjectMeta{Name: "pvc", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - p := &corev1.PersistentVolumeClaim{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-pvc", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(p). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o corev1.PersistentVolumeClaim) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.PersistentVolumeClaim{ + ObjectMeta: metav1.ObjectMeta{Name: "pvc", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/pvc/resource.go b/pkg/primitives/pvc/resource.go index 1106e234..07d85529 100644 --- a/pkg/primitives/pvc/resource.go +++ b/pkg/primitives/pvc/resource.go @@ -17,7 +17,7 @@ import ( // - concepts.Suspendable: for controlled suspension (e.g. retaining the PVC while suspending consumers). // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // PVC resources follow the Integration lifecycle: they are operationally significant // (a PVC must be Bound to be useful) and support suspension semantics. @@ -96,7 +96,7 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled PVC. // // This is called by the framework after successful reconciliation, allowing the @@ -105,9 +105,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this PVC declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the PVC's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -143,3 +156,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/pvc/resource_test.go b/pkg/primitives/pvc/resource_test.go index 5ceff000..4b51f08d 100644 --- a/pkg/primitives/pvc/resource_test.go +++ b/pkg/primitives/pvc/resource_test.go @@ -160,19 +160,3 @@ func TestResource_Suspend_And_SuspensionStatus(t *testing.T) { require.NoError(t, err) assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) } - -func TestResource_ExtractData(t *testing.T) { - pvc := newValidPVC() - - var extracted resource.Quantity - res, err := NewBuilder(pvc). - WithDataExtractor(func(p corev1.PersistentVolumeClaim) error { - extracted = p.Spec.Resources.Requests[corev1.ResourceStorage] - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, resource.MustParse("10Gi"), extracted) -} diff --git a/pkg/primitives/replicaset/builder.go b/pkg/primitives/replicaset/builder.go index aeb70630..7a62183a 100644 --- a/pkg/primitives/replicaset/builder.go +++ b/pkg/primitives/replicaset/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a ReplicaSet Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.WorkloadBuilder[*appsv1.ReplicaSet, *Mutator] @@ -153,16 +153,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// ReplicaSet after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like names or assigned -// IPs) and making them available to other components or resources via the -// framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(appsv1.ReplicaSet) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the ReplicaSet reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the ReplicaSet reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -180,3 +186,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this ReplicaSet produces the value of cell. fn +// computes the value from a copy of the reconciled ReplicaSet; the framework +// stores it in the cell and marks it present, immediately after the ReplicaSet +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(appsv1.ReplicaSet) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/replicaset/builder_test.go b/pkg/primitives/replicaset/builder_test.go index 8c4c4c3e..d61f60c3 100644 --- a/pkg/primitives/replicaset/builder_test.go +++ b/pkg/primitives/replicaset/builder_test.go @@ -193,42 +193,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - rs := &appsv1.ReplicaSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-rs", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ appsv1.ReplicaSet) error { - called = true - return nil - } - res, err := NewBuilder(rs). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&appsv1.ReplicaSet{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&appsv1.ReplicaSet{ + ObjectMeta: metav1.ObjectMeta{Name: "rs", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - rs := &appsv1.ReplicaSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-rs", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(rs). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o appsv1.ReplicaSet) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&appsv1.ReplicaSet{ + ObjectMeta: metav1.ObjectMeta{Name: "rs", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/replicaset/resource.go b/pkg/primitives/replicaset/resource.go index ddff18c0..51c0fa77 100644 --- a/pkg/primitives/replicaset/resource.go +++ b/pkg/primitives/replicaset/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for graceful scale-down or temporary deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a ReplicaSet, including initial creation, // updates via feature mutations, and status monitoring. @@ -112,15 +112,28 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro // ExtractData executes registered data extraction functions to harvest information // from the reconciled ReplicaSet. // -// Data extractors are provided with a deep copy of the current ReplicaSet to +// Declared data extractions are provided with a deep copy of the current ReplicaSet to // prevent accidental mutations during the extraction process. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this ReplicaSet declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the ReplicaSet's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -156,3 +169,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/replicaset/resource_test.go b/pkg/primitives/replicaset/resource_test.go index 314d9791..2b5c29de 100644 --- a/pkg/primitives/replicaset/resource_test.go +++ b/pkg/primitives/replicaset/resource_test.go @@ -347,29 +347,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) }) } - -func TestResource_ExtractData(t *testing.T) { - rs := &appsv1.ReplicaSet{ - ObjectMeta: metav1.ObjectMeta{Name: "test", Namespace: "default"}, - Spec: appsv1.ReplicaSetSpec{ - Template: corev1.PodTemplateSpec{ - Spec: corev1.PodSpec{ - Containers: []corev1.Container{{Name: "web", Image: "nginx:latest"}}, - }, - }, - }, - } - - extractedImage := "" - res, err := NewBuilder(rs). - WithDataExtractor(func(r appsv1.ReplicaSet) error { - extractedImage = r.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "nginx:latest", extractedImage) -} diff --git a/pkg/primitives/role/builder.go b/pkg/primitives/role/builder.go index ed92c627..4192ac78 100644 --- a/pkg/primitives/role/builder.go +++ b/pkg/primitives/role/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a Role Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -61,15 +61,22 @@ func (b *Builder) WithGuard(guard func(rbacv1.Role) (concepts.GuardStatusWithRea return b } -// WithDataExtractor registers a function to read values from the Role after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled Role. This is useful -// for surfacing generated or updated entries to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(rbacv1.Role) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Role reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Role reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -85,3 +92,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Role produces the value of cell. fn +// computes the value from a copy of the reconciled Role; the framework +// stores it in the cell and marks it present, immediately after the Role +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(rbacv1.Role) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/role/builder_test.go b/pkg/primitives/role/builder_test.go index 0f62863f..9d6033cc 100644 --- a/pkg/primitives/role/builder_test.go +++ b/pkg/primitives/role/builder_test.go @@ -1,9 +1,9 @@ package role import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" rbacv1 "k8s.io/api/rbac/v1" @@ -74,49 +74,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - role := &rbacv1.Role{ - ObjectMeta: metav1.ObjectMeta{Name: "test-role", Namespace: "test-ns"}, - } - called := false - extractor := func(_ rbacv1.Role) error { - called = true - return nil - } - res, err := NewBuilder(role). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&rbacv1.Role{ + ObjectMeta: metav1.ObjectMeta{Name: "role", Namespace: "default", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o rbacv1.Role) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&rbacv1.Role{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - role := &rbacv1.Role{ - ObjectMeta: metav1.ObjectMeta{Name: "test-role", Namespace: "test-ns"}, - } - res, err := NewBuilder(role). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&rbacv1.Role{ + ObjectMeta: metav1.ObjectMeta{Name: "role", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - role := &rbacv1.Role{ - ObjectMeta: metav1.ObjectMeta{Name: "test-role", Namespace: "test-ns"}, - } - res, err := NewBuilder(role). - WithDataExtractor(func(_ rbacv1.Role) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&rbacv1.Role{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/role/resource.go b/pkg/primitives/role/resource.go index 54bf7442..c82a9eeb 100644 --- a/pkg/primitives/role/resource.go +++ b/pkg/primitives/role/resource.go @@ -15,7 +15,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // Role resources are static: they do not model convergence health, grace periods, // or suspension. Use a workload or task primitive for resources that require those concepts. @@ -48,7 +48,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled Role. // // This is called by the framework after successful reconciliation, allowing the @@ -57,9 +57,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Role declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Role's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -95,3 +108,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/role/resource_test.go b/pkg/primitives/role/resource_test.go index 7c3795ab..642be0e3 100644 --- a/pkg/primitives/role/resource_test.go +++ b/pkg/primitives/role/resource_test.go @@ -1,7 +1,6 @@ package role import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -140,33 +139,3 @@ func TestResource_Mutate_FeatureOrdering(t *testing.T) { assert.Equal(t, []string{"secrets"}, got.Rules[0].Resources) assert.Equal(t, []string{"configmaps"}, got.Rules[1].Resources) } - -func TestResource_ExtractData(t *testing.T) { - role := newValidRole() - - var extracted []rbacv1.PolicyRule - res, err := NewBuilder(role). - WithDataExtractor(func(r rbacv1.Role) error { - extracted = r.Rules - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - require.Len(t, extracted, 1) - assert.Equal(t, []string{"pods"}, extracted[0].Resources) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidRole()). - WithDataExtractor(func(_ rbacv1.Role) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/rolebinding/builder.go b/pkg/primitives/rolebinding/builder.go index 1f045c98..1aeaacfe 100644 --- a/pkg/primitives/rolebinding/builder.go +++ b/pkg/primitives/rolebinding/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a RoleBinding Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -64,16 +64,22 @@ func (b *Builder) WithGuard(guard func(rbacv1.RoleBinding) (concepts.GuardStatus return b } -// WithDataExtractor registers a function to read values from the RoleBinding -// after it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled RoleBinding. This is -// useful for surfacing generated or updated entries to other components or -// resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(rbacv1.RoleBinding) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the RoleBinding reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the RoleBinding reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -96,3 +102,13 @@ func (b *Builder) Build() (*Resource, error) { return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this RoleBinding produces the value of cell. fn +// computes the value from a copy of the reconciled RoleBinding; the framework +// stores it in the cell and marks it present, immediately after the RoleBinding +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(rbacv1.RoleBinding) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/rolebinding/builder_test.go b/pkg/primitives/rolebinding/builder_test.go index 07e655ed..1964feed 100644 --- a/pkg/primitives/rolebinding/builder_test.go +++ b/pkg/primitives/rolebinding/builder_test.go @@ -1,9 +1,9 @@ package rolebinding import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" rbacv1 "k8s.io/api/rbac/v1" @@ -102,52 +102,56 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - rb := &rbacv1.RoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-rb", Namespace: "test-ns"}, + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&rbacv1.RoleBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "rb", Namespace: "default", Labels: map[string]string{"team": "platform"}}, RoleRef: testRoleRef(), - } - called := false - extractor := func(_ rbacv1.RoleBinding) error { - called = true - return nil - } - res, err := NewBuilder(rb). - WithDataExtractor(extractor). - Build() + }) + ExtractInto(builder, cell, func(o rbacv1.RoleBinding) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&rbacv1.RoleBinding{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - rb := &rbacv1.RoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-rb", Namespace: "test-ns"}, + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&rbacv1.RoleBinding{ + ObjectMeta: metav1.ObjectMeta{Name: "rb", Namespace: "default"}, RoleRef: testRoleRef(), - } - res, err := NewBuilder(rb). - WithDataExtractor(nil). - Build() + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - rb := &rbacv1.RoleBinding{ - ObjectMeta: metav1.ObjectMeta{Name: "test-rb", Namespace: "test-ns"}, - RoleRef: testRoleRef(), - } - res, err := NewBuilder(rb). - WithDataExtractor(func(_ rbacv1.RoleBinding) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&rbacv1.RoleBinding{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/rolebinding/resource.go b/pkg/primitives/rolebinding/resource.go index 3b436318..6cf2a7ce 100644 --- a/pkg/primitives/rolebinding/resource.go +++ b/pkg/primitives/rolebinding/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // RoleBinding resources are static: they do not model convergence health, // grace periods, or suspension. @@ -44,7 +44,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep +// ExtractData executes all declared data extractions against a deep // copy of the reconciled RoleBinding. // // This is called by the framework after successful reconciliation, allowing the @@ -53,9 +53,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this RoleBinding declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the RoleBinding's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -91,3 +104,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/rolebinding/resource_test.go b/pkg/primitives/rolebinding/resource_test.go index 68fd25a4..951e36f1 100644 --- a/pkg/primitives/rolebinding/resource_test.go +++ b/pkg/primitives/rolebinding/resource_test.go @@ -1,7 +1,6 @@ package rolebinding import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -96,32 +95,3 @@ func TestResource_Mutate_WithMutation(t *testing.T) { assert.Equal(t, "sa", got.Subjects[0].Name) assert.Equal(t, "from-mutation", got.Subjects[1].Name) } - -func TestResource_ExtractData(t *testing.T) { - rb := newValidRB() - - var extracted string - res, err := NewBuilder(rb). - WithDataExtractor(func(r rbacv1.RoleBinding) error { - extracted = r.Subjects[0].Name - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "sa", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidRB()). - WithDataExtractor(func(_ rbacv1.RoleBinding) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/secret/builder.go b/pkg/primitives/secret/builder.go index 1ee46e0b..e070e947 100644 --- a/pkg/primitives/secret/builder.go +++ b/pkg/primitives/secret/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a Secret Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -61,15 +61,22 @@ func (b *Builder) WithGuard(guard func(corev1.Secret) (concepts.GuardStatusWithR return b } -// WithDataExtractor registers a function to read values from the Secret after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled Secret. This is useful -// for surfacing generated or updated entries to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(corev1.Secret) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Secret reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Secret reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -85,3 +92,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Secret produces the value of cell. fn +// computes the value from a copy of the reconciled Secret; the framework +// stores it in the cell and marks it present, immediately after the Secret +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.Secret) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/secret/builder_test.go b/pkg/primitives/secret/builder_test.go index 3d8b7bc9..44745707 100644 --- a/pkg/primitives/secret/builder_test.go +++ b/pkg/primitives/secret/builder_test.go @@ -1,9 +1,9 @@ package secret import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" @@ -74,49 +74,55 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - s := &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{Name: "test-secret", Namespace: "test-ns"}, - } - called := false - extractor := func(_ corev1.Secret) error { - called = true - return nil - } - res, err := NewBuilder(s). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("db-host") + builder := NewBuilder(&corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: "s", Namespace: "default"}, + StringData: map[string]string{"db-host": "postgres.default.svc"}, + }) + ExtractInto(builder, cell, func(s corev1.Secret) (string, error) { + return s.StringData["db-host"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&corev1.Secret{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "db-host", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "postgres.default.svc", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - s := &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{Name: "test-secret", Namespace: "test-ns"}, - } - res, err := NewBuilder(s). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: "s", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - s := &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{Name: "test-secret", Namespace: "test-ns"}, - } - res, err := NewBuilder(s). - WithDataExtractor(func(_ corev1.Secret) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&corev1.Secret{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/secret/observation_test.go b/pkg/primitives/secret/observation_test.go index 202114e1..25db4e3a 100644 --- a/pkg/primitives/secret/observation_test.go +++ b/pkg/primitives/secret/observation_test.go @@ -20,18 +20,18 @@ import ( // check guards against silent regressions of issue #118. var _ concepts.ObservationRecorder = (*Resource)(nil) -// TestReadOnlyExtractor_ObservesClusterState reproduces the user-reported +// TestReadOnlyExtraction_ObservesClusterState reproduces the user-reported // scenario from issue #115 / #118 end-to-end against a real *Resource: build // the Resource via the public builder, fetch it through a fake client, hand // the fetched object to the framework's RecordObservation hook, run -// ExtractData, and assert that the registered extractor sees the live cluster +// ExtractData, and assert that the declared extraction sees the live cluster // data rather than the inert base used to construct the resource. // // The framework-side wiring is covered by pkg/component/read_test.go using // mock resources; this test exercises the same chain through a real primitive, // so a regression in either the wrapper's forwarding or the BaseResource // implementation is caught here. -func TestReadOnlyExtractor_ObservesClusterState(t *testing.T) { +func TestReadOnlyExtraction_ObservesClusterState(t *testing.T) { ctx := t.Context() scheme := runtime.NewScheme() @@ -56,13 +56,12 @@ func TestReadOnlyExtractor_ObservesClusterState(t *testing.T) { }, } - var captured []byte - res, err := NewBuilder(base). - WithDataExtractor(func(s corev1.Secret) error { - captured = s.Data["token"] - return nil - }). - Build() + cell := concepts.NewData[[]byte]("token") + builder := NewBuilder(base) + ExtractInto(builder, cell, func(s corev1.Secret) ([]byte, error) { + return s.Data["token"], nil + }) + res, err := builder.Build() require.NoError(t, err) // Simulate the framework's read flow: deep-copy the desired base, fetch @@ -74,6 +73,8 @@ func TestReadOnlyExtractor_ObservesClusterState(t *testing.T) { require.NoError(t, res.RecordObservation(fetched)) require.NoError(t, res.ExtractData()) + captured, ok := cell.Get() + require.True(t, ok) assert.Equal(t, []byte("from-cluster"), captured, - "the extractor must see the cluster's Secret data, not the empty base") + "the declared extraction must see the cluster's Secret data, not the empty base") } diff --git a/pkg/primitives/secret/resource.go b/pkg/primitives/secret/resource.go index 3376c70e..2b4ade05 100644 --- a/pkg/primitives/secret/resource.go +++ b/pkg/primitives/secret/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // Secret resources are static: they do not model convergence health, grace periods, // or suspension. Use a workload or task primitive for resources that require those concepts. @@ -47,7 +47,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled Secret. // // This is called by the framework after successful reconciliation, allowing the @@ -56,9 +56,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Secret declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Secret's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only Secrets after -// fetching them so that registered data extractors observe the live Secret rather +// fetching them so that declared data extractions observe the live Secret rather // than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -94,3 +107,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/secret/resource_test.go b/pkg/primitives/secret/resource_test.go index ef95e099..cac331f3 100644 --- a/pkg/primitives/secret/resource_test.go +++ b/pkg/primitives/secret/resource_test.go @@ -1,7 +1,6 @@ package secret import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -111,32 +110,3 @@ func TestResource_Mutate_FeatureOrdering(t *testing.T) { got := obj.(*corev1.Secret) assert.Equal(t, []byte("b"), got.Data["order"]) } - -func TestResource_ExtractData(t *testing.T) { - s := newValidSecret() - - var extracted []byte - res, err := NewBuilder(s). - WithDataExtractor(func(c corev1.Secret) error { - extracted = c.Data["key"] - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, []byte("value"), extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidSecret()). - WithDataExtractor(func(_ corev1.Secret) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/service/builder.go b/pkg/primitives/service/builder.go index bed2a335..5b779895 100644 --- a/pkg/primitives/service/builder.go +++ b/pkg/primitives/service/builder.go @@ -12,7 +12,7 @@ import ( // Builder is a configuration helper for creating and customizing a Service Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.IntegrationBuilder[*corev1.Service, *Mutator] @@ -152,16 +152,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// Service after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields (like assigned ClusterIP -// or LoadBalancer ingress) and making them available to other components or -// resources via the framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(corev1.Service) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the Service reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the Service reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -179,3 +185,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this Service produces the value of cell. fn +// computes the value from a copy of the reconciled Service; the framework +// stores it in the cell and marks it present, immediately after the Service +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.Service) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/service/builder_test.go b/pkg/primitives/service/builder_test.go index 641d436d..1ee98284 100644 --- a/pkg/primitives/service/builder_test.go +++ b/pkg/primitives/service/builder_test.go @@ -172,61 +172,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.False(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - svc := &corev1.Service{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-svc", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ corev1.Service) error { - called = true - return nil - } - res, err := NewBuilder(svc). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&corev1.Service{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Name: "svc", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - svc := &corev1.Service{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-svc", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(svc). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o corev1.Service) (string, error) { + return o.Labels["team"], nil }) - t.Run("WithDataExtractor error propagated", func(t *testing.T) { - t.Parallel() - svc := &corev1.Service{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-svc", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(svc). - WithDataExtractor(func(_ corev1.Service) error { - return errors.New("extractor error") - }). - Build() - require.NoError(t, err) - err = res.base.DataExtractors[0](&corev1.Service{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") - }) + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Name: "svc", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/service/resource.go b/pkg/primitives/service/resource.go index 95fbdd98..92031251 100644 --- a/pkg/primitives/service/resource.go +++ b/pkg/primitives/service/resource.go @@ -57,7 +57,7 @@ func normalizeProtocol(p corev1.Protocol) corev1.Protocol { // - concepts.Suspendable: for participating in the component suspension lifecycle. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. type Resource struct { base *generic.IntegrationResource[*corev1.Service, *Mutator] } @@ -128,7 +128,7 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled Service. // // This is called by the framework after successful reconciliation, allowing the @@ -138,9 +138,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this Service declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the Service's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -176,3 +189,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/service/resource_test.go b/pkg/primitives/service/resource_test.go index 6681a56a..fb0809ff 100644 --- a/pkg/primitives/service/resource_test.go +++ b/pkg/primitives/service/resource_test.go @@ -1,7 +1,6 @@ package service import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" @@ -168,32 +167,3 @@ func TestResource_SuspensionStatus(t *testing.T) { require.NoError(t, err) assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) } - -func TestResource_ExtractData(t *testing.T) { - svc := newValidService() - - var extracted string - res, err := NewBuilder(svc). - WithDataExtractor(func(s corev1.Service) error { - extracted = s.Spec.Selector["app"] - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "test", extracted) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidService()). - WithDataExtractor(func(_ corev1.Service) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/serviceaccount/builder.go b/pkg/primitives/serviceaccount/builder.go index 05107290..b1e7d0e5 100644 --- a/pkg/primitives/serviceaccount/builder.go +++ b/pkg/primitives/serviceaccount/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a ServiceAccount Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -61,15 +61,22 @@ func (b *Builder) WithGuard(guard func(corev1.ServiceAccount) (concepts.GuardSta return b } -// WithDataExtractor registers a function to read values from the ServiceAccount after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled ServiceAccount. This is useful -// for surfacing generated or updated entries to other components or resources. -// -// A nil extractor is ignored. -func (b *Builder) WithDataExtractor(extractor func(corev1.ServiceAccount) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the ServiceAccount reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the ServiceAccount reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -85,3 +92,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this ServiceAccount produces the value of cell. fn +// computes the value from a copy of the reconciled ServiceAccount; the framework +// stores it in the cell and marks it present, immediately after the ServiceAccount +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(corev1.ServiceAccount) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/serviceaccount/builder_test.go b/pkg/primitives/serviceaccount/builder_test.go index 5d295960..9c1bbb77 100644 --- a/pkg/primitives/serviceaccount/builder_test.go +++ b/pkg/primitives/serviceaccount/builder_test.go @@ -1,9 +1,9 @@ package serviceaccount import ( - "errors" "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" @@ -74,49 +74,54 @@ func TestBuilder_WithMutation(t *testing.T) { assert.Equal(t, "test-mutation", res.base.Mutations[0].Name) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { t.Parallel() - sa := &corev1.ServiceAccount{ - ObjectMeta: metav1.ObjectMeta{Name: "test-sa", Namespace: "test-ns"}, - } - called := false - extractor := func(_ corev1.ServiceAccount) error { - called = true - return nil - } - res, err := NewBuilder(sa). - WithDataExtractor(extractor). - Build() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&corev1.ServiceAccount{ + ObjectMeta: metav1.ObjectMeta{Name: "sa", Namespace: "default", Labels: map[string]string{"team": "platform"}}, + }) + ExtractInto(builder, cell, func(o corev1.ServiceAccount) (string, error) { + return o.Labels["team"], nil + }) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - require.NoError(t, res.base.DataExtractors[0](&corev1.ServiceAccount{})) - assert.True(t, called) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_Nil(t *testing.T) { +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { t.Parallel() - sa := &corev1.ServiceAccount{ - ObjectMeta: metav1.ObjectMeta{Name: "test-sa", Namespace: "test-ns"}, - } - res, err := NewBuilder(sa). - WithDataExtractor(nil). - Build() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&corev1.ServiceAccount{ + ObjectMeta: metav1.ObjectMeta{Name: "sa", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) -} -func TestBuilder_WithDataExtractor_ErrorPropagated(t *testing.T) { - t.Parallel() - sa := &corev1.ServiceAccount{ - ObjectMeta: metav1.ObjectMeta{Name: "test-sa", Namespace: "test-ns"}, - } - res, err := NewBuilder(sa). - WithDataExtractor(func(_ corev1.ServiceAccount) error { - return errors.New("extractor error") - }). - Build() + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() require.NoError(t, err) - err = res.base.DataExtractors[0](&corev1.ServiceAccount{}) - require.Error(t, err) - assert.Contains(t, err.Error(), "extractor error") + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/serviceaccount/resource.go b/pkg/primitives/serviceaccount/resource.go index 60f529a2..f587d7a6 100644 --- a/pkg/primitives/serviceaccount/resource.go +++ b/pkg/primitives/serviceaccount/resource.go @@ -14,7 +14,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // ServiceAccount resources are static: they do not model convergence health, grace periods, // or suspension. Use a workload or task primitive for resources that require those concepts. @@ -47,7 +47,7 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep copy +// ExtractData executes all declared data extractions against a deep copy // of the reconciled ServiceAccount. // // This is called by the framework after successful reconciliation, allowing the @@ -56,9 +56,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this ServiceAccount declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the ServiceAccount's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -94,3 +107,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/serviceaccount/resource_test.go b/pkg/primitives/serviceaccount/resource_test.go index eb30cb70..3e6da59e 100644 --- a/pkg/primitives/serviceaccount/resource_test.go +++ b/pkg/primitives/serviceaccount/resource_test.go @@ -1,7 +1,6 @@ package serviceaccount import ( - "errors" "testing" "github.com/sourcehawk/operator-component-framework/pkg/feature" @@ -97,33 +96,3 @@ func TestResource_Mutate_WithMutation(t *testing.T) { } // --- Resource.ExtractData tests --- - -func TestResource_ExtractData(t *testing.T) { - sa := newValidSA() - sa.ImagePullSecrets = []corev1.LocalObjectReference{{Name: "reg"}} - - var extractedName string - res, err := NewBuilder(sa). - WithDataExtractor(func(s corev1.ServiceAccount) error { - extractedName = s.ImagePullSecrets[0].Name - return nil - }). - Build() - require.NoError(t, err) - - require.NoError(t, res.ExtractData()) - assert.Equal(t, "reg", extractedName) -} - -func TestResource_ExtractData_Error(t *testing.T) { - res, err := NewBuilder(newValidSA()). - WithDataExtractor(func(_ corev1.ServiceAccount) error { - return errors.New("extract error") - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.Error(t, err) - assert.Contains(t, err.Error(), "extract error") -} diff --git a/pkg/primitives/statefulset/builder.go b/pkg/primitives/statefulset/builder.go index 060de024..5e14dc61 100644 --- a/pkg/primitives/statefulset/builder.go +++ b/pkg/primitives/statefulset/builder.go @@ -13,7 +13,7 @@ import ( // Builder is a configuration helper for creating and customizing a StatefulSet Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. This builder ensures that the resulting Resource is +// declared data extractions. This builder ensures that the resulting Resource is // properly initialized and validated before use in a reconciliation loop. type Builder struct { base *generic.WorkloadBuilder[*appsv1.StatefulSet, *Mutator] @@ -131,15 +131,22 @@ func (b *Builder) WithGuard( return b } -// WithDataExtractor registers a function to harvest information from the -// StatefulSet after it has been successfully reconciled. -// -// This is useful for capturing auto-generated fields and making them available -// to other components or resources via the framework's data extraction mechanism. -func (b *Builder) WithDataExtractor( - extractor func(appsv1.StatefulSet) error, -) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the StatefulSet reads the given data cells and +// must not be applied until every one of them is set. The framework generates +// the guard and its reason (waiting for data ""), and component Build +// validates that a producer for each cell is registered earlier. Data guards +// are evaluated before any custom guard registered with WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the StatefulSet reads the given data cells +// without gating on them. Component Build still validates that a producer is +// registered earlier, and the dependency stays visible to introspection. +// Consumers in this mode use Get and skip quietly when a cell is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -157,3 +164,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this StatefulSet produces the value of cell. fn +// computes the value from a copy of the reconciled StatefulSet; the framework +// stores it in the cell and marks it present, immediately after the StatefulSet +// is applied or fetched. Extracting several values means several ExtractInto +// calls, one per cell. This is a package-level function because Go methods +// cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(appsv1.StatefulSet) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/statefulset/builder_test.go b/pkg/primitives/statefulset/builder_test.go index d4172ca2..aae22bbd 100644 --- a/pkg/primitives/statefulset/builder_test.go +++ b/pkg/primitives/statefulset/builder_test.go @@ -239,42 +239,56 @@ func TestBuilder(t *testing.T) { require.NotNil(t, res.base.DeleteOnSuspendHandler) assert.True(t, res.base.DeleteOnSuspendHandler(nil)) }) +} - t.Run("WithDataExtractor", func(t *testing.T) { - t.Parallel() - sts := &appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-sts", - Namespace: "test-ns", - }, - } - called := false - extractor := func(_ appsv1.StatefulSet) error { - called = true - return nil - } - res, err := NewBuilder(sts). - WithDataExtractor(extractor). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 1) - err = res.base.DataExtractors[0](&appsv1.StatefulSet{}) - require.NoError(t, err) - assert.True(t, called) +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + builder := NewBuilder(&appsv1.StatefulSet{ + ObjectMeta: metav1.ObjectMeta{Name: "sts", Namespace: "default", Labels: map[string]string{"team": "platform"}}, }) - - t.Run("WithDataExtractor nil", func(t *testing.T) { - t.Parallel() - sts := &appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-sts", - Namespace: "test-ns", - }, - } - res, err := NewBuilder(sts). - WithDataExtractor(nil). - Build() - require.NoError(t, err) - assert.Len(t, res.base.DataExtractors, 0) + ExtractInto(builder, cell, func(o appsv1.StatefulSet) (string, error) { + return o.Labels["team"], nil }) + + res, err := builder.Build() + require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + + require.NoError(t, res.ExtractData()) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(&appsv1.StatefulSet{ + ObjectMeta: metav1.ObjectMeta{Name: "sts", Namespace: "default"}, + }).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/statefulset/resource.go b/pkg/primitives/statefulset/resource.go index a089ae04..3fdf0d4d 100644 --- a/pkg/primitives/statefulset/resource.go +++ b/pkg/primitives/statefulset/resource.go @@ -16,7 +16,7 @@ import ( // - concepts.Suspendable: for graceful scale-down or temporary deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting information after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // This resource handles the lifecycle of a StatefulSet, including initial creation, // updates via feature mutations, and status monitoring. @@ -121,9 +121,22 @@ func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this StatefulSet declares extractions into. +// It satisfies concepts.DataProducer for component topology validation and +// introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the StatefulSet's declared data reads. It satisfies +// concepts.DataConsumer for component topology validation and introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -160,3 +173,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/statefulset/resource_test.go b/pkg/primitives/statefulset/resource_test.go index 404a9b36..b8b56d2a 100644 --- a/pkg/primitives/statefulset/resource_test.go +++ b/pkg/primitives/statefulset/resource_test.go @@ -345,29 +345,3 @@ func TestResource_SuspensionStatus(t *testing.T) { assert.Equal(t, concepts.SuspensionStatusSuspended, status.Status) }) } - -func TestResource_ExtractData(t *testing.T) { - sts := &appsv1.StatefulSet{ - ObjectMeta: metav1.ObjectMeta{Name: "test", Namespace: "default"}, - Spec: appsv1.StatefulSetSpec{ - Template: corev1.PodTemplateSpec{ - Spec: corev1.PodSpec{ - Containers: []corev1.Container{{Name: "web", Image: "nginx:latest"}}, - }, - }, - }, - } - - extractedImage := "" - res, err := NewBuilder(sts). - WithDataExtractor(func(s appsv1.StatefulSet) error { - extractedImage = s.Spec.Template.Spec.Containers[0].Image - return nil - }). - Build() - require.NoError(t, err) - - err = res.ExtractData() - require.NoError(t, err) - assert.Equal(t, "nginx:latest", extractedImage) -} diff --git a/pkg/primitives/unstructured/integration/builder.go b/pkg/primitives/unstructured/integration/builder.go index b8fe3ab5..634ec089 100644 --- a/pkg/primitives/unstructured/integration/builder.go +++ b/pkg/primitives/unstructured/integration/builder.go @@ -12,7 +12,7 @@ import ( // integration Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. The operational status handler is required; all other +// declared data extractions. The operational status handler is required; all other // handlers default to safe no-ops when omitted. type Builder struct { base *generic.IntegrationBuilder[*uns.Unstructured, *unstruct.Mutator] @@ -105,10 +105,24 @@ func (b *Builder) WithGuard(guard func(uns.Unstructured) (concepts.GuardStatusWi return b } -// WithDataExtractor registers a function to read values from the object after -// it has been successfully reconciled. -func (b *Builder) WithDataExtractor(extractor func(uns.Unstructured) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the unstructured object reads the given data +// cells and must not be applied until every one of them is set. The framework +// generates the guard and its reason (waiting for data ""), and +// component Build validates that a producer for each cell is registered +// earlier. Data guards are evaluated before any custom guard registered with +// WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the unstructured object reads the given data +// cells without gating on them. Component Build still validates that a +// producer is registered earlier, and the dependency stays visible to +// introspection. Consumers in this mode use Get and skip quietly when a cell +// is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -123,3 +137,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this unstructured object produces the value of +// cell. fn computes the value from a copy of the reconciled object; the +// framework stores it in the cell and marks it present, immediately after the +// object is applied or fetched. Extracting several values means several +// ExtractInto calls, one per cell. This is a package-level function because Go +// methods cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(uns.Unstructured) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/unstructured/integration/builder_test.go b/pkg/primitives/unstructured/integration/builder_test.go index 02f4c71a..b39ccb89 100644 --- a/pkg/primitives/unstructured/integration/builder_test.go +++ b/pkg/primitives/unstructured/integration/builder_test.go @@ -62,15 +62,52 @@ func TestBuilder_Identity_Namespaced(t *testing.T) { assert.Equal(t, "example.com/v1/Gateway/default/test", res.Identity()) } -func TestBuilder_WithDataExtractor(t *testing.T) { - called := false - b := withRequiredHandlers(NewBuilder(validObject())) - b.WithDataExtractor(func(_ uns.Unstructured) error { - called = true - return nil +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + obj := validObject() + obj.SetLabels(map[string]string{"team": "platform"}) + builder := withRequiredHandlers(NewBuilder(obj)) + ExtractInto(builder, cell, func(o uns.Unstructured) (string, error) { + return o.GetLabels()["team"], nil }) - res, err := b.Build() + + res, err := builder.Build() require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + require.NoError(t, res.ExtractData()) - assert.True(t, called) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := withRequiredHandlers(NewBuilder(validObject())).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/unstructured/integration/resource.go b/pkg/primitives/unstructured/integration/resource.go index 389e2bb3..1dbc39b6 100644 --- a/pkg/primitives/unstructured/integration/resource.go +++ b/pkg/primitives/unstructured/integration/resource.go @@ -21,7 +21,7 @@ import ( // - concepts.Suspendable: for graceful deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // The operational status handler is required. All other handlers default to // safe no-ops when omitted. @@ -75,15 +75,29 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep +// ExtractData executes all declared data extractions against a deep // copy of the reconciled object. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this unstructured object declares +// extractions into. It satisfies concepts.DataProducer for component +// topology validation and introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the unstructured object's declared data reads. It +// satisfies concepts.DataConsumer for component topology validation and +// introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -119,3 +133,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/unstructured/static/builder.go b/pkg/primitives/unstructured/static/builder.go index 8a5a3ef3..8fbc8531 100644 --- a/pkg/primitives/unstructured/static/builder.go +++ b/pkg/primitives/unstructured/static/builder.go @@ -11,7 +11,7 @@ import ( // Builder is a configuration helper for creating and customizing a static // unstructured Resource. // -// It provides a fluent API for registering mutations and data extractors. +// It provides a fluent API for registering mutations and declared data extractions. // Build() validates the configuration and returns an initialized Resource // ready for use in a reconciliation loop. type Builder struct { @@ -69,13 +69,24 @@ func (b *Builder) WithGuard(guard func(uns.Unstructured) (concepts.GuardStatusWi return b } -// WithDataExtractor registers a function to read values from the object after -// it has been successfully reconciled. -// -// The extractor receives a value copy of the reconciled object. A nil extractor -// is ignored. -func (b *Builder) WithDataExtractor(extractor func(uns.Unstructured) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the unstructured object reads the given data +// cells and must not be applied until every one of them is set. The framework +// generates the guard and its reason (waiting for data ""), and +// component Build validates that a producer for each cell is registered +// earlier. Data guards are evaluated before any custom guard registered with +// WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the unstructured object reads the given data +// cells without gating on them. Component Build still validates that a +// producer is registered earlier, and the dependency stays visible to +// introspection. Consumers in this mode use Get and skip quietly when a cell +// is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -94,3 +105,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this unstructured object produces the value of +// cell. fn computes the value from a copy of the reconciled object; the +// framework stores it in the cell and marks it present, immediately after the +// object is applied or fetched. Extracting several values means several +// ExtractInto calls, one per cell. This is a package-level function because Go +// methods cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(uns.Unstructured) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/unstructured/static/builder_test.go b/pkg/primitives/unstructured/static/builder_test.go index 51827630..2ab4fee3 100644 --- a/pkg/primitives/unstructured/static/builder_test.go +++ b/pkg/primitives/unstructured/static/builder_test.go @@ -3,6 +3,7 @@ package static import ( "testing" + "github.com/sourcehawk/operator-component-framework/pkg/component/concepts" unstruct "github.com/sourcehawk/operator-component-framework/pkg/primitives/unstructured" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -95,24 +96,52 @@ func TestBuilder_WithMutation(t *testing.T) { require.NotNil(t, res) } -func TestBuilder_WithDataExtractor(t *testing.T) { +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") obj := validObject() - called := false - b := NewBuilder(obj) - b.WithDataExtractor(func(_ uns.Unstructured) error { - called = true - return nil + obj.SetLabels(map[string]string{"team": "platform"}) + builder := NewBuilder(obj) + ExtractInto(builder, cell, func(o uns.Unstructured) (string, error) { + return o.GetLabels()["team"], nil }) - res, err := b.Build() + + res, err := builder.Build() require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + require.NoError(t, res.ExtractData()) - assert.True(t, called) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) } -func TestBuilder_WithDataExtractor_NilIgnored(t *testing.T) { - b := NewBuilder(validObject()) - b.WithDataExtractor(nil) - res, err := b.Build() +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := NewBuilder(validObject()).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() require.NoError(t, err) - require.NoError(t, res.ExtractData()) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/unstructured/static/resource.go b/pkg/primitives/unstructured/static/resource.go index bbc5fe32..34ec6d13 100644 --- a/pkg/primitives/unstructured/static/resource.go +++ b/pkg/primitives/unstructured/static/resource.go @@ -18,7 +18,7 @@ import ( // - component.Resource: for basic identity and mutation behaviour. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // Static unstructured resources do not model convergence health, grace periods, // or suspension. Use the workload, integration, or task unstructured variants @@ -44,15 +44,29 @@ func (r *Resource) Mutate(current client.Object) error { return r.base.Mutate(current) } -// ExtractData executes all registered data extractor functions against a deep +// ExtractData executes all declared data extractions against a deep // copy of the reconciled object. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this unstructured object declares +// extractions into. It satisfies concepts.DataProducer for component +// topology validation and introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the unstructured object's declared data reads. It +// satisfies concepts.DataConsumer for component topology validation and +// introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -88,3 +102,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/unstructured/task/builder.go b/pkg/primitives/unstructured/task/builder.go index 3336901a..7b7e1928 100644 --- a/pkg/primitives/unstructured/task/builder.go +++ b/pkg/primitives/unstructured/task/builder.go @@ -12,7 +12,7 @@ import ( // task Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. The converging status handler is required; all other +// declared data extractions. The converging status handler is required; all other // handlers default to safe no-ops when omitted. type Builder struct { base *generic.TaskBuilder[*uns.Unstructured, *unstruct.Mutator] @@ -96,10 +96,24 @@ func (b *Builder) WithGuard(guard func(uns.Unstructured) (concepts.GuardStatusWi return b } -// WithDataExtractor registers a function to read values from the object after -// it has been successfully reconciled. -func (b *Builder) WithDataExtractor(extractor func(uns.Unstructured) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the unstructured object reads the given data +// cells and must not be applied until every one of them is set. The framework +// generates the guard and its reason (waiting for data ""), and +// component Build validates that a producer for each cell is registered +// earlier. Data guards are evaluated before any custom guard registered with +// WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the unstructured object reads the given data +// cells without gating on them. Component Build still validates that a +// producer is registered earlier, and the dependency stays visible to +// introspection. Consumers in this mode use Get and skip quietly when a cell +// is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -114,3 +128,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this unstructured object produces the value of +// cell. fn computes the value from a copy of the reconciled object; the +// framework stores it in the cell and marks it present, immediately after the +// object is applied or fetched. Extracting several values means several +// ExtractInto calls, one per cell. This is a package-level function because Go +// methods cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(uns.Unstructured) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/unstructured/task/builder_test.go b/pkg/primitives/unstructured/task/builder_test.go index 03a2cd53..ef41f271 100644 --- a/pkg/primitives/unstructured/task/builder_test.go +++ b/pkg/primitives/unstructured/task/builder_test.go @@ -62,15 +62,52 @@ func TestBuilder_Identity_Namespaced(t *testing.T) { assert.Equal(t, "example.com/v1/BatchJob/default/test", res.Identity()) } -func TestBuilder_WithDataExtractor(t *testing.T) { - called := false - b := withRequiredHandlers(NewBuilder(validObject())) - b.WithDataExtractor(func(_ uns.Unstructured) error { - called = true - return nil +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + obj := validObject() + obj.SetLabels(map[string]string{"team": "platform"}) + builder := withRequiredHandlers(NewBuilder(obj)) + ExtractInto(builder, cell, func(o uns.Unstructured) (string, error) { + return o.GetLabels()["team"], nil }) - res, err := b.Build() + + res, err := builder.Build() require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + require.NoError(t, res.ExtractData()) - assert.True(t, called) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := withRequiredHandlers(NewBuilder(validObject())).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/unstructured/task/resource.go b/pkg/primitives/unstructured/task/resource.go index 1ff8e8fa..bddf019c 100644 --- a/pkg/primitives/unstructured/task/resource.go +++ b/pkg/primitives/unstructured/task/resource.go @@ -19,7 +19,7 @@ import ( // - concepts.Suspendable: for graceful deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // The converging status handler is required; all other handlers default to // safe no-ops when omitted. @@ -67,15 +67,29 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep +// ExtractData executes all declared data extractions against a deep // copy of the reconciled object. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this unstructured object declares +// extractions into. It satisfies concepts.DataProducer for component +// topology validation and introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the unstructured object's declared data reads. It +// satisfies concepts.DataConsumer for component topology validation and +// introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -111,3 +125,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil) diff --git a/pkg/primitives/unstructured/workload/builder.go b/pkg/primitives/unstructured/workload/builder.go index 6bc4fc65..63f3b6d6 100644 --- a/pkg/primitives/unstructured/workload/builder.go +++ b/pkg/primitives/unstructured/workload/builder.go @@ -12,7 +12,7 @@ import ( // workload Resource. // // It provides a fluent API for registering mutations, status handlers, and -// data extractors. The converging status handler is required; all other +// declared data extractions. The converging status handler is required; all other // handlers default to safe no-ops when omitted. type Builder struct { base *generic.WorkloadBuilder[*uns.Unstructured, *unstruct.Mutator] @@ -105,10 +105,24 @@ func (b *Builder) WithGuard(guard func(uns.Unstructured) (concepts.GuardStatusWi return b } -// WithDataExtractor registers a function to read values from the object after -// it has been successfully reconciled. -func (b *Builder) WithDataExtractor(extractor func(uns.Unstructured) error) *Builder { - b.base.WithDataExtractor(generic.WrapExtractor(extractor)) +// WithDataGuard declares that the unstructured object reads the given data +// cells and must not be applied until every one of them is set. The framework +// generates the guard and its reason (waiting for data ""), and +// component Build validates that a producer for each cell is registered +// earlier. Data guards are evaluated before any custom guard registered with +// WithGuard. +func (b *Builder) WithDataGuard(cells ...concepts.DataCell) *Builder { + b.base.WithDataGuard(cells...) + return b +} + +// WithOptionalData declares that the unstructured object reads the given data +// cells without gating on them. Component Build still validates that a +// producer is registered earlier, and the dependency stays visible to +// introspection. Consumers in this mode use Get and skip quietly when a cell +// is absent. +func (b *Builder) WithOptionalData(cells ...concepts.DataCell) *Builder { + b.base.WithOptionalData(cells...) return b } @@ -123,3 +137,13 @@ func (b *Builder) Build() (*Resource, error) { } return &Resource{base: genericRes}, nil } + +// ExtractInto declares that this unstructured object produces the value of +// cell. fn computes the value from a copy of the reconciled object; the +// framework stores it in the cell and marks it present, immediately after the +// object is applied or fetched. Extracting several values means several +// ExtractInto calls, one per cell. This is a package-level function because Go +// methods cannot introduce the extra type parameter V. +func ExtractInto[V any](b *Builder, cell *concepts.Data[V], fn func(uns.Unstructured) (V, error)) { + generic.ExtractInto(&b.base.BaseBuilder, cell, generic.WrapExtraction(fn)) +} diff --git a/pkg/primitives/unstructured/workload/builder_test.go b/pkg/primitives/unstructured/workload/builder_test.go index e91ccfdf..6de51d29 100644 --- a/pkg/primitives/unstructured/workload/builder_test.go +++ b/pkg/primitives/unstructured/workload/builder_test.go @@ -101,15 +101,52 @@ func TestBuilder_Identity_Namespaced(t *testing.T) { assert.Equal(t, "example.com/v1/Worker/default/test", res.Identity()) } -func TestBuilder_WithDataExtractor(t *testing.T) { - called := false - b := withRequiredHandlers(NewBuilder(validObject())) - b.WithDataExtractor(func(_ uns.Unstructured) error { - called = true - return nil +func TestExtractIntoDeclaredExtraction(t *testing.T) { + t.Parallel() + cell := concepts.NewData[string]("team-label") + obj := validObject() + obj.SetLabels(map[string]string{"team": "platform"}) + builder := withRequiredHandlers(NewBuilder(obj)) + ExtractInto(builder, cell, func(o uns.Unstructured) (string, error) { + return o.GetLabels()["team"], nil }) - res, err := b.Build() + + res, err := builder.Build() require.NoError(t, err) + + produced := res.ProducedData() + require.Len(t, produced, 1) + assert.Equal(t, "team-label", produced[0].Name()) + require.NoError(t, res.ExtractData()) - assert.True(t, called) + v, ok := cell.Get() + assert.True(t, ok) + assert.Equal(t, "platform", v) +} + +func TestWithDataGuardAndOptionalDataDeclarations(t *testing.T) { + t.Parallel() + guarded := concepts.NewData[string]("db-host") + optional := concepts.NewData[string]("db-port") + builder := withRequiredHandlers(NewBuilder(validObject())).WithDataGuard(guarded).WithOptionalData(optional) + + res, err := builder.Build() + require.NoError(t, err) + + consumed := res.ConsumedData() + require.Len(t, consumed, 2) + assert.Equal(t, "db-host", consumed[0].Cell.Name()) + assert.False(t, consumed[0].Optional) + assert.Equal(t, "db-port", consumed[1].Cell.Name()) + assert.True(t, consumed[1].Optional) + + status, err := res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusBlocked, status.Status) + assert.Equal(t, `waiting for data "db-host"`, status.Reason) + + guarded.Set("postgres.default.svc") + status, err = res.GuardStatus() + require.NoError(t, err) + assert.Equal(t, concepts.GuardStatusUnblocked, status.Status) } diff --git a/pkg/primitives/unstructured/workload/resource.go b/pkg/primitives/unstructured/workload/resource.go index 04adb9d1..5e8a5877 100644 --- a/pkg/primitives/unstructured/workload/resource.go +++ b/pkg/primitives/unstructured/workload/resource.go @@ -21,7 +21,7 @@ import ( // - concepts.Suspendable: for graceful scale-down or temporary deactivation. // - concepts.Guardable: for conditional reconciliation based on a guard precondition. // - concepts.DataExtractable: for exporting values after successful reconciliation. -// - concepts.ObservationRecorder: for surfacing live cluster state to data extractors on read-only reconciliation. +// - concepts.ObservationRecorder: for surfacing live cluster state to declared data extractions on read-only reconciliation. // // The converging status handler is required; all other handlers default to // safe no-ops when omitted. @@ -75,15 +75,29 @@ func (r *Resource) SuspensionStatus() (concepts.SuspensionStatusWithReason, erro return r.base.SuspensionStatus() } -// ExtractData executes all registered data extractor functions against a deep +// ExtractData executes all declared data extractions against a deep // copy of the reconciled object. func (r *Resource) ExtractData() error { return r.base.ExtractData() } +// ProducedData returns the cells this unstructured object declares +// extractions into. It satisfies concepts.DataProducer for component +// topology validation and introspection. +func (r *Resource) ProducedData() []concepts.DataCell { + return r.base.ProducedData() +} + +// ConsumedData returns the unstructured object's declared data reads. It +// satisfies concepts.DataConsumer for component topology validation and +// introspection. +func (r *Resource) ConsumedData() []concepts.DataConsumption { + return r.base.ConsumedData() +} + // RecordObservation stores the supplied object as the resource's most recently // observed cluster state. The framework invokes this on read-only resources -// after fetching them so that registered data extractors observe the live +// after fetching them so that declared data extractions observe the live // object rather than the inert base used to construct the resource. func (r *Resource) RecordObservation(observed client.Object) error { return r.base.RecordObservation(observed) @@ -119,3 +133,5 @@ func (r *Resource) FiringSet() ([]string, error) { } var _ concepts.MutationInspector = (*Resource)(nil) +var _ concepts.DataProducer = (*Resource)(nil) +var _ concepts.DataConsumer = (*Resource)(nil)