Assemble a fabric from input objects, and migrate them by asking Ansible

[mbakalarski]

A Fabric gains four kinds of input object and an ordered list naming them, and
a migration that turns an AVD Ansible inventory into exactly that — by asking
Ansible what the inventory means rather than reproducing it.

Each input carries a fragment of the eos_designs document plus the devices that
see it; per device, the inputs that apply are layered in order with
dict.update() — Ansible's default hash_behaviour=replace.

Why nothing is merged

Two NodeSets carrying the same node-type key never meet, because no device
sees both: in a dual-DC fabric each leaf sees its own l3leaf.defaults. That is
what pyavd.get_avd_facts already expects to be handed. So a schema-aware
merge, a duplicate-key conflict rule and a defaults push-down are all avoided,
and with them any dependency on pyavd's private API.

The API

kind	carries
NodeSet	node-type blocks, type, node_type_keys — and the devices it declares
NetworkServiceSet	the tenants (network_services_keys.name) and the native network_services spelling
ConnectedEndpointSet	connected_endpoints_keys.key lists, port_profiles, network_ports
SettingSet	everything else, narrowed to part of the fabric

The kinds separate ownership, not content: RBAC is granted per kind, and
eos_designs' top-level key names come from its own content, so no schema could
partition them anyway.

Which kind a key belongs to is read from pyavd's public schema, not from a
literal — the literal it replaced was already short, missing cameras from
connected_endpoints_keys. What stays a literal is the handful of keys that are
not settings, and a test holds it against documentation_options.table in AVD's
own schema in both directions.

Only NodeSet declares devices. spec.declares is the fabric's device list
and there is no second one. Never a pattern: visibility may be matched,
existence may not, since a typo would silently drop devices.

appliesTo takes all / nodeSets / hosts / matchHostnames, and
saying nothing is the common case — the whole fabric, except on a NodeSet,
where it means what that NodeSet declares. The pattern form copies AVD's own
hostname matching from the code rather than the description:
shared_utils/node_type.py anchors the pattern for you, while the schema's
wording reads as though the author must.

Secret is in the requires enum from this first version, unimplemented.
The mechanism needs no other schema footprint, and adding the enum value after
release would be a schema change to a published API. Credentials are not
hypothetical: the bundled examples carry sha512_password and type-7 BGP
passwords, and type 7 is reversible — pyavd ships bgp_decrypt.

All six XRDs gain categories: [crossplane, netclab]. Verified on a live
cluster: kubectl api-resources --categories=netclab lists the set. It only
shows on a fresh install — Crossplane's dependency manager installs but does not
upgrade.

Collection and the gate

Fabric.spec.requires names each input by kind, name and namespace, and only
what it names takes part in the render — so the rendered document is a function
of the Fabric, not of whatever else exists in the namespace. That matters
because the render is pushed to a device as a full configuration replacement.

"Not yet fetched" and "does not exist" are distinguishable and reported
differently (WaitingForInputs against InputsMissing); the function refuses to
render without its inputs.

Cross-namespace collection is proven on a cluster, not only in render: a
Fabric in avd naming a SettingSet in net-svc reached Ready=True and
ntp server vrf MGMT appeared in spine1's rendered config. Crossplane's default
RBAC permits it; no extra grant was needed.

The migration asks Ansible

function/ansible_cli.py puts three questions to the Ansible CLI:

question	how
which group carries which variable	ansible-inventory --list --export
what a play resolves those variables to	ad-hoc debug with --tree
which play runs eos_designs, on which devices	ansible-playbook --list-hosts --list-tasks

Nothing here parses a group_vars tree, expands a host pattern or opens a vault.
--export already merges every shape a hand-written reader has to get right —
.yaml spellings, host_vars/<host>/ directories, inline group vars: blocks —
so those are closed by construction. --list-hosts resolves the play's pattern
with Ansible's own engine, including ! and :&.

function/migrate.py makes one Fabric per eos_designs play plus one input XR
per ownership fragment per category, and ships as avd-migrate with --emit.

The one rule the Ansible CLI does not print is group order (depth, then
name). The migration layers the fragments with it and refuses to emit
anything unless the result equals what Ansible reports. A wrong order cannot
reach an XR.

Values are what the play produced, not what the file says — an XR has no
templating engine, no vault and no playbook directory. The trigger is not "does
this look like Jinja" but "did the play produce something else", which needs no
pattern and catches vault too: ansible-inventory does not decrypt, it emits
{"__ansible_vault": "$ANSIBLE_VAULT;1.1;..."}, which reads like a resolved
value in a diff and is not.

ansible-core is a development dependency. The runtime is handed input XRs
and never sees an inventory — fn.py imports push, engine and kinds and
nothing else — so it reaches neither the image nor the published package.

Evidence

Measured against AVD's own corpus rather than argued, with Ansible itself as
the reference.

check	result
hostvars identical to what Ansible reports	8/8 examples + 19/19 molecule scenarios, up to 501 devices in one play
rendered configs match checked-in golden	8/8 examples · 8/12 molecule scenarios shipping one

cv-pathfinder renders clean for the first time. The four that remain are two
causes, and neither is reachable from this repository. Three of them meet the
same wall: pyavd implements no Jinja templating — get_device_structured_config
passes templar=None and the call raises NotImplementedError, and AVD's own
action plugin reaches into pyavd's internal API to hand in Ansible's templar.
The fourth loads a custom Python module at render time.

eos_designs-twodc-5stage-clos looks like a third cause (pool_manager) and is
not: supplying the public PoolManager gets it past that error and it fails
immediately on custom_templates/ip_addressing/…j2. Implementing pool_manager
would gain no golden — it is worth building for fabrics that use ID pools, not
for this metric.

avd-migrate --drop-description-templates renders past the first of those where
losing it is cosmetic — measured on evpn_underlay_ebgp_overlay_ebgp, all 16
devices render and differ from golden in 168 places, every one a description.
Addressing templates are never dropped: eos_designs-twodc-5stage-clos computes
its P2P uplink IPs that way, and one flag over both would have silently emitted a
different network.

What this replaces

An earlier implementation — a hand-written reader of inventories, group_vars and
host_vars, a fold squeezing many hostvars into one document, and their
verification harnesses — is deleted along with it (xr.py, ansible_inputs.py,
verify_xr.py, verify_example.py, verify_kinds.py, two test files).

Its equivalence test compared two of our own readers against each other, so a
source neither read vanished from both sides and the comparison agreed. Four
gaps sat behind that green result and surfaced only when a render disagreed with
AVD's own golden. Real Ansible on one side is what closed them.

Checked, not assumed

• Every new guard was verified to fail without its change: removing type
  from the not-a-setting list gives
  kind_of disagrees with AVD's own table: {'type': ('SettingSet', 'NodeSet')}.
• The order guard uses campus-fabric because precedence is load-bearing in
  only two of the eight examples; reversing the order changes nothing in
  single-dc-l3ls or dual-dc-l3ls, so a guard written against either would
  pass while proving nothing.
• Offline suite 79 tests in ~50s, plus 31 behind -m corpus in ~3.5min.
• The Configuration package still builds from apis/ with all six XRDs.
• uv sync --locked clean. No version bump in this PR — pyproject.toml
  and uv.lock must move together, in one commit, before any tag.

Not in scope

_reconcile_input does not fill status.devices: an input does not know the
fabric's device list. The Secret implementation, pool_manager, and whether
pyavd can validate a fragment standalone are all separate.

🤖 Generated with Claude Code