Add metadata UUID - #539
Open
TApplencourt wants to merge 17 commits into
Open
Conversation
print_handle_uuid_modules printed both to cuda, hip and mpi. Handle was prepended by none of the three -- ze is its only user, through an inline copy of its own -- and mpi prepends neither, so it carried both definitions for nothing. What is left prints one module, and mpi prints none: -51 lines across the three libraries, nothing added. The byte-identical oracle compares generated TEXT, so it cannot tell a struct that renders readably from one that hands its bytes over raw, and that is the whole job of these modules. Measured out of tree instead, by loading each library, filling every byte-array struct with control characters and reading back what to_s produced. That says this commit changes no rendering: cuda 5 blob structs / 4 unreadable and hip 4 / 3 both before and after, mpi 0 / 0 -- the same fact as the dead code, measured from the other side. It also puts a number on the bug underneath. Which struct gets a module is decided by a match on its class name, and nine structs across cuda, hip and ze fall outside it: CUIPCEventHandleV1, CUIPCMemHandleV1, CUMemFabricHandleV1, CUMemPoolPtrExportDataV1, HipIpcMemHandle_t, HipIpcEventHandle_t, HipMemPoolPtrExportData, ZEDeviceLuidExt and ZESFabricLinkType reach a trace as raw bytes. Fixing that means saying which rendering a type wants instead of guessing from its name, and is left to a following commit. ze 9/9 and xprof 2/2 pass on a Max 1550. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which struct got a readable rendering was decided by a match on its class
name: /UUID/ in the shared printer, /UUID/ then /Handle\z/ in ze's. A name is
not the fact the renderer needs, and it was wrong in both directions.
It missed nine structs, which reached a trace as raw bytes -- their to_s
handed the array straight to string interpolation. Four in cuda
(CUIPCEventHandleV1, CUIPCMemHandleV1, CUMemFabricHandleV1,
CUMemPoolPtrExportDataV1), three in hip (HipIpcMemHandle_t,
HipIpcEventHandle_t, HipMemPoolPtrExportData), and ZEDeviceLuidExt, whose
header calls it "opaque data representing a device LUID".
It also matched two structs that hold no bytes at all.
ZEExternalMemoryImportWin32Handle and its Export twin end in Handle and are
{stype, pNext, flags, handle, name}, so the module's self[:data] found no
such field: printing either raises ArgumentError on devel today.
A `renderings:` map in the backend's YAML says which renderer a type wants,
beside the meta-parameters that are already declared there rather than
derived. Nothing about the shape can answer it -- an opaque handle, a UUID
and a fixed-width string are all uint8_t x[N], and zes_fabric_link_type_t is
a string that renders correctly with no module at all.
The renderers read members.first instead of a member name of their own, which
is what lets one Handle serve ze's `data` and cuda's `reserved`; the two
UUID orders stay separate because cuda and hip print first byte first and ze
last byte first. ze keeps KUUID: ze_kernel_uuid_t renders kid AND mid, which
a single-member renderer cannot express.
cuda and hip UUIDs now print under their real member name, `bytes`, where the
old renderer hardcoded `id`. Nothing parses that label.
Every backend is now 0 unreadable, where cuda was 4, hip 3 and ze 2, and ze
drops from 11 structs that raise on to_s to 9 -- the rest being the Base*
classes that PR #537 fixes. On a Max 1550 the pretty-printed trace is
identical, UUID rendering included; ze 9/9 and xprof 2/2 pass. cuda and hip
have no hardware here and are covered by the rendering test alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The renderers read members.first, which is why ze_kernel_uuid_t could not use
one: it is {kid, mid}, two uint8_t[16] UUIDs side by side, and a renderer that
reads the first member would have dropped mid. It kept a hand-written KUUID
module -- 44 generated lines spelling out both byte orders index by index --
and a `name == 'ze_kernel_uuid_t'` branch in print_struct to reach it.
Mapping over every member instead makes that a `renderings:` row like any
other. For a struct holding one blob, mapping all members IS members.first, so
nothing single-member changes; the pair now renders
`{ kid: <uuid>, mid: <uuid> }`, byte for byte what KUUID produced.
Nothing about the shape can pick the renderer, which is why the row is still
needed: uint8_t[16] and char[16] are both UUIDs here (ze_uuid_t and CUuuid),
both are also opaque handles elsewhere, and cuda has no uint8_t byte array at
all. The C type does not carry the answer, so the header's answer stays
declared.
Counting only structs with exactly one byte-array member left ze_kernel_uuid_t
outside the one check that could have caught this. Counting any struct whose
members are all byte arrays takes ze from 10 to 11, still 0 unreadable, with
cuda 5 / hip 4 unchanged.
Generated ze_library.rb drops 67 lines against devel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`renderings:` keyed a renderer to a struct, so it could only speak for a struct
whose members all want the same one. Most byte arrays do not live alone:
zes_device_properties_t is six fixed-width strings among ten members, and
zet_metric_properties_t four among nine. Twenty-odd structs are that shape, and
a per-struct key cannot describe any of them.
Rows are per member now, and shaped like the meta-parameters beside them
because they are the same kind of fact -- something the header knows that the C
declaration does not carry:
meta_parameters_struct:
ze_kernel_uuid_t:
- [ UuidReversed, kid ]
- [ UuidReversed, mid ]
That retires the last of the shape-guessing. ze_kernel_uuid_t needed a renderer
that mapped over every member; now it names both. A member with no row renders
the way the base class renders it, which is what the strings want, so declaring
the 13 opaque members says everything and the 44 string members stay silent.
Handle becomes Blob, and stops formatting bytes by hand: read_bytes(size) keeps
all of them and `.b.inspect` escapes the ones that need it. A blob is not a C
string -- reading ze_ipc_mem_handle_t as text stops at the first NUL and
silently drops the other 63 bytes -- and inspect leaves printable bytes legible
where the old loop hexed every one.
The UUID renderer keeps whichever canonical dashes the array is long enough to
reach, so ze_device_luid_ext_t can ask for UUID at all: it is 8 bytes, and the
fixed 4-2-2-2-6 grouping raised NoMethodError on it (hex[10, 6] is nil). 16
bytes still render 8-4-4-4-12, and the LUID reads 17161514-1312-1110.
ze 11 byte-blob structs / 0 unreadable, cuda 5 / 0, hip 4 / 0. The kernel UUID
pair renders byte for byte what the hand-written KUUID did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three things the previous commit left half-done.
load_meta_parameters_struct read the same files load_meta_parameters had just
read. There is one meta-parameter file per backend and it now yields both of
its sections, so each backend loads it once and takes what it needs; the
separate function is gone and the other five backends ask for
[:meta_parameters].
The renderers were emitted as one module per struct: ten near-identical
ZE*Rendering modules in ze alone, five of them the same body under different
names, plus a RENDERED map and a case to dispatch through. That is more
generated code than the hand-written KUUID it replaced. There is now one
Rendering module of plain functions per backend, holding only the renderers
that backend's rows ask for, and a rendered struct carries a to_s that names
them directly:
def to_s
"{ kid: #{Rendering.uuid_reversed(self[:kid].to_a)}, ... }"
end
A row names that function, so `uuid` in the YAML is `Rendering.uuid` in the
bindings and there is nothing in between. An earlier attempt kept a shared
helper under the renderers, and in cuda the helper and the `Uuid` renderer both
emitted `def self.uuid` -- the second silently redefining the first into
infinite recursion. The name collision was the layer announcing itself.
check_renderings raises unless every row names a member of that struct, and one
whose element type is a byte. It caught CUmemFabricHandle_v1.data on the first
run: cuda spells that array `unsigned char`, a third spelling neither the code
nor these commit messages had accounted for.
Verified against the devel build, every struct holding a byte array, same
inputs: the set of structs rendering a UUID is unchanged except ZEDeviceLuidExt
joining it, every UUID value is identical, and nothing raises on either side.
Generated ze_library.rb is 45 lines smaller than devel. make check 48/48.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
@thapi-local-reviewer review |
![thapi-local-reviewer[bot]](https://avatars.githubusercontent.com/u/6524907?s=60&v=4){kind=small}
The rendering rows had four names that did not say what they held. `print_renderer_modules` emits one module, not several, and took a list of struct names its `naming` argument already carried. `RENDERERS` held renderer bodies, not renderers or structs. `spec` and `STRUCT_SPEC` named the `meta_parameters_struct` YAML section without saying so. The checks also ran per struct as it was printed. They now run once, before anything is emitted, so a bad row stops the build at the row rather than part way through a library. `presence` and the `rendered` temporary go with them, and the comment repeated across five YAML files moves to backends/README.md, which the backends had no shared page for until now. Generated ze, cuda and hip libraries are byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A member that is not there and a member that is not bytes fail for the same reason -- the renderer has nothing to read -- so they no longer get a branch and a message each. `byte_array_members` returns the readable members and the caller subtracts, which also names every bad member in one raise rather than stopping at the first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Rendering` named nothing: a library generator renders everything. The module holds the functions that turn a byte array into text, so it is `Bytes`, and a call site reads `Bytes.uuid_reversed(...)`. The UUID bodies were one template with an `ORDER` sentinel substituted by gsub, which put a word in the source that is not Ruby and is not in the output. Each body now opens by putting the bytes in its own order and they share the rest. Two duplicate checks kept a hash and raised on the key already in it. Counting with tally and subtracting says the same thing without the accumulator, and names every duplicate rather than stopping at the first. Generated output changes only in the module name and the added `ordered =` line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`puts initializer if initializer` prints a blank line for an empty string, which is why the caller was handing it nil instead. The printer now asks whether there is text, so the caller joins what it has and stops caring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`meta_parameters_struct` reaches a UUID only when a struct wraps it. A bare byte array has no struct to hang a renderer on, so `cuDeviceGetLuid`'s 8-byte `luid` printed as `"n\x16?\xBE\xB9XDM"` where ze, which wraps the same bytes in `ze_device_luid_ext_t`, prints `6e163fbe-b958-444d`. `meta_parameters_function` says the same thing about a function's parameters that `meta_parameters_struct` says about a struct's members, in the same `[ renderer, thing ]` rows and reaching the same `Bytes` functions. A parameter is traced under a name the tracepoint decorates -- `luid` arrives as `luid_vals` -- so the rows are re-keyed from the command index rather than by guessing the suffix. A parameter the function does not trace as bytes raises, which a wrong name would otherwise only show at trace time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`meta_parameters_function` names a parameter, `luid`; the payload carries the field the tracepoint decorates, `luid_vals`. The first version re-keyed the rows from the command index, so every babeltrace-library generator took `COMMANDS` to learn a name its own model already spells out. Measured across the 7 backends: 973 string fields, none of which collide once `_val`/`_vals` is stripped. The model is enough, and `print_babeltrace_lib` goes back to taking a naming and its rows. That restatement hid a bug. `print_bytes_module` emitted the `Bytes` functions the struct rows named, so a renderer only a function row asks for was called and never defined: `[ uuid_reversed, luid ]` generated a call to `CUDA::Bytes.uuid_reversed`, which is not there. It passed only because `CUuuid` happens to name `uuid` too. Both sections now say what to emit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TApplencourt
force-pushed
the
add_metadata_uuid
branch
from
September 11, 2026 19:45
7b2325c to
fba7c8f
Compare
Four notes had drifted or restated what is already written down elsewhere. print_typedefs still said cuda "prepends a UUID module to matching structs": that mechanism is gone, replaced by the declared rows. BYTE_TYPES claimed cuda uses all three byte spellings; measured, cuda and hip write char and unsigned char, ze writes char and uint8_t, and no backend emits int8. The byte_array_members example was written as a doctest but names NAMING, which only a backend defines, so it could never run -- it is prose now. print_bytes_module's closing paragraph is its own commit message. backends/README.md named the three renderers without saying what any of them prints, so it now carries a table. Also fixes "menbers" and "stoping". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
print_struct_with_namespace took a `prepends:` list so a backend could mix a hand-written module into a struct class. That was how Handle and UUID reached the structs that needed them; declaring the rendering per member retired the last caller. The `prepend Version` lines in ze_library.rb come from a heredoc in gen_ze_library.rb, not from here. With the loop gone the two heredocs around it join back into one, and print_struct_rendered is a two-line wrapper over a single call, so its two callers make that call themselves. Generated output is unchanged: all 62 files byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ared struct
Two ways a wrong row stayed quiet.
A backend that never called print_bytes_module could carry both rendering
sections and have them dropped on the floor: itt, omp and mpi read only
[:meta_parameters]. Adding meta_parameters_struct to mpi's yaml built clean and
generated nothing. Every backend calls print_bytes_module now -- it emits
nothing when there are no rows, but the check comes with it, so a section that
does nothing says so instead of passing.
Declaring one member of a struct replaces the whole to_s, so a byte array left
undeclared is interpolated raw. Removing ze_kernel_uuid_t's `mid` row built
clean and rendered `{ kid: 100f0e0d-..., mid: <16 control characters> }`. A
declared struct must now name a renderer for each of its byte arrays; every
struct declared today already does.
The three constants each model exported collapse to one META_PARAMETERS, since
both readers now take the whole spec and pick their own section.
Generated output is unchanged: all 62 files byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TApplencourt
force-pushed
the
add_metadata_uuid
branch
from
September 11, 2026 20:52
fba7c8f to
cf3b896
Compare
event_function_name and parameter_name were written as new helpers, but parameter_name already existed inline in gen_babeltrace_model_helper.rb as the same gsub. Both undo a decoration LTTng.rb's own constants create -- an event is <provider>:<function>_<suffix>, a parameter is traced under _val or _vals -- so they belong there, next to START and STOP, and the second copy is gone. Three names claimed more than the code does. rendering_rows does nothing rendering-specific: it indexes a section whose rows read [value, key], so it is indexed_rows, and its raise says "declared twice" rather than "rendered twice". rendered_to_s returns the TEXT of a to_s for the generated file, so it is struct_to_s_definition. print_struct_with_namespace's `initializer:` predates carrying a to_s as well, so it is `body:` -- which is what ze passes it, two definitions joined. The prose framed meta_parameters_struct and meta_parameters_function as "the two rendering sections", a separate species that happens to share a file. They are not: a meta-parameter is a fact the header knows that the C declaration cannot carry, and [InArray, globalDim, tensorRank] and [uuid_reversed, id] are the same kind of fact. They differ in what a row keys on, nothing more. Also: payload_fields names the &.[](:members) dance that appeared twice, the reject+map over it is one filter_map, load_meta_parameters' three copies of merge-raise-on-duplicate are one loop, and the comments that had grown into essays are cut to what the code cannot say. Generated output is unchanged: all 62 files byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TApplencourt
force-pushed
the
add_metadata_uuid
branch
from
September 11, 2026 22:31
cf3b896 to
ed40b51
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.