diff --git a/CHANGELOG.md b/CHANGELOG.md index 8d9e8eb..e45332e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ All notable changes to the CacheKit Protocol Specification. ## [Unreleased] +### Wire format — vendored-fixture coverage note corrected (LAB-1750) + +- [`spec/wire-format.md`](spec/wire-format.md) no longer says `cachekit-core` vendors + fixture 1.1.0. It pins 1.1.1, so `width_boundary_bin16_bin` has a canonical-writer + (`lz4_flex`) compressed-byte and xxh3-64 checksum check. The section now states the rule + for anyone vendoring the fixture: derive each `*_bin` twin's expected marker from its + decoded `compressed_data` length, never assume bin8 or accept any `bin` width. + ### Encryption — default-tenant conformance vector (LAB-4666) - [`test-vectors/encryption.json`](test-vectors/encryption.json) gains a `default_tenant` diff --git a/spec/wire-format.md b/spec/wire-format.md index e0d3ae3..ba3b97e 100644 --- a/spec/wire-format.md +++ b/spec/wire-format.md @@ -41,8 +41,7 @@ This document specifies two layers: decode byte-identity for every vector and re-encode byte-identity for the canonical `*_bin` vectors only — legacy array-of-integers vectors are decode-only, retained as legacy-read proof. That re-encode assertion covers - only the vectors the pinned file contains (core currently vendors 1.1.0, with - the resulting gap detailed below). Byte-canonicity scopes to the + only the vectors the pinned file contains. Byte-canonicity scopes to the envelope's MessagePack encoding and to the **canonical writer's** output: the LZ4 bytes inside `compressed_data` are not reproducible across conforming compressors — see @@ -287,31 +286,23 @@ bytes are therefore valid LZ4 block satisfies, so it accepts a re-pin to unrelated bytes. Neither half runs `lz4_flex`, so neither can detect an `lz4_flex` **behaviour** change; that remains the job of the re-encode assertions in `cachekit-core` described - below, subject to the vendored-version gap noted there. + below. This is the same doctrine [interop v2](interop-v2.md) records for its compressed-values profile. The pinned bytes are the **canonical implementation's** output (`lz4_flex` via `cachekit-core`), enforced by the re-encode byte-identity -assertions in `cachekit-core/tests/wire_format_vectors.rs` — **but only for the -vectors present in the fixture that repo vendors**. That matters today: -cachekit-core vendors 1.1.0 and pins `version == "1.1.0"`, so -`width_boundary_bin16` (added at 1.1.1) has **no canonical-writer (`lz4_flex`) -compressed-byte check anywhere in the fleet**, and its pinned xxh3-64 checksum -is recomputed nowhere. Its MessagePack encoding *is* covered: this repo's -`tools/wire-format-reference.py verify` asserts legacy and bin re-encode -byte-identity for it on every run, and liblz4 reproduces its compressed bytes -on the optional `lz4` leg — so do not read this gap as "the vector is -unverified". Closing it means re-vendoring 1.1.1 into cachekit-core, which -requires three changes together, not one: bump `FIXTURE_SHA256`, bump the -`version == "1.1.0"` pin to `1.1.1`, and relax -`assert_eq!(twin_bytes[1], 0xc4)` to accept `0xc5` — that assertion currently -requires *every* twin to be bin8, and `width_boundary_bin16_bin` is bin16 -(marker `0xc5`, 303-byte `compressed_data`), which is the whole point of the -vector. A drop-in re-vendor fails that test. The reference liblz4 mapping -above (`lz4.block`) is **decode-verified against every vector** in this repo's -CI (`tools/wire-format-reference.py verify`, optional `lz4` leg); on encode it -reproduces every pair except `large_compressible` byte-for-byte, which is an -observation, not a guarantee — but one this repo's CI pins (see +assertions in `cachekit-core/tests/wire_format_vectors.rs` — **for the vectors +present in the fixture that repo vendors**, which recompute each twin's +`lz4_flex` bytes and xxh3-64 checksum. Anyone vendoring the fixture should +derive each `*_bin` twin's expected marker from its decoded `compressed_data` +length (`≤255 → 0xc4`, `≤65535 → 0xc5`, else `0xc6`), as cachekit-core does. An +assertion that every twin is bin8 fails on `width_boundary_bin16_bin` (`0xc5`, +303-byte `compressed_data`), and one that accepts all three widths cannot +detect a non-shortest header. The reference liblz4 mapping above (`lz4.block`) is **decode-verified against +every vector** in this repo's CI (`tools/wire-format-reference.py verify`, +optional `lz4` leg); on encode it reproduces every pair except +`large_compressible` byte-for-byte, which is an observation, not a guarantee — +but one this repo's CI pins (see `LZ4_ENCODE_DIVERGENT`), so a toolchain change that alters the divergent set fails CI rather than quietly making this paragraph wrong.