From 02575187eb81aa4ee6a127798bf037c7f6c00ad5 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Tue, 29 Sep 2026 19:19:36 +1000 Subject: [PATCH 1/2] docs(matrix): cachekit-core envelope pre-scan in review (LAB-3479) Wire format (ByteStorage): the Rust (canonical cachekit-core) cell moves to in review with cachekit-core#80, which runs the Retrieve Flow step-2 pre-scan in ByteStorage::retrieve and executes decode-bounds.json 1.1.0 through it. Python and TypeScript cells and the TypeScript test-vectors cell stay warnings, name the core PR, and flip on each SDK's core bump. --- changelog.d/20260929_lab-3479.md | 15 +++++++++++++++ sdk-feature-matrix.md | 8 ++++---- 2 files changed, 19 insertions(+), 4 deletions(-) create mode 100644 changelog.d/20260929_lab-3479.md diff --git a/changelog.d/20260929_lab-3479.md b/changelog.d/20260929_lab-3479.md new file mode 100644 index 0000000..2d4a524 --- /dev/null +++ b/changelog.d/20260929_lab-3479.md @@ -0,0 +1,15 @@ +### Wire format — cachekit-core envelope pre-scan in review (LAB-3479) + +- [Feature matrix](sdk-feature-matrix.md#protocol-compliance) Wire format + (ByteStorage) row: the Rust (canonical `cachekit-core`) cell moves from ⚠️ + "no step-2 pre-scan" to 🚧 in review — + [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) + runs the [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 structural + pre-scan in `ByteStorage::retrieve` and `validate`, and core CI drives + `decode-bounds.json` 1.1.0 through `retrieve` asserting the pre-scan error. + crates.io 0.6.0 does not carry it, so the cell is not ✅. +- The Python and TypeScript Wire format cells, and the TypeScript Test vectors + in CI cell, stay ⚠️: each now names the core PR and flips when that SDK bumps + to a core release carrying the pre-scan and drives the vectors through its + envelope entry point (py `retrieve`, ts `unpack`). +- Footnote ¹⁶ records that `cachekit-core` vendors `decode-bounds.json` 1.1.0. diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 54610a7..80a9f9d 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -6,7 +6,7 @@ **Feature parity and compliance status across all CacheKit SDK implementations.** -*Last updated: 2026-09-29 — LAB-523 hardware-acceleration detection (following LAB-687's keyring reconciliation and LAB-1400's matrix baseline correction). Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.* +*Last updated: 2026-09-29 — LAB-3479 cachekit-core envelope pre-scan (following LAB-523's hardware-acceleration detection and LAB-687's keyring reconciliation). Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.* *__Cells that reversed — check these if you built on them:__ Rust `::secure` preset and Rust sync support (both ✅ → do not exist), Builder API (py/ts ✅ → ❌), Hardware acceleration (rs ✅ → not re-exported, ts N/A → ❌), TypeScript Arrow (🔜 → ❌), Python's encrypted read path (documented fail-closed → **fail-open by default**), and `cache.secure.wrap()` in TypeScript (implied encryption → no guarantee → **enforced since LAB-513: throws without encryption**). The TypeScript protocol-1.1 `bin` rollout also reversed twice in two days: it is **not** shipped on either ts path (per-artifact evidence in the [cachekit-core architecture note](#architecture-notes)).* @@ -293,12 +293,12 @@ its spec: | Requirement | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | | Key generation (Blake2b) | ✅ Compliant | N/A auto mode¹⁴ — interop/v1 keygen ✅ merged ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)); `#[cachekit]` mints interop keys ([#35](https://github.com/cachekit-io/cachekit-rs/pull/35)) | ✅ Compliant | ⚠️ Untested | -| Wire format (ByteStorage) | ⚠️ Compliant¹⁵ except the envelope pre-scan: payload decodes are pre-scanned (`unpackb_bounded`), but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve`, which has no [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 pre-scan | ⚠️ Canonical (`cachekit-core`) — unused for stored values¹⁵; `ByteStorage::retrieve` decodes the envelope (typed `rmp_serde::from_slice` into `StorageEnvelope`) with no step-2 pre-scan | ⚠️ Compliant except the envelope pre-scan: payload decodes are pre-scanned, but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve` (`cachekit-core-ts` `unpack`), which has no step-2 pre-scan | ⚠️ Untested | +| Wire format (ByteStorage) | ⚠️ Compliant¹⁵ except the envelope pre-scan: payload decodes are pre-scanned (`unpackb_bounded`), but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve`, which has no [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 pre-scan in any published core; it lands in [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80), and this cell flips when cachekit-py bumps to a core release that carries it | 🚧 Canonical (`cachekit-core`) — unused for stored values¹⁵; step-2 pre-scan in review — [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80): `ByteStorage::retrieve` and `validate` walk the envelope bytes (depth 100, slot budget, fail closed) before the typed `rmp_serde::from_slice` into `StorageEnvelope`, and core CI drives `decode-bounds.json` 1.1.0 through `retrieve` asserting the pre-scan error; absent from crates.io 0.6.0 | ⚠️ Compliant except the envelope pre-scan: payload decodes are pre-scanned, but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve` (`cachekit-core-ts` `unpack`), which has no step-2 pre-scan in any published core; it lands in [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80), and this cell flips when cachekit-ts bumps to a core release that carries it | ⚠️ Untested | | Storage container (auto mode)¹⁵ | CK v3 frame (Python-internal) | Plain MessagePack (`rmp` named) — no envelope | Bare ByteStorage envelope (default) | — | | Encryption (AES-256-GCM) | ✅ Compliant | ✅ Canonical (cachekit-core) | ✅ Compliant | ⚠️ Untested | | AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented | | SaaS API | ✅ Compliant | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented | -| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503) — ⚠️ asserts rejection and a peak-memory budget, not that the structural guard rejected (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`) — ⚠️ asserts the error type only, not that the structural guard rejected | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`cachekit-core-ts` `unpack`), which has no guard to assert (see the Wire format row) | ⚠️ Pending | +| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503) — ⚠️ asserts rejection and a peak-memory budget, not that the structural guard rejected (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`) — ⚠️ asserts the error type only, not that the structural guard rejected | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`cachekit-core-ts` `unpack`), which has no guard to assert: the guard lands in [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80), and this flips when cachekit-ts bumps to a core release that carries it and drives the vectors through `unpack` (see the Wire format row) | ⚠️ Pending | | Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented | > [!NOTE] @@ -306,7 +306,7 @@ its spec: > > ¹⁵ Auto-mode **stored bytes** are SDK-internal and differ per SDK — see [wire-format.md → SDK Storage Containers](spec/wire-format.md#sdk-storage-containers-auto-mode). Python stores the ByteStorage envelope *inside* its CK v3 frame; `cachekit-rs` does not use the envelope for values at all (it uses `cachekit-core` only for encryption). Cross-SDK value compatibility is exclusively an [interop-mode](spec/interop-mode.md) property (protocol#11). > -> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)), and — since LAB-2503 — `decode-bounds.json` ([`tools/decode-bounds-reference.py`](tools/decode-bounds-reference.py), `verify` in both the stdlib and the optional-deps legs) against reference implementations. The SDKs vendor earlier revisions of `decode-bounds.json`, all labelled `1.0.0`: cachekit-py and cachekit-ts the 13-reject / 2-accept revision, cachekit-rs a 10-reject / 2-accept one. This repo's file is `1.1.0` (17 reject, 3 accept); vectors newer than an SDK's copy run only here until that SDK re-vendors. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). +> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)), and — since LAB-2503 — `decode-bounds.json` ([`tools/decode-bounds-reference.py`](tools/decode-bounds-reference.py), `verify` in both the stdlib and the optional-deps legs) against reference implementations. The SDKs vendor earlier revisions of `decode-bounds.json`, all labelled `1.0.0`: cachekit-py and cachekit-ts the 13-reject / 2-accept revision, cachekit-rs a 10-reject / 2-accept one. This repo's file is `1.1.0` (17 reject, 3 accept); vectors newer than an SDK's copy run only here until that SDK re-vendors. `cachekit-core` vendors `1.1.0` and drives it through `ByteStorage::retrieve` in [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) (in review). `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). > > ¹⁷ Version cells are **floors** (`X+`), not snapshots — they stay true as new versions publish; check the registry for the current release. Python's floor is the first *installable* one: interop merged under the `v0.13.0` tag, but neither `0.12.0` nor `0.13.0` was ever published to PyPI, so `0.14.0` is the earliest PyPI release containing interop mode. Do not "correct" this to 0.13.0 from the cachekit-py changelog alone. From b8d9fc3727486f62edbb9edab1d646e5de92c4b8 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Tue, 29 Sep 2026 19:32:35 +1000 Subject: [PATCH 2/2] docs(changelog): separate the Wire format and Test vectors flip conditions (LAB-3479) --- changelog.d/20260929_lab-3479.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/changelog.d/20260929_lab-3479.md b/changelog.d/20260929_lab-3479.md index 2d4a524..1dfaaa5 100644 --- a/changelog.d/20260929_lab-3479.md +++ b/changelog.d/20260929_lab-3479.md @@ -8,8 +8,9 @@ pre-scan in `ByteStorage::retrieve` and `validate`, and core CI drives `decode-bounds.json` 1.1.0 through `retrieve` asserting the pre-scan error. crates.io 0.6.0 does not carry it, so the cell is not ✅. -- The Python and TypeScript Wire format cells, and the TypeScript Test vectors - in CI cell, stay ⚠️: each now names the core PR and flips when that SDK bumps - to a core release carrying the pre-scan and drives the vectors through its - envelope entry point (py `retrieve`, ts `unpack`). +- The Python and TypeScript Wire format cells stay ⚠️: each now names the core + PR and flips when that SDK bumps to a core release carrying the pre-scan. +- The TypeScript Test vectors in CI cell stays ⚠️ for the envelope entry point: + it flips once cachekit-ts has bumped core and drives the vectors through + `unpack`. - Footnote ¹⁶ records that `cachekit-core` vendors `decode-bounds.json` 1.1.0.