diff --git a/changelog.d/20260929_lab-3479.md b/changelog.d/20260929_lab-3479.md new file mode 100644 index 0000000..7417075 --- /dev/null +++ b/changelog.d/20260929_lab-3479.md @@ -0,0 +1,19 @@ +### Wire format — cachekit-core envelope pre-scan on core `main`, unreleased (LAB-3479) + +- [Feature matrix](sdk-feature-matrix.md#protocol-compliance) Wire format + (ByteStorage) row: the Rust (canonical `cachekit-core`) cell moves from ⚠️ + "no step-2 pre-scan" to 🚧 unreleased — + [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80), + merged to core `main`, runs the + [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 structural + pre-scan in `ByteStorage::retrieve` and `validate`, and core CI drives + `decode-bounds.json` 1.1.0 through `retrieve` asserting the pre-scan error. + crates.io 0.6.0 does not carry it, so the cell is not ✅. +- The Python and TypeScript Wire format cells stay ⚠️: each now names the core + PR and flips when that SDK moves to a core release carrying the pre-scan. + For TypeScript that means both bindings, `cachekit-core-ts` and + `cachekit-core-wasm`. +- The Python and TypeScript Test vectors in CI cells stay ⚠️ for the envelope + entry point, and name the same condition. TypeScript also needs the vectors + driven through `unpack`. +- Footnote ¹⁶ records that `cachekit-core` vendors `decode-bounds.json` 1.1.0. diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 26ac45a..dc8be14 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -6,7 +6,7 @@ **Feature parity and compliance status across all CacheKit SDK implementations.** -*Last updated: 2026-09-29 — LAB-523 hardware-acceleration detection (following LAB-687's keyring reconciliation and LAB-1400's matrix baseline correction). Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.* +*Last updated: 2026-09-30 — LAB-3479 cachekit-core envelope pre-scan on core `main`, unreleased (following LAB-3481's decode-bounds 1.1.0 re-vendor and LAB-523's hardware-acceleration detection). Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.* *__Cells that reversed — check these if you built on them:__ Rust `::secure` preset and Rust sync support (both ✅ → do not exist), Builder API (py/ts ✅ → ❌), Hardware acceleration (rs ✅ → not re-exported, ts N/A → ❌), TypeScript Arrow (🔜 → ❌), Python's encrypted read path (documented fail-closed → **fail-open by default**), and `cache.secure.wrap()` in TypeScript (implied encryption → no guarantee → **enforced since LAB-513: throws without encryption**). The TypeScript protocol-1.1 `bin` rollout also reversed twice in two days: it is **not** shipped on either ts path (per-artifact evidence in the [cachekit-core architecture note](#architecture-notes)).* @@ -293,13 +293,13 @@ its spec: | Requirement | Python | Rust | TypeScript | PHP | | :--- | :---: | :---: | :---: | :---: | | Key generation (Blake2b) | ✅ Compliant | N/A auto mode¹⁴ — interop/v1 keygen ✅ merged ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)); `#[cachekit]` mints interop keys ([#35](https://github.com/cachekit-io/cachekit-rs/pull/35)) | ✅ Compliant | ⚠️ Untested | -| Wire format (ByteStorage) | ⚠️ Compliant¹⁵ except the envelope pre-scan: payload decodes are pre-scanned (`unpackb_bounded`), but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve`, which has no [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 pre-scan | ⚠️ Canonical (`cachekit-core`) — unused for stored values¹⁵; `ByteStorage::retrieve` decodes the envelope (typed `rmp_serde::from_slice` into `StorageEnvelope`) with no step-2 pre-scan | ⚠️ Compliant except the envelope pre-scan: payload decodes are pre-scanned, but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve` (`cachekit-core-ts` `unpack`), which has no step-2 pre-scan | ⚠️ Untested | +| Wire format (ByteStorage) | ⚠️ Compliant¹⁵ except the envelope pre-scan: payload decodes are pre-scanned (`unpackb_bounded`), but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve`, which has no [Retrieve Flow](spec/wire-format.md#retrieve-flow) step-2 pre-scan in any published core; it is on core `main` ([cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80)), unreleased, and this cell flips when cachekit-py bumps to a core release that carries it | 🚧 unreleased — Canonical (`cachekit-core`), unused for stored values¹⁵; step-2 pre-scan on core `main` ([cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80)): `ByteStorage::retrieve` and `validate` pre-scan the envelope bytes before the typed `rmp_serde::from_slice` into `StorageEnvelope`; absent from crates.io 0.6.0 | ⚠️ Compliant except the envelope pre-scan: payload decodes are pre-scanned, but the envelope goes through `cachekit-core`'s `ByteStorage::retrieve` (`unpack` in both `cachekit-core-ts` and `cachekit-core-wasm`), which has no step-2 pre-scan in any published core; it is on core `main` ([cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80)), unreleased, and this cell flips when both bindings move to a core release that carries it | ⚠️ Untested | | Storage container (auto mode)¹⁵ | CK v3 frame (Python-internal) | Plain MessagePack (`rmp` named) — no envelope | Bare ByteStorage envelope (default) | — | | Encryption (AES-256-GCM) | ✅ Compliant | ✅ Canonical (cachekit-core) | ✅ Compliant | ⚠️ Untested | | AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented | | SaaS API | ✅ Compliant except path encoding (row below) | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented | | SaaS API — cache-key path encoding ([spec](spec/saas-api.md#cache-key-path-encoding)) | ⚠️ Partial — rules 1/3/4 since 0.18.0+ ([cachekit-py#279](https://github.com/cachekit-io/cachekit-py/pull/279)); rule 2 not implemented | ✅ Compliant on `main`, unreleased ([cachekit-rs#76](https://github.com/cachekit-io/cachekit-rs/pull/76)) | ✅ Compliant on `main`, unreleased ([cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118)) | ❌ Not implemented | -| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503); `1.1.0` in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), unreleased, asserting the structural guard's own error for every reject vector at every payload read path (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) — ⚠️ except the envelope entry point (`ByteStorage::retrieve`), which has no guard to assert (see the Wire format row) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`); `1.1.0` in [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93), unreleased, asserting the structural guard's own `decode bound:` error for every reject vector through both decoders and `get` / `interop_get` / `interop_get_swr` — the guard enforces depth itself since that PR (no envelope entry point¹⁵) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), `1.1.0` in [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152), unreleased, asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`cachekit-core-ts` `unpack`), which has no guard to assert (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118), unreleased | ⚠️ Pending | +| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) — fixture 1.1.0 (`ns`/`nsapi` namespace reservation) in [cachekit-py#350](https://github.com/cachekit-io/cachekit-py/pull/350), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-py#276](https://github.com/cachekit-io/cachekit-py/pull/276) (LAB-2503); `1.1.0` in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), unreleased, asserting the structural guard's own error for every reject vector at every payload read path (the [Decode bounds](spec/interop-mode.md#decode-bounds) MUST) — ⚠️ except the envelope entry point (`ByteStorage::retrieve`), which has no guard to assert until cachekit-py moves to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) (see the Wire format row) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) — fixture 1.1.0 in [cachekit-rs#89](https://github.com/cachekit-io/cachekit-rs/pull/89), unreleased; `decode-bounds.json` vendored + CI-executed since [cachekit-rs#73](https://github.com/cachekit-io/cachekit-rs/pull/73) (LAB-2503; default CI green on `main`); `1.1.0` in [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93), unreleased, asserting the structural guard's own `decode bound:` error for every reject vector through both decoders and `get` / `interop_get` / `interop_get_swr` — the guard enforces depth itself since that PR (no envelope entry point¹⁵) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors — fixture 1.1.0 in [cachekit-ts#143](https://github.com/cachekit-io/cachekit-ts/pull/143), unreleased; decode bounds enforced ([#112](https://github.com/cachekit-io/cachekit-ts/pull/112)); `decode-bounds.json` vendored + CI-executed since [cachekit-ts#121](https://github.com/cachekit-io/cachekit-ts/pull/121) (LAB-2737), `1.1.0` in [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152), unreleased, asserting the guard error each vector trips (pre-scan, or the event size cap ahead of it) — ⚠️ except the envelope entry point (`unpack` in `cachekit-core-ts` and `cachekit-core-wasm`), which has no guard to assert until both bindings move to a core release carrying [cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80) and the vectors run through `unpack` (see the Wire format row); `path-encoding.json` vendored + CI-executed since [cachekit-ts#118](https://github.com/cachekit-io/cachekit-ts/pull/118), unreleased | ⚠️ Pending | | Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented | > [!NOTE] @@ -307,7 +307,7 @@ its spec: > > ¹⁵ Auto-mode **stored bytes** are SDK-internal and differ per SDK — see [wire-format.md → SDK Storage Containers](spec/wire-format.md#sdk-storage-containers-auto-mode). Python stores the ByteStorage envelope *inside* its CK v3 frame; `cachekit-rs` does not use the envelope for values at all (it uses `cachekit-core` only for encryption). Cross-SDK value compatibility is exclusively an [interop-mode](spec/interop-mode.md) property (protocol#11). > -> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)), and — since LAB-2503 — `decode-bounds.json` ([`tools/decode-bounds-reference.py`](tools/decode-bounds-reference.py), `verify` in both the stdlib and the optional-deps legs), and — since LAB-2879 — `path-encoding.json` ([`tools/path-encoding-verify.py`](tools/path-encoding-verify.py)) against reference implementations. This repo's `decode-bounds.json` is `1.1.0` (17 reject, 3 accept), and cachekit-py, cachekit-rs and cachekit-ts vendor it byte for byte in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93) and [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152) (unreleased); their earlier copies were `1.0.0` revisions (13 reject / 2 accept in py and ts, 10 / 2 in rs). `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). +> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)), and — since LAB-2503 — `decode-bounds.json` ([`tools/decode-bounds-reference.py`](tools/decode-bounds-reference.py), `verify` in both the stdlib and the optional-deps legs), and — since LAB-2879 — `path-encoding.json` ([`tools/path-encoding-verify.py`](tools/path-encoding-verify.py)) against reference implementations. This repo's `decode-bounds.json` is `1.1.0` (17 reject, 3 accept), and cachekit-py, cachekit-rs and cachekit-ts vendor it byte for byte in [cachekit-py#363](https://github.com/cachekit-io/cachekit-py/pull/363), [cachekit-rs#93](https://github.com/cachekit-io/cachekit-rs/pull/93) and [cachekit-ts#152](https://github.com/cachekit-io/cachekit-ts/pull/152) (unreleased); their earlier copies were `1.0.0` revisions (13 reject / 2 accept in py and ts, 10 / 2 in rs). `cachekit-core` vendors `1.1.0` too and drives it through `ByteStorage::retrieve`, on core `main` ([cachekit-core#80](https://github.com/cachekit-io/cachekit-core/pull/80)), unreleased. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425). > > ¹⁷ Version cells are **floors** (`X+`), not snapshots — they stay true as new versions publish; check the registry for the current release. Python's floor is the first *installable* one: interop merged under the `v0.13.0` tag, but neither `0.12.0` nor `0.13.0` was ever published to PyPI, so `0.14.0` is the earliest PyPI release containing interop mode. Do not "correct" this to 0.13.0 from the cachekit-py changelog alone.