Validate serialized sketches and preserve capacity in a versioned format - #3
Merged
jhurliman merged 1 commit intoSep 10, 2026
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe077d1a74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This was referenced Sep 10, 2026
jhurliman
added this pull request to stack #5
September 10, 2026 16:25
jhurliman
added a commit
that referenced
this pull request
Sep 10, 2026
…#2) Fix CountMinSketch handling of keys inherited from Object.prototype, isolate returned top-k tuples from internal state, and correct serialized-size estimates to account for power-of-two bucket widths. Repair only the affected heap path on increments instead of sorting the whole heap. Replace the broken Vows runner with Node's built-in runner while retaining the existing ordered scenarios, add five regressions, exclude development files from npm archives, use Buffer.alloc, and add Node 22/24/26 CI. The runtime floor becomes Node 6; this requires a major version when released. No version is bumped in this PR. Validation: 24 tests pass on Node 24, including existing million-item cardinality/merge checks, reserved-key roundtrips, tuple isolation, size estimates and 100,000 tracked-key updates. Packed CommonJS and ESM consumers pass. A seven-run local benchmark of 100,000 repeat updates across 1,000 keys improved from 1,141 ms median to 29 ms on macOS arm64/Node 24; this is specific to that workload. The original three regressions failed before the fixes. Stacked release follow-ups: #3 completes input/serialization validation and preserves capacity in CMS2; #4 adds public/deep-import declarations, installed-package checks and the 2.0.0 migration. This parent PR retains the legacy binary layout; the next layer introduces the versioned format.
jhurliman
added a commit
that referenced
this pull request
Sep 10, 2026
Stacked on #3 (`release/streamcount-validation`), which is stacked on #2. Completes the remaining public declarations and distributed-package validation, and prepares version 2.0.0. Declarations cover every root export and the existing CountMinSketch, HyperLogLog, MinHeap and PRNG deep imports. Tests install the tarball independently, exercise CommonJS/ESM, and compile positive/negative NodeNext and Node16 TypeScript consumers. The package includes its format/migration documentation. Tests run before publishing; no automatic publishing is configured. Validation: 37 tests pass locally, including the signed-minimum hash regression and all parent serialization/estimator tests. CI installs the development dependencies on Node 22/24/26 and adds a separate Node 6 runtime-floor smoke test. Removes the obsolete Travis matrix identified by the review of #2. npm publication and merging are not performed by this PR. Stacked follow-up: #6 adds validated weighted increments, superseding #1. Merge this PR before #6.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2 (
modernization/test-runtime). This completes construction/deserialization validation and capacity preservation from that PR's release checklist.CountMinSketch now writes CMS2 with original capacity and 32-bit UTF-8 key lengths. It still reads legacy bytes and supports explicit legacy export. Legacy imports accept the original capacity when known; the documented fallback uses the stored entry count (at least one). Readers validate exact byte windows, dimensions, allocation budgets, hash metadata, keys, counts and HLL registers before using them. Counter overflow and invalid merges fail before mutation.
Validation: 35 tests pass locally, retaining million-item estimator checks and adding legacy byte fixtures, every truncated prefix, malformed dimensions/keys, Unicode, empty/partially full capacity roundtrips and atomic failure cases. Default CMS output and stricter input rules are breaking changes; SERIALIZATION.md documents the migration. The next stacked PR supplies declarations and reproducible package/type validation. This PR is ready for review, not merged or published.