fix(rar3): reset the low-distance repeat state at each LZ block header - #130
Merged
Conversation
Fixes #126. libarchive's `test_read_format_rar3_lowdist_reset.rar` decoded to the right length with the wrong contents from offset 46 on. The member is packed as two LZ blocks; `parse_block_header` carried `last_low_offset` and `num_low_offset_repeats` across the block boundary, so the first low-distance match of the second block added a stale addend and every byte after it was wrong. The low-distance repeat state belongs to the table domain of the block that established it. unrar and libarchive both clear it on every LZ block header — crucially *before* the keep-table flag is read, so a block that reuses the previous code lengths still starts a fresh low-distance run. libarchive's `parse_codes` puts it exactly there: "Low-distance repeat state belongs to the current LZ table and must not be reused after starting a new table." The failure mode is the bad kind: wrong bytes at the correct length, which a size check cannot catch. A caller verifying the archive's FILE_CRC does catch it (as the reporter's did), so this is a correctness gap rather than a safety one. The regression test asserts against the archive's own stored FILE_CRC (0x6FF838DC), which makes the expected plaintext verifiable independently of any other reader — the 64 expected bytes CRC to exactly that value. Both the single-shot and byte-at-a-time paths are covered, the latter so the block boundary carrying the reset lands mid-call. Both tests were confirmed to fail without the fix. No regressions: the rar3/rar5/rar2/ppmd suites and the differential fixtures from #122 all still pass (1769 total, 0 failed).
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #126.
Confirmed
Reproduced exactly as reported, from libarchive's
test_read_format_rar3_lowdist_reset.rar(memberlowdist-reset.bin, method 0x35, version 29, not solid, 490 packed → 64 unpacked):I verified the expected plaintext independently rather than taking it on trust: the 64 bytes in the issue CRC to exactly the
0x6FF838DCstored in the archive's own file header, so they are a property of the archive, not of libarchive's reader.One correction to the report: the divergence begins at offset 46, not 40. The issue's prose says 40 but its own hex dump is consistent with 46.
Cause
The member is packed as two LZ blocks.
parse_block_headercarriedlast_low_offsetandnum_low_offset_repeatsacross the block boundary, so the first low-distance match in the second block added a stale addend and everything after it was wrong.The low-distance repeat state belongs to the table domain of the block that established it. unrar and libarchive both clear it on every LZ block header, and — the part that matters — before the keep-table flag is read, so a block reusing the previous code lengths still starts a fresh low-distance run. libarchive's
parse_codesplaces it exactly there:The fix is those two lines at the matching point in
parse_block_header.Why this one is worth flagging
The failure mode is wrong bytes at the correct length — the shape a size check cannot catch. A caller verifying FILE_CRC does catch it (as the reporter's did), so this is a correctness gap rather than a safety one, but any consumer that trusts length alone would have silently propagated corrupt data.
Tests
tests/fixtures/rar3/lowdist_reset.bin(the 490-byte packed run) plus two tests: single-shot and byte-at-a-time, the latter so the block boundary carrying the reset lands mid-call. Both assert against the archive's stored FILE_CRC as well as the literal bytes. Both were confirmed to fail without the fix (and pass with it).Fixture provenance follows the convention already in
tests/rar3.rs: the bytes are the output of an unrelated proprietary encoder, not part of libarchive's source.No regressions — rar3/rar5/rar2/ppmd suites and the #122 differential fixtures all pass.
cargo test --all-features: 1769 pass, 0 fail; fmt and clippy clean.The second observation in the issue
The reporter also notes (explicitly "not a bug") that
rar5::Decoderdecodes a whole block perdecodecall andrar3::Decoderis buffer-then-drain, so a caller cannot bound per-call memory except viawith_unpack_size. That is accurate and is a real ergonomic limit for very large members. It is a design change to the streaming contract rather than a fix, so I have left it out of this PR; worth its own issue if you want it pursued.