Skip to content

fix(rar3): reset the low-distance repeat state at each LZ block header - #130

Merged
MagicalTux merged 1 commit into
masterfrom
fix/rar3-lowdist-reset
Sep 4, 2026
Merged

fix(rar3): reset the low-distance repeat state at each LZ block header#130
MagicalTux merged 1 commit into
masterfrom
fix/rar3-lowdist-reset

Conversation

@MagicalTux

Copy link
Copy Markdown
Member

Fixes #126.

Confirmed

Reproduced exactly as reported, from libarchive's test_read_format_rar3_lowdist_reset.rar (member lowdist-reset.bin, method 0x35, version 29, not solid, 490 packed → 64 unpacked):

crc(got)      = 0x05d56889   <- matches the reporter's figure
crc(expected) = 0x6ff838dc   <- the archive's stored FILE_CRC

I verified the expected plaintext independently rather than taking it on trust: the 64 bytes in the issue CRC to exactly the 0x6FF838DC stored in the archive's own file header, so they are a property of the archive, not of libarchive's reader.

One correction to the report: the divergence begins at offset 46, not 40. The issue's prose says 40 but its own hex dump is consistent with 46.

Cause

The member is packed as two LZ blocks. parse_block_header carried last_low_offset and num_low_offset_repeats across the block boundary, so the first low-distance match in the second block added a stale addend and everything after it was wrong.

The low-distance repeat state belongs to the table domain of the block that established it. unrar and libarchive both clear it on every LZ block header, and — the part that matters — before the keep-table flag is read, so a block reusing the previous code lengths still starts a fresh low-distance run. libarchive's parse_codes places it exactly there:

/*
 * Low-distance repeat state belongs to the current LZ table and
 * must not be reused after starting a new table.
 */
rar->lastlowoffset = 0;
rar->numlowoffsetrepeats = 0;

The fix is those two lines at the matching point in parse_block_header.

Why this one is worth flagging

The failure mode is wrong bytes at the correct length — the shape a size check cannot catch. A caller verifying FILE_CRC does catch it (as the reporter's did), so this is a correctness gap rather than a safety one, but any consumer that trusts length alone would have silently propagated corrupt data.

Tests

tests/fixtures/rar3/lowdist_reset.bin (the 490-byte packed run) plus two tests: single-shot and byte-at-a-time, the latter so the block boundary carrying the reset lands mid-call. Both assert against the archive's stored FILE_CRC as well as the literal bytes. Both were confirmed to fail without the fix (and pass with it).

Fixture provenance follows the convention already in tests/rar3.rs: the bytes are the output of an unrelated proprietary encoder, not part of libarchive's source.

No regressions — rar3/rar5/rar2/ppmd suites and the #122 differential fixtures all pass. cargo test --all-features: 1769 pass, 0 fail; fmt and clippy clean.

The second observation in the issue

The reporter also notes (explicitly "not a bug") that rar5::Decoder decodes a whole block per decode call and rar3::Decoder is buffer-then-drain, so a caller cannot bound per-call memory except via with_unpack_size. That is accurate and is a real ergonomic limit for very large members. It is a design change to the streaming contract rather than a fix, so I have left it out of this PR; worth its own issue if you want it pursued.

Fixes #126.

libarchive's `test_read_format_rar3_lowdist_reset.rar` decoded to the right
length with the wrong contents from offset 46 on. The member is packed as
two LZ blocks; `parse_block_header` carried `last_low_offset` and
`num_low_offset_repeats` across the block boundary, so the first
low-distance match of the second block added a stale addend and every byte
after it was wrong.

The low-distance repeat state belongs to the table domain of the block that
established it. unrar and libarchive both clear it on every LZ block header
— crucially *before* the keep-table flag is read, so a block that reuses the
previous code lengths still starts a fresh low-distance run. libarchive's
`parse_codes` puts it exactly there: "Low-distance repeat state belongs to
the current LZ table and must not be reused after starting a new table."

The failure mode is the bad kind: wrong bytes at the correct length, which a
size check cannot catch. A caller verifying the archive's FILE_CRC does
catch it (as the reporter's did), so this is a correctness gap rather than a
safety one.

The regression test asserts against the archive's own stored FILE_CRC
(0x6FF838DC), which makes the expected plaintext verifiable independently of
any other reader — the 64 expected bytes CRC to exactly that value. Both the
single-shot and byte-at-a-time paths are covered, the latter so the block
boundary carrying the reset lands mid-call. Both tests were confirmed to
fail without the fix.

No regressions: the rar3/rar5/rar2/ppmd suites and the differential fixtures
from #122 all still pass (1769 total, 0 failed).
@MagicalTux
MagicalTux merged commit 4cf822d into master Sep 4, 2026
46 checks passed
@MagicalTux MagicalTux mentioned this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rar3: libarchive's test_read_format_rar3_lowdist_reset.rar decodes to wrong bytes after offset 40

1 participant