Tracks wip - #380
Merged
Merged
Conversation
Deliberately not exported and deliberately not used for tonight's run. It is here
so the work and the measurements behind it are not lost, and so the next session
starts from the bug rather than from the idea.
WHAT IT IS FOR. mg_motion is built for a clip and for a person looking at the
result; on 120 s of 1080p the cost decomposes as 245 s for motion_analysis='all'
with motiongrams, 215 s for qom with motiongrams, and 62 s for qom without. The
motiongrams are 71 per cent and the area of motion another 12. This converts each
motion frame to greyscale ONCE and takes qom and both videogram columns from it,
writes into preallocated memmaps so nothing reallocates, and stores the videogram
as a pyramid of extremes so a two-hour session can be zoomed like an audio
waveform. Serially it ran 120 s of video in 60 s; over 8 processes, 27 s.
WHY IT IS NOT TRUSTED. Its qom differs from mg_motion's, and worse, it differs
BETWEEN IDENTICAL RUNS OF ITSELF. Non-determinism means frames are being
misaligned somewhere in how this drives ffmpeg_cmd, not that a parameter is
wrong. Two real bugs were found and fixed on the way and neither was the cause:
- `-t` was placed between the video input and the `color=` inputs that
filter_frame_ffmpeg appends, so it bound to one of THOSE inputs instead of
the output, changing which frames the filter saw.
- pipe reads were not looped to a full frame; a short read slides every later
frame across the boundary.
The same two commands run through plain subprocess.Popen produce identical
frames, so the fault is on this side, not ffmpeg's.
WHAT WAS MEASURED AND REJECTED, so nobody repeats it: ffmpeg emitting gray is
slower than rgb24 because the chain is RGB-native; signalstats on MGT's own chain
gives r=0.90 against QomRaw because the threshold filter's floor adds a constant;
scale=1:H:flags=area inside ffmpeg takes 106 s against 60 s; packet size from the
bitstream is free but correlates only r=0.33; and gray+tblend+signalstats takes
118 s for r=0.94. ffmpeg wins at decoding and filtering and loses at reductions.
Next step is a test that fails on the current code: extract two overlapping
ranges and assert the shared frames are equal, which the non-determinism will
break immediately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg1f739wddu5M4s3UwdNkn
ffmpeg_cmd(pipe="read") appends its own output arguments --- -f image2pipe -pix_fmt bgr24 -vcodec rawvideo - --- and this module had already appended its own. ffmpeg was given two outputs and wrote BOTH into the same stdout, interleaved, so the frames were wrong and, because interleaving depends on buffering, different between identical runs. mg_motion works because it stops at -filter_complex and lets ffmpeg_cmd finish the command. SERIAL IS NOW EXACT: byte-for-byte equal to mg_motion's QomRaw over 6005 frames of 1080p. Two earlier fixes on the way --- a -t binding to a color= input rather than the output, and pipe reads not looped to a whole frame --- were real bugs but not this one. THE PARALLEL PATH IS STILL NOT EXACT and is not to be used. On 1920x1080 at 50 fps with eight workers and 15 s chunks it repeats one frame at the LAST chunk seam: -ss before -i seeks to a keyframe, so the frames decoded before the target are not always the single frame the worker drops. All seven interior seams are exact; one frame in 6005 is not. tests/test_tracks.py asserts agreement with mg_motion and determinism, both of which fail on the old code. Its third test, parallel against serial, PASSES on a small synthetic clip and does NOT reproduce the seam artefact --- so it says so in its own docstring rather than reading like cover. It was first written as a strict xfail and XPASSed, which is how that was discovered. mypy clean across 71 files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xg1f739wddu5M4s3UwdNkn
…seek Chunks were seeked with -ss and then had exactly one frame dropped as the difference filter's lead-in. -ss before -i lands on a keyframe, so the frames arriving before the target are not always one, and a chunk could repeat its predecessor's value --- one wrong frame in 6,005 on 1080p/50 fps. Fixed structurally rather than by adjusting the count: seek a whole second early, then let ffmpeg's `trim` keep the wanted range BY TIMESTAMP. There is no count left to get wrong. Parallel output is now byte-identical to serial AND to mg_motion's QomRaw across all 6,005 frames of the 120 s 1080p clip. Cost of the lead-in is one second of decode per chunk: 6.7 per cent at the 15 s chunks used for testing, under 0.2 per cent at the 600 s chunks a real session would use. THE TEST FOR THIS DOES NOT COVER IT, AND SAYS SO. Written twice --- once with a default GOP, once with keyframes forced off the chunk grid --- and both versions pass against the broken code. The artefact needs the real material to appear, and a 1080p 120 s fixture does not belong in a unit suite. Rather than leave a test whose name implies coverage it does not have, the docstring states what it does guard, what it does not, and that the real protection is structural: if the worker ever goes back to counting frames after a seek, this test will not notice. 722 tests pass, mypy clean across 71 files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xg1f739wddu5M4s3UwdNkn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.