Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 82 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1657,8 +1657,89 @@ value here shows up as "this GPU doesn't work" rather than as an error:
(a later `-pix_fmt` wins), which is the escape hatch for 10-bit HEVC on
hardware that supports it.

- **A hardware encoder's declared pix_fmt list is not a statement about the
machine (issue #74).** The list ffmpeg negotiates against is compiled in;
NVENC's real capabilities are queried from the driver at `avcodec_open2`. A
recent ffmpeg built against NVENC SDK 13 advertises `yuv422p` on
`h264_nvenc` for Blackwell's 4:2:2 support, so negotiation picks it for any
4:2:2 source — and every pre-Blackwell card then fails the job outright with
*"YUV422P not supported / No capable devices found"*, zero frames written.
Reported on an RTX 4070 Super with a CineForm `yuv422p10le` capture at the
default "Match source" colour format. **ffmpeg cannot negotiate its way out
of this**, so `forced_pix_fmt` picks the format instead.

`VideoCodec::forced_pix_fmt` therefore takes the format the pipeline will
actually hand the encoder — `VideoJob::encoder_input_pix_fmt`, which is the
output conversion when one is selected and the pipe format otherwise. Three
things about it are load-bearing:

- **The NVENC/QSV arm is conditional and the other two are not.** HuffYUV and
AMF take one format whatever the source was; NVENC does not. Pinning it
unconditionally the way AMF is pinned would flatten a 10-bit 4:2:0 source to
8-bit for everyone it already serves correctly, so the guard fires only on
4:2:2, 4:4:4, or >8-bit into an H.264 encoder (neither family has a 10-bit
H.264 mode). A 4:2:0 job emits no `-pix_fmt` at all, exactly as before.
- **HEVC keeps the depth, H.264 cannot.** 4:2:2 10-bit into `hevc_nvenc`
becomes `p010le`, not `yuv420p` — only the chroma has to go. NVENC gets the
planar `yuv420p` and QSV the semi-planar `nv12`, each family's native name.
- **VideoToolbox is deliberately excluded.** It never advertises a mode it
lacks, so its negotiation is trustworthy and forcing a format would only
throw away chroma it could have kept. QSV *is* included, preventively rather
than on a report: `hevc_qsv` advertises `y210le` on builds whose hardware may
not have it, which is the same trap.

None of the AMF behaviour can be verified in CI or on macOS — there is no AMD
hardware in the matrix — so changes here rest on reporter confirmation.
hardware in the matrix — and the same is true of NVENC and QSV, so these rest
on unit tests over the emitted arguments plus reporter confirmation. Note the
functional probe in `HardwareEncoderDetector` cannot catch the #74 class at
all: it encodes one `yuv420p` frame from lavfi, so it correctly reports NVENC
as *available* — the device works, only the format doesn't. Don't try to fix a
format problem in the device probe; the format isn't known until a file loads.

### The user-facing half of #74

The guard above keeps the job running, but silently changing someone's output
is only acceptable if they can see it coming and choose otherwise. Two things
shipped with it:

**A 4:2:0 10-bit output format.** `ChromaSubsampling::Yuv420P10` /
`ChromaSubsampling.yuv420p10` — the only 10-bit layout NVENC, QSV and AMF can
encode, so it is how a 10-bit source keeps its grading through a GPU encoder by
the user's own choice rather than by the guard's fallback. Verified end to end
(`integration_high_bit_depth_filters_test`): a 10-bit 4:2:2 source comes out
`yuv420p10le`, profile **High 10**. Adding an option touches four places —
`vapoursynth_format`, `ffmpeg_pix_fmt`, the Dart enum with its `outputBitDepth`,
and `chromaFormatHelpSections`, which `settings_chroma_help_test` fails on if
the new label goes unmentioned.

**A warning under the dropdown.** `hardwareEncoderChromaWarning` in
`app/lib/utils/pixel_format.dart` says which format will be substituted and
why, before the job runs. It covers both routes into #74 — a 4:2:2 *source* at
"Match source", and an explicitly chosen 4:2:2 *output* — and it stays silent
for everything encodable, for VideoToolbox and AMF, and until a file is loaded.

> **It is a second implementation of the worker's decision, and that is the
> risk.** If the two disagree the interface promises one thing and the encode
> does another, which is worse than either being wrong alone. Both sides are
> therefore pinned to **the same table of cases** — `substitutions match the
> worker, case for case` in `hardware_encoder_chroma_warning_test.dart` against
> `test_nvenc_cannot_be_handed_422` and its neighbours in `video_job.rs`. Change
> one and change both. `pixelFormatChromaLayout` is likewise a coarse Dart twin
> of `ChromaClass`; the Dart side already reimplements this kind of pix_fmt
> parsing in `pixelFormatBitDepth`, so it follows that precedent rather than
> inventing a new one.

**A second help dialog**, `ColourPipelineHelpIcon` /
`colourPipelineHelpSections`, beside the existing `ChromaFormatHelpIcon`. The
two answer different questions and both are worth having: the first is *what
are these formats and which do I pick*, the second is *what does the app do to
my colour* — the pipe source normalising upward on the way in, filters
converting down and back per pass, every UI threshold being in 8-bit units and
rescaled to the clip depth, the output conversion dithering, the Y4M pipe
stripping SAR and colour tags so they must be re-stamped, and the encoder
having the last word. A deliberately distinct icon (`schema_outlined`, not a
second `info_outline`), asserted, because two identical adjacent buttons read
as one control repeated.

## QTGMC Parameters Reference

Expand Down
4 changes: 4 additions & 0 deletions app/lib/models/encoding_settings.dart
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ enum ChromaSubsampling {
original('original', 'Match source', null, null),
/// Convert to 8-bit YUV420 for maximum compatibility (smaller files).
yuv420('yuv420', '4:2:0 8-bit', 'most compatible', 8),
/// Convert to 10-bit YUV420. The only 10-bit layout NVENC, QSV and AMF can
/// encode, so it is the way to keep a 10-bit source's grading on a GPU
/// encoder — 4:2:2 fails outright on most of them (issue #74).
yuv420p10('yuv420p10', '4:2:0 10-bit', 'best 10-bit for GPU encoders', 10),
/// Convert to 8-bit YUV422 for higher chroma quality.
yuv422('yuv422', '4:2:2 8-bit', 'more colour detail', 8),
/// Convert to 10-bit YUV422: keeps a 10-bit source's precision while
Expand Down
159 changes: 159 additions & 0 deletions app/lib/utils/pixel_format.dart
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
// Utilities for interpreting FFmpeg pixel-format strings (e.g. the `pix_fmt`
// reported by ffprobe: "yuv420p", "yuv422p10le", "yuv420p16le").

import '../models/encoding_settings.dart';
import '../models/video_job.dart';

/// Best-effort per-component bit depth for an FFmpeg pixel-format string.
///
/// Returns 8 for the common 8-bit formats (yuv420p, yuv422p, nv12, rgb24, …),
Expand Down Expand Up @@ -71,3 +74,159 @@ String? chromaConversionBitDepthWarning({
'colour format. Choose "Match source" to keep the source\'s bit depth'
'${targetBitDepth < 10 ? ', or 4:2:2 10-bit to keep more of it' : ''}.';
}

/// Chroma layout of an FFmpeg pixel-format string, coarse enough for the one
/// question the UI asks of it: can a hardware encoder take this?
///
/// Mirrors `ChromaClass` in `worker/src/pixel_format.rs`, which is the authority
/// — it decides what the worker actually does. This copy only decides what the
/// warning *says*, and `hardware_encoder_chroma_warning_test.dart` pins the two
/// to the same table so they cannot drift into disagreeing on screen.
enum ChromaLayout {
/// 4:2:0 — one colour sample per 2x2 block.
c420,

/// 4:2:2 and anything hardware treats like it (4:4:0, 4:1:1, 4:1:0).
c422,

/// 4:4:4, and RGB, which carries full colour resolution by construction.
c444,
}

/// Best-effort chroma layout for an FFmpeg pixel-format string.
///
/// Unknown and null formats fall back to [ChromaLayout.c420] — the conservative
/// choice here, because 4:2:0 is what every encoder accepts, so an unrecognized
/// format produces no spurious warning. (`pixelFormatBitDepth` falls back the
/// same way and for the same reason.)
ChromaLayout pixelFormatChromaLayout(String? pixFmt) {
if (pixFmt == null || pixFmt.trim().isEmpty) return ChromaLayout.c420;
final f = pixFmt.trim().toLowerCase();

// Planar YUV: the three digits after the family prefix are the subsampling.
final planar = RegExp(r'^yuv[aj]?(\d{3})').firstMatch(f);
if (planar != null) {
switch (planar.group(1)!) {
case '420':
return ChromaLayout.c420;
// 4:1:0 and 4:1:1 subsample more coarsely than 4:2:2 horizontally and
// 4:4:0 more coarsely vertically, but none of them is 4:2:0, and no
// hardware encoder takes any of them — so they warn alongside 4:2:2.
case '410':
case '411':
case '422':
case '440':
return ChromaLayout.c422;
default:
return ChromaLayout.c444;
}
}

// Semi-planar: nv12/nv21 are 4:2:0, nv16 4:2:2, nv24/nv42 4:4:4; p010/p016
// are 4:2:0, p210/p216 4:2:2, p410/p416 4:4:4 (the middle digit is the
// subsampling, as in `pixelFormatBitDepth`).
const semiPlanar = {
'nv12': ChromaLayout.c420,
'nv21': ChromaLayout.c420,
'nv16': ChromaLayout.c422,
'nv24': ChromaLayout.c444,
'nv42': ChromaLayout.c444,
};
final stem = f.replaceFirst(RegExp(r'(le|be)$'), '');
final named = semiPlanar[stem];
if (named != null) return named;
final p = RegExp(r'^p(\d)(?:10|12|16)$').firstMatch(stem);
if (p != null) {
switch (p.group(1)!) {
case '0':
return ChromaLayout.c420;
case '2':
return ChromaLayout.c422;
default:
return ChromaLayout.c444;
}
}

// Gray has no chroma at all, so every encoder can hold it.
if (stem.startsWith('gray') || stem.startsWith('ya')) return ChromaLayout.c420;

// RGB and planar GBR carry full colour resolution.
if (RegExp(r'^(a?rgb|a?bgr|gbra?p)').hasMatch(stem)) return ChromaLayout.c444;

return ChromaLayout.c420;
}

/// Warning message when the chosen output colour format is one the selected
/// hardware encoder cannot take, and the worker will therefore substitute
/// another — or null when there is nothing to say.
///
/// This is the UI half of issue #74. An encoder's declared format list is
/// compiled into ffmpeg, but NVENC's real capabilities are queried from the
/// driver at open time: a recent ffmpeg advertises `yuv422p` on `h264_nvenc`
/// for Blackwell's 4:2:2 support, and every earlier card then failed the whole
/// job. The worker now substitutes an encodable format instead of failing, so
/// this exists to say so *before* the job runs rather than only in its log.
///
/// **Keep in step with `VideoCodec::forced_pix_fmt` in
/// `worker/src/models/video_job.rs`**, which is what actually happens. The two
/// share a table of cases in their tests.
///
/// [chromaSubsampling] decides the format outright unless it is
/// [ChromaSubsampling.original], in which case the source's [pixelFormat] does.
String? hardwareEncoderChromaWarning({
required VideoCodec codec,
required ChromaSubsampling chromaSubsampling,
String? pixelFormat,
}) {
// Only NVENC and QSV advertise formats their hardware may not have.
// VideoToolbox never does, so its negotiation is trustworthy; AMF is pinned
// to nv12 unconditionally and has been since issue #51, so it converts
// whatever it is given and there is no surprise to warn about.
final isNvenc = codec.isNvenc;
final isQsv = codec == VideoCodec.h264Qsv || codec == VideoCodec.h265Qsv;
if (!isNvenc && !isQsv) return null;

// What the encoder will actually be handed: the output conversion when one is
// selected, the source's own format otherwise.
final ChromaLayout layout;
final int depth;
final String describedAs;
switch (chromaSubsampling) {
case ChromaSubsampling.original:
// Nothing to warn about until a file is loaded and we know its format.
if (pixelFormat == null) return null;
layout = pixelFormatChromaLayout(pixelFormat);
depth = pixelFormatBitDepth(pixelFormat);
describedAs = 'Your source is $pixelFormat, and "Match source" keeps it';
default:
layout = chromaSubsampling == ChromaSubsampling.yuv420 ||
chromaSubsampling == ChromaSubsampling.yuv420p10
? ChromaLayout.c420
: ChromaLayout.c422;
depth = chromaSubsampling.outputBitDepth ?? 8;
describedAs = '${chromaSubsampling.label} is selected';
}

final chromaUnsupported = layout != ChromaLayout.c420;
// Neither family has a 10-bit H.264 mode at all.
final depthUnsupported = depth > 8 && codec.isH264;
if (!chromaUnsupported && !depthUnsupported) return null;

// Mirrors forced_pix_fmt: H.264 has only 8-bit 4:2:0; HEVC keeps the depth.
final substitute = codec.isH264
? '4:2:0 8-bit'
: (depth > 8 ? '4:2:0 10-bit' : '4:2:0 8-bit');

final reason = chromaUnsupported
? '${codec.encoderFamily} cannot encode ${layout == ChromaLayout.c444 ? "4:4:4" : "4:2:2"} '
'on most GPUs'
: '${codec.encoderFamily} has no 10-bit H.264 mode';

final advice = codec.isH264 && depth > 8
? ' Choose an H.265 encoder to keep the 10-bit grading, or a software '
'encoder to keep the colour detail as well.'
: ' Choose a software encoder to keep it as it is.';

return '$describedAs, but $reason, so the output will be converted to '
'$substitute.$advice';
}
Loading
Loading