Skip to content

Separate required HTML validation outcomes from detailed reports - #1627

Merged
chubes4 merged 3 commits into
trunkfrom
refactor/1360-required-validation-outcomes
Sep 9, 2026
Merged

Separate required HTML validation outcomes from detailed reports#1627
chubes4 merged 3 commits into
trunkfrom
refactor/1360-required-validation-outcomes

Conversation

@chubes4

@chubes4 chubes4 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Continue #1360 by separating required HTML validation facts from detailed report maps at the diagnostics-consumer boundary.

  • HtmlCompilation evaluates the existing validators once and constructs HtmlValidationOutcome in the producer-owned BlockCompilationOutput.
  • DiagnosticsCollector consumes the compact outcome rather than full block-validity, semantic-parity and content-round-trip reports.
  • Keep only original finding keys used by diagnostics, preserving mixed values and leaving existing fallback messages/defaults with the collector.
  • Uncomputed outcomes explicitly say not_evaluated, not pass. Successfully parsed empty documents still evaluate validators normally.
  • Preserve full reports and conversion-report projections, canonical serialization, status/acceptance policy and all block/artifact capabilities.

There is no reduced-evidence mode, skipped validator or performance claim in this bite. Detailed reports are still computed; this establishes the consumer boundary needed before a later evidence policy. The internal outcome is not added to the public serialized result.

The required/detail consumer inventory is documented in php-transformer/docs/html-validation-outcome-consumer-inventory.md.

Verification

Baseline: c302eaaa26681462a784c0d341f2702ce7d464e7.
Candidate: b36079ffb337b3518a93fc3d6b8a54cb27907ac8.

  • All 385 discovered HTML fixtures retain identical serialized-block hashes and complete result envelopes after excluding only recursive transform_duration_ms. PHP serialization preserves binary bytes and array ordering.
  • Nine targeted HTML/artifact cases also match, including nonzero companion, runtime, stylesheet, shared-shell and responsive-counterpart outputs.
  • Coordinator independently compared the complete baseline/candidate evidence files; both have SHA-256 a3a2c5c327cace26a2a69aaa52d9e81fbf7e2fef9134259678700c8517d28d56.
  • Direct diagnostic assertions cover failure findings for all three validators, missing/null summary fallbacks, mixed severity/location values and uncomputed outcomes.
  • Full composer test passed end-to-end in one PHP 8.4 container with a 512 MB PHP configuration inherited by child processes: canonical/unit tests, 303 parity fixtures, dist shape and package-install proof.
  • Baseline and candidate used the same task-local Composer lock for matching dependencies; the repository does not commit a lock.
  • WordPress integration was skipped locally because WP_TESTS_DIR was unavailable. Actual CI integration and solved-site gates remain required.

Reproduce

From php-transformer, with dependencies installed and PHP memory configured for child processes:

composer test
REQUIRE_WP_TESTS=1 WP_TESTS_DIR=/path/to/wordpress-tests composer test:wordpress-integration

For corpus comparison, run this from both repository roots with matching dependencies and compare the resulting output:

php -d memory_limit=512M -r '
require "php-transformer/vendor/autoload.php";
$strip = function (array &$value) use (&$strip): void {
    unset($value["transform_duration_ms"]);
    foreach ($value as &$item) if (is_array($item)) $strip($item);
};
$rows = [];
foreach (new RecursiveIteratorIterator(new RecursiveDirectoryIterator("fixtures", FilesystemIterator::SKIP_DOTS)) as $file) {
    if (!$file->isFile() || strtolower($file->getExtension()) !== "html") continue;
    $result = (new Automattic\BlocksEngine\PhpTransformer\HtmlToBlocks\HtmlTransformer())->transform(file_get_contents($file->getPathname()));
    $array = $result->toArray();
    $strip($array);
    $rows[$file->getPathname()] = [hash("sha256", $result->serializedBlocks), hash("sha256", serialize($array))];
}
ksort($rows, SORT_STRING);
echo json_encode($rows, JSON_UNESCAPED_SLASHES), PHP_EOL;
'

AI Assistance

OpenAI GPT-5.6 Terra via direct OpenCode mapped consumers, implemented the boundary and ran lab verification. GPT-6 Astra via OpenCode coordinated and reviewed the work, identified unevaluated-status and diagnostic-normalization problems, requested behavioral corrections and full-suite verification, independently checked comparison evidence, and prepared this PR under Chris Huber's direction. No release or deployment was performed.

@chubes4
chubes4 marked this pull request as ready for review September 9, 2026 16:51
@chubes4
chubes4 merged commit 6a50fba into trunk Sep 9, 2026
10 checks passed
@chubes4
chubes4 deleted the refactor/1360-required-validation-outcomes branch September 9, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant