Skip to content

fix(xcresult): keep objects open when a property cannot be modelled - #1162

Merged
trunk-io[bot] merged 1 commit into
claude/xcresult-attribution-fixturesfrom
claude/xcresult-schema-unmodellable-properties
Aug 19, 2026
Merged

fix(xcresult): keep objects open when a property cannot be modelled#1162
trunk-io[bot] merged 1 commit into
claude/xcresult-attribution-fixturesfrom
claude/xcresult-schema-unmodellable-properties

Conversation

@dfrankland

@dfrankland dfrankland commented Aug 18, 2026

Copy link
Copy Markdown
Member

Stack 4/4 — depends on #1160.

Apple declares SortedKeyValueArrayPair as key: String and value: SchemaSerializable. SchemaSerializable is never defined in the format description, so the generator skips the property — but still emitted additionalProperties: false, which typify turns into deny_unknown_fields. The object then claims to hold exactly _type and key while every real payload also carries value:

failed to parse json from xcresulttool output:
unknown field `value`, expected `key` or `_type`

Why this matters more than it looks

That array holds attachment metadata, so any test summary with an attachment is unparseable. find_file_in_test_summary swallows the error and returns None, which means --use-experimental-failure-summary has silently been a no-op for every UI test bundle. Attribution falls through to the workspace document location — the one source that points at wherever the failure surfaced rather than at the test.

Confirmed on a customer UI test bundle from this thread, whose single failure is a
UI test with attachments. This is the difference between:

reported file
before a path under .../Tuist/.build/checkouts/<dependency>/...
with #1159 only (none — the vendored path is correctly rejected, but the call stack is never read)
with this PR the test's own source file, from its call-stack frame

The test's own frame was in the call stack the whole time; we just never got far enough to look at it.

The change

The generator tracks whether it had to drop a property and leaves those objects open, inheriting the flag through sub-types. It stays off everywhere else, because deny_unknown_fields is what keeps the untagged oneOf sub-type unions apart — I tried relaxing it wholesale first and ActionTestSummaryIdentifiableObject resolved to the wrong variant, losing summaryRef entirely. Four objects change: SortedKeyValueArrayPair and three ActivityLog*Sections.

Regenerating under a newer Xcode also picks up unrelated format drift (~21 types), so only the rule this changes is applied to the checked-in schema — a full regeneration belongs in its own change.

build.rs now declares rerun-if-changed for the schemas. Without it an edit to a schema appears to do nothing until something else forces the build script to rerun.

Tests

A regression test deserializes a summary carrying SortedKeyValueArrayPair; it fails on main with exactly the production error and passes here. Pure JSON, no macOS required.

🤖 Generated with Claude Code

`SortedKeyValueArrayPair` is declared by Apple as `key: String` and
`value: SchemaSerializable`. `SchemaSerializable` is never defined in the format
description, so the generator skips the property — but still emitted
`additionalProperties: false`, which typify turns into `deny_unknown_fields`. The
object then claims to hold exactly `_type` and `key` while every payload also
carries `value`, and the whole enclosing object fails to deserialize:

    failed to parse json from xcresulttool output:
    unknown field `value`, expected `key` or `_type`

That array holds attachment metadata, so any test summary with an attachment is
unparseable. `find_file_in_test_summary` swallows the error and returns `None`,
which means `--use-experimental-failure-summary` has silently been a no-op for
every UI test bundle: attribution falls through to the workspace document
location, the one source that points at wherever the failure surfaced. On a real
UI test bundle this is the difference between reporting a vendored page object
and reporting the test's own file.

The generator now tracks whether it had to drop a property and leaves those
objects open, inheriting the flag through sub-types. It stays off everywhere
else, because `deny_unknown_fields` is what keeps the untagged `oneOf` sub-type
unions apart — relaxing it wholesale makes `ActionTestSummaryIdentifiableObject`
resolve to the wrong variant and loses `summaryRef` entirely. Four objects
change: `SortedKeyValueArrayPair` and three `ActivityLog*Section`s.

Regenerating under a newer Xcode also picks up unrelated format drift, so only
the rule this changes is applied to the checked-in schema; a full regeneration
belongs in its own change.

`build.rs` now declares `rerun-if-changed` for the schemas. Without it an edit
appears to do nothing until something else forces the build script to rerun,
which is a confusing way to lose an afternoon.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dfrankland
dfrankland force-pushed the claude/xcresult-schema-unmodellable-properties branch from fca29c6 to 16a1392 Compare August 18, 2026 23:32
@trunk-staging-io

trunk-staging-io Bot commented Aug 18, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

Failed Test Failure Summary Logs
pending_quarantine_test should be quarantined when run with variant A test marked as pending was expected to fail but unexpectedly passed. Logs ↗︎
variant_quarantine_test should be quarantined when run with variant A test expected the sum of 2 + 2 to be 5, but it was actually 4, indicating a failing assertion. Logs ↗︎

View Full Report ↗︎Docs

@trunk-io

trunk-io Bot commented Aug 18, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@codecov-commenter

codecov-commenter commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.37%. Comparing base (4cff9e3) to head (16a1392).

Additional details and impacted files
@@                           Coverage Diff                            @@
##           claude/xcresult-attribution-fixtures    #1162      +/-   ##
========================================================================
+ Coverage                                 83.34%   83.37%   +0.02%     
========================================================================
  Files                                        72       72              
  Lines                                     16180    16197      +17     
========================================================================
+ Hits                                      13486    13504      +18     
+ Misses                                     2694     2693       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dfrankland

Copy link
Copy Markdown
Member Author

/trunk merge

@trunk-io

trunk-io Bot commented Aug 19, 2026

Copy link
Copy Markdown

😎 Stack merged successfully - details.

@trunk-io

trunk-io Bot commented Aug 19, 2026

Copy link
Copy Markdown

This PR is already queued as a stacked merge. Cancel it first to re-submit.

@trunk-io
trunk-io Bot merged commit c1423be into main Aug 19, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants