test(xcresult): real bundles reproducing dependency misattribution

[dfrankland]

Stack 3/3 — depends on #1159.

The file sources have only synthesized-JSON unit tests behind them, and the one real SnapshotTesting bundle does not reproduce the bug: assertSnapshot defaults filePath to #filePath at the call site, so its fileName already points at the test's own source and the fixture passes with or without the fix.

tests/fixture-src/ holds five SwiftPM packages, one per shape, plus the script that captures and checks them. Each bundle is verified to exhibit its shape before it is packaged, since a fixture that stops reproducing keeps passing while guarding nothing.

Each test names the FileSource it exercises:

Scenario	Shape
dependency-raises-failure	every source but TestFrame is under SourcePackages/checkouts/
in-repo-helper-raises-failure	TestFrame versus RaisedFrom with nothing to separate them by path
crash-in-dependency	two tests that never reach their own frame: one fatalErrors inside the dependency, one is failed by the dependency's TestScoping trait after its body returned
objc-xctest	TestIdentity::is_named_by against real -[Suite testCase] symbolication
toplevel-swift-testing	a suiteless @Test func, matched by bare name

Reverting the fix fails seven of the ten new cases, on exactly the paths it was written to stop reporting.

Why the two cases expect different JUnit

Only the experimental path reads the per-test failure summary, and so the call stack, so it is the only one that can produce a TestFrame candidate. The legacy path sees DocumentLocation alone. Snapshotting a shared expectation would hide that; two expectations state it.

Bundle size

Xcode 26 writes ~95MB of unreferenced dyld shared-cache symbolication data into every result bundle, which would have made these 96MB apiece. prune-bundle.py walks the object graph from the root and keeps only what it reaches — 9–11 objects out of ~1570 files, 96MB → 84KB — and regeneration fails if either xcresulttool API we call returns anything different afterwards. Total added: 168KB.

Regenerating

tests/fixture-src/regenerate.sh (macOS + Xcode). See that directory's README for what each bundle must exhibit, and the two things that are easy to get wrong: the dependency has to be a git repository to be checked out into SourcePackages/checkouts, and absolute paths are baked in at capture time.

🤖 Generated with Claude Code

[trunk-staging-io]

   

Failed Test	Failure Summary	Logs
pending_quarantine_test should be quarantined when run with variant	A test marked as pending was expected to fail but unexpectedly passed.	Logs ↗︎
variant_quarantine_test should be quarantined when run with variant	A test expected the sum of 2 + 2 to be 5, but it was actually 4, indicating a failing assertion.	Logs ↗︎

View Full Report ↗︎ ⋅ Docs

[trunk-io]

   

View Full Report ↗︎ ⋅ Docs

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.34%. Comparing base (52b4923) to head (4cff9e3).

Additional details and impacted files

@@                            Coverage Diff                             @@
##           claude/xcresult-test-frame-attribution    #1160      +/-   ##
==========================================================================
+ Coverage                                   80.29%   83.34%   +3.05%     
==========================================================================
  Files                                          72       72              
  Lines                                       16081    16180      +99     
==========================================================================
+ Hits                                        12912    13486     +574     
+ Misses                                       3169     2694     -475     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[TylerJang27]

🙌

[trunk-io]

This pull request was merged into main as part of stacked PR 1162.