trunk-merge/pr-1162/ea28c3a1-4aad-4feb-8bd9-73af55499ab0 - #1164
Closed
trunk-io[bot] wants to merge 5 commits into
Closed
trunk-merge/pr-1162/ea28c3a1-4aad-4feb-8bd9-73af55499ab0#1164trunk-io[bot] wants to merge 5 commits into
trunk-io[bot] wants to merge 5 commits into
Conversation
The file we report for a failed test was assembled by an `.or_else` chain of `Option<String>`, where every link meant something different and none of them said so. `fileName` is the site an assertion fired, which for a helper is the helper's file; the call-stack fallback is whichever frame happened to be outermost. Reading the chain, they are indistinguishable. `FileSource` names each place we look and `FileCandidate` carries it alongside the path, so a candidate can be traced back to where it came from instead of arriving as an anonymous string. The extraction for each source moves into `file_attribution` as its own named function, and the module doc says the thing the code could not: an `.xcresult` records where a failure was *raised*, never where a test is *declared*, so every source here is a proxy for the question we are actually asking. `ReportedPath` normalizes once at construction. It was previously applied by hand at each call site, which is a silent bug the moment one is missed or two paths are compared having been normalized differently. No behaviour change: the sources are offered in the order the chain tried them and the first is taken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…endency's The file we report for a test is whatever the failure summary points at, and for a failure raised inside a helper that is the helper's file: a snapshot trait, a mocking framework, a page object, a launch helper. The path lands under the package checkout (`Tuist/.build/checkouts/...`, `DerivedData/SourcePackages/checkouts/...`), and since codeowners are resolved from that path the test is then owned by whoever owns the vendored directory. Adds `FileSource::TestFrame`, the frame whose symbol names the test. Frames run innermost first, so the test's own frame sits in the middle of the stack — helpers it called below it, the framework that invoked it above — which is why taking the last frame lands on a dependency. `TestIdentity` owns the spellings a frame can use: Swift (`Suite.testCase()`), Objective-C (`-[Suite testCase]`), closures declared inside the test, and top-level swift-testing functions, which have no suite. That frame is the only source that identifies the test rather than the failure, which `FileSource::is_positive_identification` states. Everything else is vetted against `ReportedPath::is_vendored_dependency` in a single `find`, so a source added later cannot quietly skip the check — previously the same filter was repeated at each link of the chain. The stack fallback now yields its frames outermost-first instead of collapsing to a single "last frame", so rejecting one lands on the next frame out rather than giving up. When a test crashes or fails to launch it never reaches its own frame and every source points into a dependency; we then report no file at all rather than one that would re-own the test. Consumers already treat a missing file as "unchanged" rather than "cleared", so the test keeps the path and owners it last had. Note this also changes attribution for a failure raised inside an in-repo helper: the test's own file now wins over the helper's. That is the intended reading of "the file of the test case". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The file sources have only synthesized-JSON unit tests behind them, and the one real SnapshotTesting bundle does not reproduce the bug: `assertSnapshot` defaults `filePath` to `#filePath` at the call site, so its `fileName` already points at the test's own source and the fixture passes with or without the fix. `tests/fixture-src/` holds five SwiftPM packages, one per shape, plus the script that captures and checks them. Each bundle is verified to exhibit its shape before it is packaged, since a fixture that stops reproducing keeps passing while guarding nothing. Each test names the `FileSource` it exercises: - `dependency-raises-failure` every source but `TestFrame` is under `SourcePackages/checkouts/` - `in-repo-helper-raises-failure` `TestFrame` versus `RaisedFrom` with nothing to separate them by path - `crash-in-dependency` two tests that never reach their own frame: one `fatalError`s inside the dependency, one is failed by the dependency's `TestScoping` trait after its body returned - `objc-xctest` `TestIdentity::is_named_by` against real `-[Suite testCase]` symbolication - `toplevel-swift-testing` a suiteless `@Test func`, matched by bare name Reverting the fix fails seven of the ten new cases, on exactly the paths it was written to stop reporting. The experimental and legacy cases expect different JUnit, which is the honest result rather than a shared snapshot: only the experimental path reads the failure summary, so it is the only one that can produce a `TestFrame` candidate. Bundles are pruned before packaging. Xcode 26 writes ~95MB of unreferenced dyld shared-cache symbolication data into every result bundle, which would have made these 96MB apiece; `prune-bundle.py` keeps only what the object graph reaches (9-11 objects out of ~1570 files) and regeneration fails if either xcresulttool API we call returns anything different afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`SortedKeyValueArrayPair` is declared by Apple as `key: String` and
`value: SchemaSerializable`. `SchemaSerializable` is never defined in the format
description, so the generator skips the property — but still emitted
`additionalProperties: false`, which typify turns into `deny_unknown_fields`. The
object then claims to hold exactly `_type` and `key` while every payload also
carries `value`, and the whole enclosing object fails to deserialize:
failed to parse json from xcresulttool output:
unknown field `value`, expected `key` or `_type`
That array holds attachment metadata, so any test summary with an attachment is
unparseable. `find_file_in_test_summary` swallows the error and returns `None`,
which means `--use-experimental-failure-summary` has silently been a no-op for
every UI test bundle: attribution falls through to the workspace document
location, the one source that points at wherever the failure surfaced. On a real
UI test bundle this is the difference between reporting a vendored page object
and reporting the test's own file.
The generator now tracks whether it had to drop a property and leaves those
objects open, inheriting the flag through sub-types. It stays off everywhere
else, because `deny_unknown_fields` is what keeps the untagged `oneOf` sub-type
unions apart — relaxing it wholesale makes `ActionTestSummaryIdentifiableObject`
resolve to the wrong variant and loses `summaryRef` entirely. Four objects
change: `SortedKeyValueArrayPair` and three `ActivityLog*Section`s.
Regenerating under a newer Xcode also picks up unrelated format drift, so only
the rule this changes is applied to the checked-in schema; a full regeneration
belongs in its own change.
`build.rs` now declares `rerun-if-changed` for the schemas. Without it an edit
appears to do nothing until something else forces the build script to rerun,
which is a confusing way to lose an afternoon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1164 +/- ##
==========================================
+ Coverage 82.97% 83.37% +0.40%
==========================================
Files 71 72 +1
Lines 16044 16197 +153
==========================================
+ Hits 13312 13504 +192
+ Misses 2732 2693 -39 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
|
trunk-io
Bot
deleted the
trunk-merge/pr-1162/ea28c3a1-4aad-4feb-8bd9-73af55499ab0
branch
August 19, 2026 22:43
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request was created and is being managed by Trunk Merge.
This pull request is based on the main branch at SHA 64052e551359bc46c18d93112041a4a6f82555b8.
See more details here.
When CI completes, this pull request will be closed automatically.
Pull Requests Being Tested
This pull request is testing the changes from pull request 1162, stacked on pull requests 1158, 1159, and 1160.