[ci] Run MAUI R2R tests with XHarness - #12666
Open
simonrozsival wants to merge 2 commits into
Open
Conversation
Use Arcade's XHarness Android work items so transient device installation failures get the standard reboot and retry behavior while retaining the existing app launch smoke checks and diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new Helix MSBuild project currently allows XHarnessApkToTest to run without validating AndroidPackageName, which can lead to confusing failures and should be gated/validated before merge.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
 |
build-tools/automation/MauiR2RHelix.proj — ❌ error — XHarnessApkToTest is enabled when WorkItemApk is set, but AndroidPackageName can… |
 |
build-tools/automation/MauiR2RHelix.proj — Set-Content/Add-Content calls don’t specify an encoding. On Windows… |
What changed in this PR
This PR updates the MAUI R2R Helix submission flow in dotnet/android CI to use Arcade/Helix’s XHarnessApkToTest integration (instead of a bespoke adb-driven work-item script), aiming to improve resiliency on device install failures by leveraging XHarness’s retry/reboot behavior while preserving the existing smoke-test behavior and diagnostics uploads.
Changes:
- Adds an
eng/Versions.propspin for the XHarness CLI version intended to match MAUI’s usage. - Updates the MAUI R2R Helix matrix template to stage only the signed APK and submit Helix work items using
WorkItemApk/AndroidPackageNameenv vars, with a longer work-item timeout. - Switches
MauiR2RHelix.projfromHelixWorkItemtoXHarnessApkToTestand embeds PowerShell-based custom commands for install + monkey launch +pidofsmoke check + log capture/uninstall.
| File | Description |
|---|---|
| eng/Versions.props | Introduces a pinned XHarness CLI version property. |
| build-tools/automation/yaml-templates/run-maui-r2r-helix-matrix.yaml | Removes the custom adb/payload script generation and submits Helix work items via WorkItemApk + AndroidPackageName, increasing timeout. |
| build-tools/automation/MauiR2RHelix.proj | Uses XHarnessApkToTest with embedded custom commands to install/launch/validate/log/uninstall the APK. |
| <Command>$(WorkItemCommand)</Command> | ||
| <Timeout Condition="'$(WorkItemTimeout)' != ''">$(WorkItemTimeout)</Timeout> | ||
| </HelixWorkItem> | ||
| <XHarnessApkToTest Include="$(HelixWorkItemName)" Condition="'$(WorkItemApk)' != ''"> |
Comment on lines
+27
to
+29
| "Scenario: $(HelixTestRunName)" | Set-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") | ||
| "PackageName: $package_name" | Add-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") | ||
| "APK: $app" | Add-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") |
Resolve the central versions conflict by retaining the updated build task dependency and the XHarness CLI pin. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
adbwork-item script with Arcade'sXHarnessApkToTestintegrationxharness android installso package-manager failures receive XHarness's device reboot and install retry behaviormonkeylaunch,pidofsmoke check, and uploaded device/package/logcat diagnosticsContext
The existing script exits immediately on
INSTALL_FAILED_INSUFFICIENT_STORAGE, bypassing XHarness's recovery path. This contributed to the repeated Helix device failures tracked by dotnet/arcade#17483 and #12658.Validation
XHarnessApkToTestwork-item archive with the pinned Helix SDK11.0.0-prerelease.26428.1is available from the configured engineering feed