Skip to content

[6.1] Fix flaky code coverage job: pin tool versions and harden thread-job handling - #4611

Draft
paulmedynski wants to merge 1 commit into
release/6.1from
dev/paul/release/6.1/fix-ci
Draft

[6.1] Fix flaky code coverage job: pin tool versions and harden thread-job handling#4611
paulmedynski wants to merge 1 commit into
release/6.1from
dev/paul/release/6.1/fix-ci

Conversation

@paulmedynski

Copy link
Copy Markdown
Contributor

Problem

The Publish Code Coverage job has been failing in CI. Example: run 23310, build 168798.

No test failed. The Convert coverage files to xml step died with a bare ##[error]PowerShell exited with code '1'.

Root cause: dotnet-coverage 18.10.0 emits a first-run telemetry notice on stderr:

Telemetry
---------
The code coverage tools collect usage data in order to help us improve your experience.
The data is collected by Microsoft. You can opt out of telemetry by setting the
DOTNET_COVERAGE_TELEMETRY_OPTOUT environment variable to '1' ...

PowerShell converts native-command stderr into ErrorRecords, which land in each thread job's error stream. PowerShell@2 defaults to errorActionPreference: stop, so the first Receive-Job that replayed the notice raised a terminating error and pwsh exited 1.

Evidence from the log:

  • Merging started... appears once — the script died draining the very first netFx job, so MergeFiles for netCore never ran.
  • All 151 individual merges reported Merged into file ... successfully. The coverage data itself was fine.
  • The log ends mid-banner at Telemetry, immediately followed by the error.

The tool install was unpinned, so 18.10.0 floated in on its own. The last green run of this job (build 167409, 2026-08-16) ran dotnet-coverage v18.9.0.0 with zero occurrences of "Telemetry" across 36,065 log lines.

Changes

Pin the tool versions

  • dotnet-coverage18.9.0, dotnet-reportgenerator-globaltool5.5.11. These are exactly the versions from the last green run of this job.
  • Installs moved from a bare dotnet tool install script to DotNetCoreCLI@2 tasks, with NuGet.config copied to $(Agent.TempDirectory) so tool restores honour the repo's feeds. This matches the pattern already used on main.

Opt out of the telemetry notice

  • Added DOTNET_COVERAGE_TELEMETRY_OPTOUT: 1 as a job variable so the banner is never emitted, independent of tool version.

Harden the thread-job steps

  • Tool invocations now redirect 2>&1 and capture $LASTEXITCODE, so informational stderr becomes ordinary output and only a genuine non-zero exit throws. Previously a real dotnet-coverage failure was silently ignored — the job still reported Completed.
  • Draining changed to Receive-Job -ErrorAction Continue 2>&1 | Out-String | Write-Host plus explicit Remove-Job, with job state checked after Wait-Job and an explicit throw naming how many jobs failed.
  • Write-Error + exit 1 for the "no .coverage files" case became a throw, so it doesn't abort the whole script from inside the function.
  • ReportGenerator's three near-identical blocks collapsed into a StartReportGenerator helper using -ArgumentList.

Notes on version choice

main pins dotnet-coverage to 18.3.2, but its coverage job is a different shape — Linux, a single dotnet-coverage merge in a plain script: step, no ReportGenerator, no thread jobs — so it never exercised this path. Pinning 6.1 to 18.3.2 would be a six-version downgrade to something this job has never run, hence 18.9.0.

Validation

  • YAML parses cleanly; all 13 embedded pwsh scripts pass [Parser]::ParseFile with no errors.
  • The new thread-job pattern was smoke-tested locally under $ErrorActionPreference = 'Stop' with a stub tool: a noisy-but-successful tool no longer fails the step, and a tool exiting non-zero still does.
  • Full CI validation pending on this PR.

Checklist

  • Verified against the failing CI run
  • Ensure no breaking changes introduced
  • Tests added or updated — n/a, pipeline-only change
  • Public API changes documented — n/a

…d-job handling

The Publish Code Coverage job started failing when dotnet-coverage 18.10.0
shipped a first-run telemetry notice on stderr.  PowerShell turns native
stderr into error records, and the PowerShell task defaults to
ErrorActionPreference=Stop, so the first Receive-Job that replayed the notice
raised a terminating error and killed the step before the netCore merge even
started.

- Pin dotnet-coverage to 18.9.0 and dotnet-reportgenerator-globaltool to
  5.5.11, the versions from the last green run of this job (build 167409).
  Installs now go through DotNetCoreCLI@2 with NuGet.config copied to a temp
  working directory, matching the approach on main.
- Set DOTNET_COVERAGE_TELEMETRY_OPTOUT so the notice is never emitted.
- Redirect tool stderr into stdout and check $LASTEXITCODE explicitly, so
  informational output can no longer fail the step while a genuine tool
  failure now does.  Previously a failed merge was silently ignored.
- Drain thread jobs with -ErrorAction Continue and fail on job state instead
  of on replayed error records.
Copilot AI balanced review requested due to automatic review settings August 28, 2026 16:50
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Aug 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes the Windows code-coverage pipeline by preventing telemetry output from failing PowerShell thread jobs and reliably detecting tool failures.

Changes:

  • Pins coverage-tool versions and configures repository package feeds.
  • Disables dotnet-coverage telemetry.
  • Hardens concurrent merge and report-generation job handling.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

2 participants