Stop guessing. Start knowing which tests you can trust.
Test reliability analysis for .NET — accumulates history across dotnet test runs
so flaky tests stop being anecdotes.
Local mode requires no account and no network access.
| Local | Cloud | |
|---|---|---|
| Setup | SDK package + CLI tool | + an API key |
| Account | Not required | Required (invite-only) |
| Your data | Stays in .xping/ in your repo |
Uploaded to Xping Cloud |
| History | Your machine, your runs | Every machine, every branch, CI included |
| You get | Observations and evidence from local runs | Confidence scores, evidence sufficiency, trends, PR comments |
Start local. Nothing leaves your machine, and you can add an API key later without changing a line of test code.
dotnet add package Xping.Sdk.XUnit # or Xping.Sdk.NUnit / Xping.Sdk.MSTest
dotnet new tool-manifest # skip if you already have one
dotnet tool install Xping.CliThe SDK records what happens during each run; the CLI reads what it recorded. You need both — the SDK alone writes history nothing reads.
xUnit — add one line to AssemblyInfo.cs:
[assembly: TestFramework("Xping.Sdk.XUnit.XpingTestFramework", "Xping.Sdk.XUnit")]NUnit — add one assembly-level attribute:
[assembly: XpingTrack]MSTest — inherit from the base class:
[TestClass]
public class MyTests : XpingTestBase { }dotnet testOne run tells you nothing about reliability. A dozen tells you plenty. Run them the way you normally would and let the history build up.
dotnet xping reportXping · SampleApp.MSTest · 9 runs · 2026-08-20 07:52 → 09:02 · main@bdbafba
4 findings (4 high) · 17 tests · 14 healthy
HIGH flaky FlakyTest_PassesOnRetry
failed 9 of 18 executions (50%) in 9 of 9 runs, 1 failure mode
evidence moderate | f_2b84a621
HIGH always failing ThrowingTestIsTracked
failed 9 of 9 executions (100%), one failure mode:
System.InvalidOperationException
evidence low | f_c1774d82
HIGH flaky FlakyTest_RaceCondition_FailsIntermittently
failed 4 of 9 executions (44.4%) in 4 of 9 runs, 1 failure mode
evidence low | f_d24c5aa9
HIGH masked by retry FlakyTest_PassesOnRetry
passed on retry 9 times in 9 of 9 runs, up to attempt 2
evidence moderate | f_e98db1e6
No API key, no signup, no network calls. Everything lives in .xping/ in your machine.
- NUnit Setup Guide — attributes, filtering, best practices
- xUnit Setup Guide — custom framework configuration and examples
- MSTest Setup Guide — base class usage and TestContext integration
dotnet test is amnesiac. Every run starts from zero and discards everything the last one
knew. A test that failed on Tuesday and passed on Wednesday leaves no trace that anyone can
point at on Thursday, so "is that one flaky?" gets answered from memory and vibes — and
usually settled by re-running until it goes green.
Xping is the accumulation layer underneath that. The SDK records each execution with its outcome, duration, and environment; the CLI reads that history back and tells you which tests behaved consistently and which didn't. Nothing about your test code changes.
The local CLI reports what it observed. It does not assign confidence scores or name causes — a handful of runs on one machine isn't an evidence base strong enough to support that kind of claim, and we'd rather show you the runs than a number that implies more certainty than the data has. Scoring lives in Xping Cloud, where there's enough history across machines, branches, and CI to justify it.
| Local | Cloud | |
|---|---|---|
| Run-by-run pass/fail history | ✓ | ✓ |
| Inconsistent and newly-failing tests | ✓ | ✓ |
| Consistently-failing tests (real bugs) | ✓ | ✓ |
| Duration and environment capture | ✓ | ✓ |
| Per-test confidence score | — | ✓ |
| Evidence sufficiency (ESS) | — | ✓ |
| Root-cause categorisation | — | ✓ |
| Reliability trends over time | — | ✓ |
| Cross-environment comparison | — | ✓ |
| History across CI, branches, teammates | — | ✓ |
| GitHub PR comments | — | ✓ |
Set two environment variables. Nothing else changes — same packages, same attributes.
export XPING_APIKEY="your-api-key"
export XPING_PROJECTID="your-project-id"Runs are uploaded as they finish and analysed at app.xping.io: confidence scores per test, evidence sufficiency, root-cause categorisation, trends across environments, and PR comments on GitHub.
Xping Cloud is currently invite-only. We're running a small, high-touch pilot while the scoring model settles. Request access — or keep working locally, which stays free and account-free regardless.
For CI setup (GitHub Actions, Azure DevOps, Jenkins, GitLab), see the Configuration Reference.
Your test project (xUnit · NUnit · MSTest)
│
▼
Xping.Sdk.<framework> adapter
│
▼
Xping.Sdk.Core
tracking · environment detection · batching
│
┌─────────────────┴─────────────────┐
│ no API key │ API key set
▼ ▼
.xping/ (local store) Xping Cloud
│ │
▼ ▼
dotnet xping report scores · trends · PR comments
Adapters are thin — they hook the framework's execution pipeline and hand results to
Xping.Sdk.Core, which owns collection, environment detection, and delivery. Overhead is
under 5 ms per test.
Configure via environment variables, appsettings.json, or programmatically.
# Cloud only
export XPING_APIKEY="your-api-key"
export XPING_PROJECTID="your-project-id"
# Optional, either way
export XPING_ENABLED="true"
export XPING_BATCHSIZE="100"Without XPING_APIKEY, the SDK writes to .xping/ and makes no network calls. Full
options in the Configuration Reference.
Per test execution — name and fully qualified name, outcome, duration, start and end timestamps (UTC), error message and stack trace on failure, categories and traits.
Per environment — OS and version, .NET runtime version, machine name, CI platform detection, build and branch information from the CI environment, network metrics.
No source code. No assertion values. No credentials or secrets. No personally identifiable information.
Everything is written to .xping/ in your repository and stays there. No network calls are
made without an API key. Add .xping/ to your .gitignore — the history is machine-local
and isn't meant to be shared through version control. To start over, delete the folder.
Data is transmitted over HTTPS. Keep API keys in environment variables or CI secrets, never in source control. Stack trace capture and sampling are configurable, and retention is set per workspace. The SDK is MIT-licensed and open source — read exactly what it sends.
Patterns that show up in accumulated execution history:
- Race conditions — intermittent failures with no code change between runs
- External dependencies — failures that track network or service availability
- Shared state — tests that pass alone and fail in a suite
- Time-based flakiness — failures clustered around dates, times, or timezones
- Resource exhaustion — degradation over the course of a long run
- Non-deterministic data — random inputs that occasionally hit an edge case
Working locally surfaces the behaviour; Xping Cloud attributes the cause. See the Common Flaky Patterns Guide for worked examples.
| Path | Contents |
|---|---|
src/Xping.Sdk.Core |
Collection, environment detection, configuration, delivery |
src/Xping.Sdk.XUnit · .NUnit · .MSTest |
Framework adapters |
src/Xping.Cli |
dotnet xping — local analysis and reporting |
samples/ |
Runnable examples per framework |
docs/ |
Source for docs.xping.io |
Full documentation at docs.xping.io.
Bug reports, feature requests, documentation fixes, and code contributions are all welcome.
git clone https://github.com/xping-dev/sdk-dotnet.git
cd sdk-dotnet
dotnet restore
dotnet build
dotnet testSee CONTRIBUTING.md for guidelines.
Tracked in Milestones: Working Set · Backlog
Currently on the list:
- Quarantine — mark known-flaky tests so CI stops failing on them
- Richer local analysis: duration regression, failure signature grouping
xping mcp— expose local history to AI coding agents- Azure DevOps and GitLab integration
- GitHub Discussions — questions and ideas
- Issue Tracker — bugs and feature requests
- support@xping.io — direct support
MIT — see LICENSE.