跳至內容

Rust flaky tests

Tuist can detect inconsistent nextest results and help you investigate them in Tests → Flaky Tests.

Requirements

How flaky detection works#

Tuist detects flaky tests from retry attempts and results across CI runs.

Test retries#

Enable nextest retries on the command line:

bash
tuist cargo nextest run --retries 2

Or configure your existing .config/nextest.toml:

toml
[profile.ci]
retries = 2

Run the selected profile:

bash
tuist cargo nextest run --profile ci

When a test fails and then passes on a retry, Tuist marks it as flaky. Nextest controls the retries and the final result of the run.

Cross-run detection#

Even without retries, Tuist can detect flaky tests by comparing CI runs on the same commit. A test that passes in one run and fails in another can be marked as flaky.

Managing flaky tests#

Automatic clearing#

Detection and recovery run through the project's "Flaky test detection" automation. By default, the flaky flag clears after 14 days without re-triggering. Edit the recovery window and actions under Settings → Automations.

Manual management#

Open Tests → Flaky Tests to inspect a test's history and retry attempts. You can manually mark or unmark it as flaky from its detail page.

Quarantining and stress-testing#

The Cargo integration does not apply quarantine yet: tests muted or skipped in the dashboard still run with nextest, and their failures still affect the result. It does not stress-test new tests either. Use test retries while fixing a flaky test.

Slack notifications#

Configure Slack flaky test alerts to notify your team when a test becomes flaky.