Skip to content
lube

Test intelligence

8 formats3 detectorsRegressions

Stop re-investigating the same flaky test.

Push results from any runner. Every test keeps a stable identity across runs, and three independent flakiness detectors tell a real failure from noise. The red build stops being a coin toss.

  • checkout.spec
  • auth.spec
  • billing.spec
  • search.spec
pass flake fail
Outcomes across recent runs

Its place in the loop

Tests verify the work coming out of Engineering.

One connected loop, held on the stage this capability serves. The other stages stay as context so you can see what feeds in and what comes next.

BD
Discovery
Feedback
Product
Engineering
Release
Reliability
Evidence

Stage inventory

Engineering

Build and verify the change.

VCS insights

Inspect delivery workflow activity in the connected release workspace.

Available now

Test intelligence

Investigate test outcomes and flakiness evidence across runs.

Available now

API Explorer

Keep API contracts inspectable next to the release that changed them.

Available now

Compute pricing

Compare public cloud compute pricing as planning evidence.

Available now

Coming later reflects product vision, not a delivery commitment.

Reporting, not a runner

lube reads what your CI already produces — it never runs, gates, quarantines, or retries a test. Keep your runner exactly as it is. Raw per-test rows live for 7 days while daily and monthly rollups persist, so the history you look back on stays cheap and fast. No CI gating, no coverage ingestion.

The shape of it

8

report formats ingested, auto-detected from the file

3

independent flakiness detectors, each with its own rule

4

rollup tables keep history cheap past raw retention

7d · 90d · 365d

raw / daily / monthly retention, pruned independently

Flakiness, shown not stated

A flaky test flips pass to fail on the same commit.

Watch one test flap between pass and fail while everything around it stays green. That non-determinism is exactly what the three flakiness detectors catch.

latest run · per test

history → latest
  • checkout.spece2estable
  • auth.spec · token refreshintegrationflaky
  • search.specunitstable
  • billing.specintegrationstable
Latest result, per test

Three detectors, not one guess

Each detector names the exact rule it fires on.

Rerun, alternation, and windowed instability each catch a different shape of flake — so a test that only passes on retry and one that flips between runs are both caught, and neither is a heuristic you have to trust blind.

  • Rerunretry

    A test that only went green after CI retried it — passed despite retry_count > 0.

  • Alternationbranch flips

    Status flips between consecutive runs on the same branch, never a clean pass or fail.

  • Windowed7-day

    A 7-day pass rate stuck strictly between 5% and 95% — never reliably green, never reliably red.

Flakiness detectors · per identity, per day

Pass rate over time

The trend, per app, per day.

A per-day pass-rate trend built from the run-daily rollup, toned on the same 95% / 80% thresholds the dashboard uses — so a slow slide into a bad week is obvious before anyone files a ticket.

pass rate · per day

96%
Jul 1Jul 14
Pass-rate trend · last 14 days

Open any run

Classified the moment it lands.

A run detail sorts every test into flaky, failing now, unstable, or stable, then shows each test's identity-history sparkline — because a stable identity across runs is what makes a green-red-green streak legible at a glance.

  • 3

    flaky

  • 2

    failing now

  • 4

    unstable

  • 214

    stable

  • auth.spec · token refreshintegration
  • checkout.spec · full flowe2e
  • orders.spec · bulk importapi
  • pricing.spec · roundingunit
Run detail · #4821 on main

The insights beside the commit

A single red build rarely tells the whole story.

Suite timing

Find the long pole in the run.

Every suite's duration fills to scale as the run is measured, and the slowest one is flagged so the bottleneck is obvious.

p50 / p95 / maxslowest flaggedper suite

suite duration

4m 12s
  • e2e checkout2m 41sslowest
  • integration1m 08s
  • unit38s
  • smoke12s
Suite duration this run

Slowest tests

The wall-clock hogs, ranked by p95.

The tests dragging CI, sorted by their p95 duration across recent runs, so the expensive few are separated from the fast many.

p95 per testtop-N leaderboardfrom the daily rollup
  • e2e/checkout · full flowp95 2.4s
  • api/orders · bulk importp95 1.6s
  • integration/authp95 0.9s
  • unit/pricingp95 0.3s
Slowest tests · p95 across recent runs

Regressions

A regression inbox, not a wall of logs.

A test that passed yesterday but failed today opens one entry per test-day and stays open until it passes again, so new breakage is separated from old noise.

passed yesterdayfailed todayone per test-day
  • billing.spec · tax roundingfailed today
  • checkout.spec · retry pathstill open
  • search.spec · rankingresolved
Regression inbox · passed yesterday, failed today

Everything it ingests, everything it slices

Point it at your reports and it takes it from there.

The uploader auto-detects the file and format, captures your CI context, and the server keys every result to a stable identity. Then you slice the run any way you read it.

Report formats

Auto-detected from the filename; override with --format.

JUnit XMLVitestJestgo testgotestsumcargo-nextestpytestRSpec

CI providers

Detected from the environment; falls back to local.

GitHub ActionsBuildkiteCircleCIGitLab CITravislocal

Captured per run

Attached from the CI context, no config.

commit SHAbranchorigin ci / localenvironment labelservice

Filter & group

On the run detail.

all / failedgroup by suitefailed-firstshow file path

Classification

Per test, per run.

flakyfailing nowunstablestablepass / fail / skip

Retention windows

Pruned independently.

raw 7ddaily 90dmonthly 365dresolved 30d

Under the hood

A reporting pipeline, not a black box.

Every claim on this page is a shipped surface. Here is the machinery it runs on.

Test identity
A stable hash over test name, suite path, and file path, so a test keeps one identity as runs come and go — and its history stays a single legible streak.
identity hashcross-run
Storage
Two stores: PlanetScale PostgreSQL holds run headers and identities, ClickHouse Cloud holds the raw per-test rows. Run headers persist beyond raw retention through the rollups.
PostgreSQLClickHousetwo-store
Rollups
Four runners keep history cheap: per-identity daily, per-identity monthly, per-application run-daily, and the new-failure regression inbox — each cursor-tracked so a crash replays the same bucket.
result-dailyresult-monthlyrun-dailynew-failure
Flake detectors
Rerun (passed despite a retry), alternation (status flips on the same branch), and windowed-unstable (7-day pass rate strictly in 5%–95%) — three orthogonal signals computed on the daily rollup.
rerunalternationwindowed
Limits
10 MB per report and 50,000 results per run, enforced at upload.
10 MB50k results
Ingest & insights
POST a report as multipart; the format parser normalizes it, the identity hash keys each result, then it lands in both stores. Read it back through flaky, pass-rate trend, slowest, and open-regression endpoints, plus per-run and per-identity history.
multipart uploadflakytrendslowestregressions

Still just a reporting layer

lube never blocks a merge, quarantines a test, or triggers a retry — those decisions stay in your CI. It ingests what your runner emits and turns it into evidence you can read beside the commit. That is the whole job.

Connected context

See what a test result belongs to.

Find the flake before it finds you.

Start free

Push one report from your CI and start.