Glossary · Testing · also always-valid inference, continuous monitoring
What is sequential testing?
Sequential testing is a family of statistical methods that stay valid while a test is being monitored, so results can be checked at any point without inflating the false-positive rate. The mixture sequential probability ratio test (mSPRT) is a widely used variant in online testing.
- Reviewed 6 October 2026
In more detail
The practical benefit for a smaller store is stopping early and honestly. A fixed-horizon test makes you wait for a pre-computed sample even when the effect turns out to be large; a sequential test lets you stop as soon as the evidence is sufficient.
The cost is a modest loss of power against a fixed-horizon test that is genuinely never looked at early. Since few teams achieve that in practice, the trade is usually a large net gain in reliability.
Key points
- Valid under continuous monitoring: looking early is allowed by construction.
- Stops early when effects are large, without inflating false positives.
- Slightly less powerful than a perfectly run fixed-horizon test.
How Liftable relates
Sources
- Johari, Koomen, Pekelis and Walsh, “Always Valid Inference: Continuous Monitoring of A/B Tests” (Operations Research, 2022). Defines always-valid p-values and confidence intervals for online tests, built on the mixture sequential probability ratio test, and shows they stay valid under continuous monitoring.
Related terms
Guides that go deeper
- Testing · 5 minHow to split-test when you don’t have the trafficMost stores can’t read a purchase-level A/B test in a sensible time. Why, and the adjustments that make testing work at the volumes stores actually have.Read
- Testing · 7 minHow to test a fix without breaking your live storeTest through a route that never edits your theme’s code, read the change first, run one test per surface, keep the rollback to one switch. The safe sequence.Read
Last reviewed against the product on 6 October 2026.
Scan your store.
See where your store is losing revenue and what to fix first. It is free, takes about a minute, and needs no install.