Glossary · Testing
What is a holdback?
A holdback is a small share of traffic deliberately kept on the original experience after a winning variant has gone live. Comparing the two afterwards measures whether the improvement persisted in production, which is a different and stricter question than whether the test won.
- Reviewed 6 October 2026
In more detail
Tests win for reasons that do not last. Novelty, seasonality, a concurrent promotion or a subtle flaw in assignment can all produce a genuine-looking result that fades.
A holdback is the mechanism that catches this after the fact, and it is what makes a claimed revenue figure defensible rather than optimistic. It costs a small share of traffic on the old experience, which is the price of being able to say a number is real.
Key points
- A small slice kept on the original after the winner goes live.
- Tests whether the effect persisted, not whether the test won.
- The basis for claiming recovered revenue honestly.
How Liftable relates
Related terms
Guides that go deeper
Last reviewed against the product on 6 October 2026.
Scan your store.
See where your store is losing revenue and what to fix first. It is free, takes about a minute, and needs no install.