Glossary · Testing
What is peeking?
Peeking is checking a fixed-horizon test before its planned sample size is reached and acting on what you see. It inflates the false-positive rate, because each look is another chance for random variation to cross the significance threshold, and the effect grows with the number of looks.
- Reviewed 6 October 2026
In more detail
Peeking is near-universal, because the dashboard is right there and the test is about something the team cares about. It is a common reason organisations build up a history of wins that never show up in the revenue line.
The honest options are to commit to a sample size and genuinely not look, or to use a sequential method whose validity survives being watched.
Key points
- Each additional look raises the chance of a spurious win.
- A usual explanation for wins that do not repeat.
- Solved by sequential inference, not by discipline.
How Liftable relates
Sources
- Kohavi, Tang and Xu, Trustworthy Online Controlled Experiments (Cambridge University Press, 2020). Standard reference on online testing practice, including guardrail metrics, sample ratio mismatch and the cost of peeking.
Related terms
Guides that go deeper
Last reviewed against the product on 6 October 2026.
Scan your store.
See where your store is losing revenue and what to fix first. It is free, takes about a minute, and needs no install.