Glossary · Testing
What is the novelty effect?
The novelty effect is a temporary change in behaviour caused by an experience being new rather than being better. Returning visitors notice the difference and engage with it more, which inflates early test results and then fades as the change stops being new.
- Reviewed 6 October 2026
In more detail
It matters most for changes visible to repeat customers: navigation, layout, promotional elements. It matters least for changes that only new visitors encounter.
There are two defences: run long enough for the curve to flatten, and check whether the effect is concentrated in returning visitors. A win that exists only among returning visitors in the first week is a novelty effect until shown otherwise.
Key points
- Inflates early results, then fades.
- Concentrated among returning visitors.
- Caught by a holdback or by segmenting new against returning visitors.
Related terms
Guides that go deeper
Last reviewed against the product on 6 October 2026.
Scan your store.
See where your store is losing revenue and what to fix first. It is free, takes about a minute, and needs no install.