How to test a fix without breaking your live store

The two ways to run a storefront test

The first is a script in your page that rewrites it after load. The visitor gets the original page, then JavaScript decides which variant they are in and edits what they are looking at. Many third-party testing tools work this way, because it needs no platform integration and works anywhere.

The second is a change Shopify itself serves or renders: a second theme through Rollouts, a block from an app’s theme app extension placed where you chose in the theme editor, or a Shopify Function running in checkout. Nothing has to be rewritten in the shopper’s browser after the page has drawn.

Most of the risks merchants associate with testing (the flash of the old version, layout shifting as content is replaced, the test tool failing and taking a page with it, the variant losing only because it was slower) belong to the first approach.

Why flicker is a correctness problem, not a cosmetic one

The visible flash is the smaller half. The larger half is that a variant which paints later than its control is being measured against a control it was never comparable to, and slower pages convert worse whatever you changed.

So a rewriting tool can report a loss for a change that was an improvement, or a win that is an artefact of which arm got the extra render. You cannot separate the two afterwards, because the confound is inside the measurement rather than beside it.

That is the strongest argument for platform-served changes, and it is a statistical one rather than an aesthetic one.

One test per surface, always

Two live tests touching the same page template make both results unattributable. Not less precise: unattributable, because every visitor is in both, and no arm of either test isolates a single change.

The failure is quiet, which is what makes it expensive. Both tests conclude, both report a figure, and both figures are wrong in an unknowable direction.

The guard is a rule about surfaces rather than about counting: do not start a second test on a surface that already has one running. Different templates can test at the same time. If you run more than one tool, this is the thing to coordinate by hand, because neither tool can see the other’s tests.

What a real rollback looks like

Ask what reverting does. Switching off a block or ending a rollout restores a known-good state and needs no reasoning about what was changed. Reverse-editing a live theme is a second change with its own chance of being wrong, applied under pressure.

Be most careful with a rollback that depends on the testing tool. If the change lives in a script served by a vendor, reverting needs that vendor to be up. A block you can switch off in your own theme editor reverts without anyone else’s help, which is the property you want on the worst day.

And check what happens to shoppers mid-session. A clean revert keeps a visitor’s experience stable rather than switching the page under someone with a full cart.

Why Liftable never edits your theme’s code

Shopify’s App Store requirements expect apps to change the storefront through theme app extensions rather than by editing theme files. Liftable follows that strictly: it never writes theme files, and it never asks for permission to.

The practical benefit beyond compliance is that the change stays visible and removable. A theme edit made by an app is archaeology in six months. A block in the theme editor has a name, a place and an off switch.

How Liftable runs it

Every change starts as a draft from one of 20 fix templates, for you to read and approve, edit or reject, and every decision goes in a log you can read later. A page change can only be shown through Liftable’s theme app extension, which covers five trust and delivery fixes so far. Shipping and discount tests run as Shopify Functions.

Before a test starts, a power check refuses it if your traffic cannot read it. While it runs, guardrails are checked on every read: a drop in revenue per visitor stops it, and a sample ratio mismatch pauses it. Liftable also caps how many tests a store runs at once, and only the store owner can raise the cap. A shipped change keeps 10% of visitors on the original, and the receipt measures revenue on Shopify orders against that holdback.

Where this stands: a shipping test has run on our own development store, and no test has run on a merchant’s store. Our founding-team programme implements and measures the first fix with you.

The safe sequence

1. 01

   Choose a route that leaves your theme’s code alone

   Rollouts for a theme change, a theme app extension block for a page change, a Shopify Function for shipping or discounts. Each can be switched off without anyone editing your theme files.

2. 02

   Read the change before anything runs

   You should be able to see what changes, where, and why, before a shopper does. If the change is described to you rather than shown, you are approving a summary of a change, which is a different thing.

3. 03

   Preview it on a phone and a desktop

   Walk the actual path (collection, product, cart) in the theme editor’s preview or the test’s preview link, on a phone as well as a desktop. Most bad variants are caught here in minutes.

4. 04

   Split traffic with sticky assignment

   Assignment should be per visitor and stable, so nobody sees the page change under them mid-session and nobody counts in both arms.

5. 05

   Size the test, and judge it where your traffic can read it

   Check before you start that your traffic can read the result in a sensible time. If it cannot at purchase level, judge it at the step the fix acts on, such as reaching checkout for a cart change. The longer a test runs, the more chances something unrelated has to disturb it.

6. 06

   Ship behind a holdback

   Send the winner to most visitors and keep a small share on the original. The step-level win said the change did something. The holdback is what says whether it made money.

7. 07

   Keep the rollback to one action

   Reverting should mean switching a block off, ending the rollout or deactivating the Function: instant, complete and requiring no judgement at two in the morning. If reverting means undoing edits, you do not have a rollback.

Sources

1. Liftable: what is built today. Liftable’s own account of what the product does, checked against the code on 2 October 2026 and shown on /how-it-works and /compare. Every figure this page gives about Liftable comes from it. No test has run on a merchant’s store and there are no published merchant results yet.
2. Shopify App Store requirements. Apps that change a merchant’s storefront are expected to do it through theme app extensions rather than by modifying theme files. Liftable’s install requests no permission to write themes.

Questions