Why A/B testing apps break Shopify themes

The mechanism: testing apps that live inside your theme

To run a test on a Shopify storefront, an app has to change what a visitor sees. There are broadly two ways to do that, and they carry very different risks.

The first is to integrate with the theme itself: duplicate it, write into its Liquid files, or install blocks and snippets in it. This is powerful — it is the only way to test one whole theme against another, and it puts the variant into the server-rendered HTML. It is also how a testing app acquires the ability to damage the thing it lives in.

The second is to change the rendered page in the browser, from a script, and never write to the theme at all. This cannot test a whole theme, and the change happens after the page arrives rather than inside it. In exchange, the app has no ability to modify the merchant's theme files, because it never touches them.

Almost every serious app in this category takes the first route. That is a reasonable engineering choice, and it is why the two complaint categories above exist at all.

Why uninstalling does not clean up

Here is the part that surprises merchants. Uninstalling a Shopify app revokes its access — it does not revert the edits the app already made to your theme. If an app added a snippet, an include, or a block while it was installed, that code is now part of your theme file, and removing the app leaves it exactly where it is.

That is the whole explanation for reviews like "have uninstalled but the code remains slowing my homepage down" and "after uninstall, it still leaves blocks of trash code on my page." Nobody is being malicious. The app simply cannot clean up after itself once its access is gone, and the merchant is left doing theme surgery in a file they did not write.

It compounds with themes. A merchant who duplicates their theme to try a redesign carries every leftover fragment into the copy. Six months and three apps later, the theme contains code from tools nobody on the team remembers installing.

How to check an app before you install it

None of these questions are about statistics or features. They are about what happens on your worst day with the tool, which is the only thing a review at one star is ever about.

• Ask directly: does this app write to my theme files, and what exactly does it write? A vendor who cannot answer precisely has not thought about your cleanup.
• Ask what happens on uninstall. "Everything is removed automatically" deserves a follow-up: removed by what, if the app no longer has access?
• Duplicate your live theme before installing anything, and keep that copy untouched. It is the cheapest rollback in commerce.
• After installing, diff the theme. Shopify's theme editor shows you the files; a new snippet or a modified layout is what you are looking for.
• Check whether the app fails open. If the vendor's API is down, does your store render its original content, or does it hang waiting?
• Check whether you can stop everything yourself, without emailing support. A kill switch you have to request is not a kill switch.

What we chose, and what it costs us

Looplift takes the second route. There is no code anywhere in our product that writes to a Shopify theme, and we do not request the permission that would let us. The integration is one snippet you paste in yourself; deleting that line removes all of Looplift, because there is nothing else of ours anywhere in your store.

We are not presenting this as a free win, because it is not one. It costs us whole-theme testing and template testing outright — if that is the test you want to run, an app that integrates with your theme is the correct purchase, and we will tell you so. It also means our changes are applied in the browser rather than server-rendered, which is why our snippet briefly hides the affected page while it works, capped at 1.5 seconds.

That is the trade we made: less reach, no ability to damage the thing we are testing. The category's own reviews are the argument for it. Whatever you install — ours or someone else's — install it knowing which of the two routes it takes, because the reviews above are what the other route costs when it goes wrong.