What a compounding CRO loop actually means

The four things a loop has to do

The first three are engineering. The fourth is the one that separates a claim from a marketing sentence, and it is the one almost nobody publishes.

• Record outcomes in a comparable form. Every experiment classified by the lever it pulled, before results arrive — not a free-text note in a spreadsheet.
• Gate what is allowed to teach. Only statistically credible results may enter memory. A noise-win that gets recorded as a lesson pollutes every ranking that follows it.
• Feed memory back into selection. The next round of proposals has to be ranked with the accumulated evidence in the room, or the memory is a museum.
• Be measurable as a loop. You must be able to show that later recommendations are better than earlier ones, not merely that later recommendations exist.

Why the market is quiet about this

When we read the listings of 74 Shopify CRO and A/B testing apps in July 2026, the word "hypothesis" appeared in exactly zero of them. One app out of 74 claimed to learn from past tests. Three claimed to recommend what to test next — the strongest of them, Intelligems, sells that in its entry plan, so "we tell you what to test" is neither new nor expensive.

The gap is between a recommendation and accumulated memory. Nobody says "the system learns your store and measurably gets better at proposing tests for it." It is tempting to read an empty corridor as an opportunity nobody noticed. The more likely reading is less flattering: the claim is empty because proving it is hard, and a claim you cannot prove is weaker than a competitor's shipped feature.

The honest test

Here is the question to put to any vendor claiming a learning loop, including us: show me a recommendation this system made for my store that it could not have made before my earlier tests concluded, and show me the earlier result it came from.

That question cannot be answered with an architecture diagram. It needs a chain of custody: this experiment concluded with this measured effect, that result was credible enough to be recorded, and this later proposal cites it. If a vendor cannot walk that chain for a real customer, the loop is a design, not a result.

A second, harder question follows it: across your customers, do proposals made after ten concluded experiments outperform proposals made after one? That is the compounding claim stated as a measurement. It is answerable, and it requires a lot of concluded experiments before it can be answered at all.

Where we actually stand

Looplift is built as a loop and every mechanism above exists in the product: proposals are classified by primary lever when they are generated, concluded experiments are interpreted into recorded learnings, a credibility gate decides what is allowed to teach, and the next audit ranks its proposals with that evidence.

What we do not have yet is the fourth item. We are early, and we have not accumulated enough concluded experiments to demonstrate that later recommendations are measurably better than earlier ones. So we describe the loop as a mechanism and we do not claim the outcome. When the chain of custody above can be walked end to end for a real store, we will publish it — and until then, treat anyone's compounding claim, ours included, as a design rather than a proven result.

That is not modesty for its own sake. A CRO product that overstates its evidence is contradicting the only thing it is really selling, which is the discipline of not believing a result until it has earned it.