Run an A/B test on bundles or the cart drawer
How A/B tests work in Section Store, what you can test, and the steps from first draft to picking a winner.
A/B tests compare up to four versions of a published bundle or your live cart drawer, using paid-order results. They're part of Plus. Open them with the A/B Tests button on the Bundles or Cart Drawer page.
Before you start
- You need Plus. A/B testing is included with Plus. A free trial counts.
- Bundle tests: the bundle must be published, and shown in your live theme.
- Cart tests: the cart must be on, with the Cart drawer app embed on in your live theme.
- When you start the first test, Shopify asks for analytics and order permissions. Click Review Access Scopes and approve them.
The steps
- Create an A/B test. Version A is your live bundle or cart. Change Version B, and add C and D if you like.
- Set the traffic split and click Save traffic. Start test uses the saved split, and the split locks when the test starts.
- Click Start test.
- Read your results once data comes in.
- End the test and pick a winner.
To see the report first, click View sample report on the A/B Tests page. It uses example data and creates no test.
Quick facts
- Versions: 2 to 4, labelled A to D.
- Traffic: each version gets 5% to 95%. The total must be 100%.
- Length: we recommend 2 to 5 weeks. A test stops collecting after 90 days.
- A winner needs: at least 7 complete active days, and at least 500 visitors and 20 paid orders on every version.
- Main measure: revenue per visitor.
- One test at a time: a bundle or cart can have one open test.
- During a test: don't edit the bundle or cart by hand. Saved changes go live for Version A, shared settings change every version, and your results get harder to compare.
- Usage: Plus includes visitor exposures each billing period. See A/B test visitor allowance.
What you can't test
- Conversion Blocks, sections and AI Pages. A/B tests cover bundles and carts only.
- A bundle or cart that isn't live.
AI assistants you connected to Section Store can create and manage tests too.
If it doesn't work
Related
Also searched as: ab test, a/b test, a b test, ab testing, a/b testing, split test, split testing, test two bundles, test two versions, compare bundles, compare cart drawer, compare versions, which bundle converts better, which bundle sells more, test cart drawer, cart drawer ab test, bundle ab test, quantity break test, version a version b, variant test, experiment, conversion blocks ab test, view sample report, how long to run a test
Did this answer your question?