How does A/B test traffic split between versions?

Each visitor gets one version at random, in line with your percentages. They keep that version on later visits in the same browser. You set the split before you start. It locks when the test starts.

Set the split

1. Open your draft test.
2. Under Traffic split, drag the slider or click Split evenly.
3. Click Save traffic. Start test uses the saved split, so a change you don't save is lost.

Each version needs at least 5% and at most 95%. The total must be 100%.

How visitors get a version

• Random, not by time. Visitors are spread across versions from the first day.
• Sticky. The same browser sees the same version for the whole test.
• New browser or device. A visitor on another device, or who cleared their browser data, can get another version.
• Checkout matches. Each shopper gets the discount or rewards of the version they saw.
• You too. Your own visits get a version. To see a certain one, use Preview. Previews don't count.

Who counts as a visitor

Not every store visit counts.

• Bundle test: a visitor counts once the bundle widget is on their screen.
• Cart test: a visitor counts once the drawer opens.
• Shoppers who decline analytics tracking still see a version. We don't count them.

So Visitors in your test is lower than your store's traffic. That's normal.

Can I change the split during a test?

No. This traffic split was locked when the test started. To use another split, end the test and start a new one.

1. In Shopify admin, open Apps → Section Store → Bundles or Cart Drawer.
2. Click A/B Tests and open the test.
3. Click End test. Under Final version, pick the version to keep.
4. Click End test with selected version.
5. Create a new test. Its Version B starts as a copy of your live item, so make your changes again.
6. Set the new split and click Save traffic.
7. Click Start test.

Related

• Create an A/B test
• Read your A/B test results
• Why does my A/B test show no results yet?

Also searched as: traffic split, split traffic, 50/50, 50 50 split, split evenly, change traffic during test, traffic percentage, visitors assigned, sticky version, same version each visit, random visitors, who counts as a visitor, visitors lower than store traffic, i see a different version, save traffic, change split during test, change 50/50 to 80/20, change traffic while test is running