# Run an A/B test with an AI assistant

> Ask a connected assistant to test two versions of a live bundle or cart, read the results and keep the winner. Needs Plus.

Ask your assistant to test a change on a live bundle or cart. It makes a draft, shows you both versions and starts the test after your yes. When you have results, it ends the test and keeps the version you pick. Starting and ending each go in two steps. A started test can't change. To fix a version, end the test and ask for a new one. A/B tests need Section Store Plus.

## Before you start

- The bundle or cart must be on before the test starts.
- Your theme must show it. For a cart, the **Cart drawer** app embed must be on. For a bundle, the **Bundles** app embed or a **Bundle** block must be on.
- The first test may need a permission for analytics. Accept it in the app.

## Run a test

1. Ask: "Test 15% off against my current 10% on [bundle]."
2. The assistant makes a draft. Version A is the item as it is. Version B starts as a copy, with your change. A draft shows shoppers nothing.
3. Check the pictures of both versions. Ask for changes to B. These save at once.
4. Set the traffic if you want. The default splits evenly. You can add versions C and D.
5. Ask to start. Nothing starts yet. You see a picture of each version, what each changes and the traffic split.
6. Say yes. The assistant confirms within 15 minutes. Shoppers each see one version.

**After the start, no version can change, and the start can't be undone.** The only way out is to end the test.

## Read the results

Ask: "How is my test doing?" You get visitors, paid orders, conversion, revenue, revenue per visitor and lift against A for each version.

We judge a test after 7 full days, with 500 visitors and 20 paid orders per version. Until then you see lines like **Each version needs 500 visitors before we judge.** After that you see one of:

- **Version B wins: 12.5% more revenue per visitor than A.**
- **No version beats A by enough to trust. Keeping A is the safe pick.**

## End the test

1. Ask: "End the test and keep version B."
2. Nothing ends yet. You see what the results say, a picture of that version and what changes on the live item. The assistant tells you if we haven't called that version the winner.
3. Say yes. The assistant confirms within 15 minutes. Every shopper now sees that version.

Keeping A leaves the item as it was. Any other version replaces the live item. You can undo that for 30 days. An ended test can't restart.

## Rules

- A test has two to four versions, A to D.
- Each version gets 5 to 95 percent of traffic, and the total is 100.
- A bundle or cart has one open test at a time.
- Custom CSS, a bundle's products, and the cart's **Open drawer when products are added** and **Open drawer when cart icons are clicked** settings belong to every version. They change on version A.
- Switching the item off pauses the test. Switching it on resumes it.
- A test runs at most 90 days. Then traffic stops, shoppers see version A, and the test waits for your pick.
- A draft can be deleted. A started test must end.

## Test views

Each billing period includes 1,000,000 visitor exposures on the $15 plan, 650,000 on the $10 plan and 500,000 on the $8 plan.

When a store runs out, the app shows **A/B testing is paused for this billing period**. We end running and paused tests on version A, and every shopper sees version A. Those tests don't restart. You can still read their results. New tests can start in the next billing period.

## If it doesn't work

- **"A/B tests need the Section Store Plus plan":** start Plus. Every test tool needs it, reading results too.
- **"The bundle is off, and a test starts only on a live bundle":** switch it on first.
- **"Add the Section Store Bundle app block to a product template":** turn on the **Bundles** app embed or add the block. The embed counts too.
- **"Turn on the Section Store Cart app embed":** turn on the **Cart drawer** app embed in the theme editor.
- **"This bundle already has an A/B test":** finish or delete that test first.
- **"The store used all its A/B test views for this billing period":** wait for the date in the message, then start a new test.

## Related

- [Change a live bundle or cart with ChatGPT, Claude or another AI assistant](https://section-store.orka.help/a/change-a-live-bundle-or-cart-with-an-ai-assistant)
- [Undo a change an AI assistant made](https://section-store.orka.help/a/undo-a-change-an-ai-assistant-made)
- [My AI assistant says it needs Plus](https://section-store.orka.help/a/my-ai-assistant-says-it-needs-plus)
- [My AI assistant says Section Store needs permission](https://section-store.orka.help/a/my-ai-assistant-says-section-store-needs-permission)
- [Bundles guide](https://section-store.orka.help/a/support-guide-bundles)
- [Cart drawer guide](https://section-store.orka.help/a/support-guide-cart-drawer)

*Also searched as:* ab test with ai assistant, a/b test with claude, ab test with chatgpt, ask ai to run ab test, split test with assistant, test bundle with ai, test cart with ai, test two discounts with ai, ai split test, assistant test results, end test with assistant, keep the winner with ai, mcp ab test, ab test over mcp, fix version b after starting, change a test after it started, can claude run an ab test, can chatgpt run an ab test