A/B Test your Shopify Product Page
Here’s your guide to starting your first A/B Test
1. Prepare your Banners. Next, make sure you have at least 1 Banner ready to test. You may choose to test one banner versus no banner to understand the impact of adding Because. Or you may choose to test one banner versus another - in which case you'll need two
I know you're probably thinking... what the heck do I test, so here are a few ideas to get you started:
You can test:
- 📝 Messaging (tone, voice, short/long, fun/serious, etc.)
- 🎨 Visuals (colors, fonts, icons, layout)
- 💲 Discount types ($ or %)
- 🌱 Types of content (shipping versus promotion, review versus give back)
- ⏳ BOGOS: Limited time BOGOs
- 🎄 Holiday promotions just for the day, week, or month
- 🎁 New product releases
- 🐑 Product ingredients, materials, & how it's made
- 🌎 Give back, social mission, sustainability, or charity
- 💜 Company information, brand, beliefs, & Founding story
- 📦 Out of stock messages based on when you plan to re-stock
- 🚚 Shipping delays, timelines, discounts, or offers
- 💳 Payment types, security, etc.
- 📣 News or COVID updates
- 📈 Really anything that you want to test out and understand how it drives add to carts & or sales
2. Create your test. Next, click A/B Testing in your top navigation and click "Create New A/B Test" to get started.
3. Name your A/B Test. This is just an internal name- not part of the test itself.
4. Choose your two Banners to test! The screen below will default to your most recent Banners, but you can definitely change it to whichever ones you want. Click on "Choose Banner" and it will open a modal with all of your existing Banners. If you want to test against "no banner," just scroll to the bottom of the modal on either Banner A or Banner B and choose the grey box option that says "No Banner."
5. Choose your products to run your A/B Test on. The products you test on must be the same for the two Banners to ensure consistency of data and results, but you can choose to add or change the products you target based on a starting combination of the original Banner's targeted products.
6. Make sure you make all your changes now as you can't make any changes after you publish (again, to keep the results as statistically accurate as possible😜). This includes any design edits you want to make (go back into your Banners dashboard and make edits there).
7. Next, choose your success criteria for your A/B Test. Each test you run may have a different success criteria, so feel free to change it up. The two options we offer today (more coming soon) are Add to Cart % Rate and Revenue.
The Add to Cart rate is the number of times the products for each Banner gets added to cart divided by the total number of Banner views for that Banner. Each Banner in the A/B Test will have a different Add to Cart rate and if you are measuring for this as your success criteria, you'll be able to measure which one got more add to carts.
Now I'm sure you're thinking... but what about the people who purchased? This is the next step for us in our analytics. Most Shopify apps actually don't calculate this very accurately (it's a difficult science), so we're taking the extra time to make sure we feel confident in the data we give you.
That being said, we do have a workaround if you want to start measuring your Product Page Conversion Rate and Add to Cart Conversion Rate immediately. It just requires us adding a tracking line item to your orders to ensure we know exactly which Banner (A or B) was shown to a site visitor that then enticed them to purchase. If this is of interest, just chat us below or in the app. We gotchyu 😜
8. Go Live. The last step is to confirm everything looks good and take it live!!!
9. Measure impact & turn it off. Your A/B Test will run until you manually turn it off (we'll be introducing set times in the future!). We recommend at least 1-2 weeks to get a good amount of data. If your store has less than 50 orders, you may even want to run it for at least a month.
After you determine which Banner "won" your A/B Test, you can then re-launch it to 100% of your site visitors. We are building this automated functionality soon, but for now you can do it manually😀
1. Prepare your Banners. Next, make sure you have at least 1 Banner ready to test. You may choose to test one banner versus no banner to understand the impact of adding Because. Or you may choose to test one banner versus another - in which case you'll need two
I know you're probably thinking... what the heck do I test, so here are a few ideas to get you started:
You can test:
- 📝 Messaging (tone, voice, short/long, fun/serious, etc.)
- 🎨 Visuals (colors, fonts, icons, layout)
- 💲 Discount types ($ or %)
- 🌱 Types of content (shipping versus promotion, review versus give back)
**Here are more ideas on types of content you can test:**
- ⏳ BOGOS: Limited time BOGOs
- 🎄 Holiday promotions just for the day, week, or month
- 🎁 New product releases
- 🐑 Product ingredients, materials, & how it's made
- 🌎 Give back, social mission, sustainability, or charity
- 💜 Company information, brand, beliefs, & Founding story
- 📦 Out of stock messages based on when you plan to re-stock
- 🚚 Shipping delays, timelines, discounts, or offers
- 💳 Payment types, security, etc.
- 📣 News or COVID updates
- 📈 Really anything that you want to test out and understand how it drives add to carts & or sales
2. Create your test. Next, click A/B Testing in your top navigation and click "Create New A/B Test" to get started.
3. Name your A/B Test. This is just an internal name- not part of the test itself.
4. Choose your two Banners to test! The screen below will default to your most recent Banners, but you can definitely change it to whichever ones you want. Click on "Choose Banner" and it will open a modal with all of your existing Banners. If you want to test against "no banner," just scroll to the bottom of the modal on either Banner A or Banner B and choose the grey box option that says "No Banner."
5. Choose your products to run your A/B Test on. The products you test on must be the same for the two Banners to ensure consistency of data and results, but you can choose to add or change the products you target based on a starting combination of the original Banner's targeted products.
6. Make sure you make all your changes now as you can't make any changes after you publish (again, to keep the results as statistically accurate as possible😜). This includes any design edits you want to make (go back into your Banners dashboard and make edits there).
7. Next, choose your success criteria for your A/B Test. Each test you run may have a different success criteria, so feel free to change it up. The two options we offer today (more coming soon) are Add to Cart % Rate and Revenue.
The Add to Cart rate is the number of times the products for each Banner gets added to cart divided by the total number of Banner views for that Banner. Each Banner in the A/B Test will have a different Add to Cart rate and if you are measuring for this as your success criteria, you'll be able to measure which one got more add to carts.
Now I'm sure you're thinking... but what about the people who purchased? This is the next step for us in our analytics. Most Shopify apps actually don't calculate this very accurately (it's a difficult science), so we're taking the extra time to make sure we feel confident in the data we give you.
That being said, we do have a workaround if you want to start measuring your Product Page Conversion Rate and Add to Cart Conversion Rate immediately. It just requires us adding a tracking line item to your orders to ensure we know exactly which Banner (A or B) was shown to a site visitor that then enticed them to purchase. If this is of interest, just chat us below or in the app. We gotchyu 😜
8. Go Live. The last step is to confirm everything looks good and take it live!!!
9. Measure impact & turn it off. Your A/B Test will run until you manually turn it off (we'll be introducing set times in the future!). We recommend at least 1-2 weeks to get a good amount of data. If your store has less than 50 orders, you may even want to run it for at least a month.
After you determine which Banner "won" your A/B Test, you can then re-launch it to 100% of your site visitors. We are building this automated functionality soon, but for now you can do it manually😀
Updated on: 19/11/2021
Thank you!