This guide will take you step-by-step through creating an A/B test for embedded quizzes.
💡 What you'll learn
What to know before A/B testing (jump to section)
Setting up an A/B test (jump to section)
Looking at A/B tested analytics (jump to section)
Ending an A/B test (jump to section)
💡 Prerequisites
Direct A/B testing for quizzes is available for Octane Plus plans in Octane AI. Other plans can A/B test through a linked pop-up.
What to know before A/B testing
A/B testing can be done for quizzes that are embedded into a webpage (either in the page body or fullscreen). A/B testing for quizzes published inside of a pop-up is not currently available.
At least 2 quizzes will need to be present in your account before you can create an A/B test.
❓ How are revenue & conversions attributed in Octane AI?
Octane AI tracks purchases from customers that complete a quiz with a 7 day attribution window. For more details, check out our guide on Octane AI revenue & analytics tracking.
Setting up an A/B test
Follow the video or the steps below to create an A/B test between 2 quizzes.
📕 Creating an A/B test
In the left menu, click on Quizzes.
Next, click on A/B Testing in the Quizzes submenu.
On the A/B Testing page, click on "New A/B test".
In the list of quizzes that appears, click on "Add to A/B test" next to a quiz to add it to your A/B test.
Set a weight for each A/B test. In this example, we'll use 2 quizzes and set each one at 50%.
Once your weights equal 100% total, click on "Create A/B test".
After creating an A/B test, the test will be set to Active but will need to be manually published before it appears on your website.
📕 Publishing an A/B test
Once the A/B test is created, click on "View" from the main A/B Testing page.
Under the list of quizzes, click on "Publish A/B test".
A window will appear with the embed code for this A/B test. From here, the steps to publishing an A/B test are the same as publishing an individual quiz.
For next steps, check out this guide on adding a quiz to your Shopify store.
💡 A/B tests must be published separately from individual quizzes.
Creating an A/B test will not replace any published quizzes on your website. A/B tests have their own unique embed code that will need to be added to your website to be visible.
To replace an existing quiz with an A/B test, you'll just need to replace the embed code of the original quiz with the A/B test's embed code.
Looking at A/B tested analytics
A/B tested analytics will be displayed inside of each A/B test page. Stats will be separated into 4 categories:
Graphs: comparison graphs for engagements, opt-ins, revenue & conversions.
Result page sharing stats: stats for result page URLs shared.
Top 5 products ordered: list of top 5 products ordered from a quiz result page.
Engagement stats: comparison stats for engagement metrics per question.
Analytics can be compared from before, during and after the A/B test using the date picker.
Ending an A/B test
Ending an A/B test will allow you to choose a winner, which will be the quiz that will be displayed after the test has been ended.
Any quizzes that are present inside of an A/B test will not be able to be added to another A/B test, even if the test has been ended.
To add a quiz inside of a current test (including ended tests), the current test must first be deleted. We recommend following the steps below to make sure your website quiz experience isn't disrupted.
📕 Publishing an A/B tested quiz inside of another A/B test
Click on "End test" to choose a winner of your A/B test.
Replace the A/B test embed code on your website with the embed code of the winner's quiz. You can find instructions on how to do this here.
Back in the A/B test, click on "Delete".
You can now create your new A/B test.
Deleting the A/B test will not delete any stats, but it does remove the comparison view from the A/B test that was deleted.
You can recreate the same A/B test to compare stats between quizzes from a deleted A/B test.