All Collections
Quiz Data
Creating a Quiz A/B Test
Creating a Quiz A/B Test

#ab-testing #analytics #quiz

Mark Baek avatar
Written by Mark Baek
Updated over a week ago

This guide will take you step-by-step through creating an A/B test for embedded quizzes.

πŸ’‘ What you'll learn

πŸ’‘ Prerequisites

Direct A/B testing for quizzes is available for Octane Plus plans in Octane AI. Other plans can A/B test through a linked pop-up.

What to know before A/B testing

A/B testing can be done for quizzes that are embedded into a webpage (either in the page body or fullscreen). A/B testing for quizzes published inside of a pop-up is not currently available.

At least 2 quizzes will need to be present in your account before you can create an A/B test.

❓ How are revenue & conversions attributed in Octane AI?

Octane AI tracks purchases from customers that complete a quiz with a 7 day attribution window. For more details, check out our guide on Octane AI revenue & analytics tracking.

Setting up an A/B test

Follow the video or the steps below to create an A/B test between 2 quizzes.

πŸ“• Creating an A/B test

  1. In the left menu, click on Quizzes.

  2. Next, click on A/B Testing in the Quizzes submenu.

  3. On the A/B Testing page, click on "New A/B test".

  4. In the list of quizzes that appears, click on "Add to A/B test" next to a quiz to add it to your A/B test.

  5. Set a weight for each A/B test. In this example, we'll use 2 quizzes and set each one at 50%.

  6. Once your weights equal 100% total, click on "Create A/B test".

After creating an A/B test, the test will be set to Active but will need to be manually published before it appears on your website.

πŸ“• Publishing an A/B test

  1. Once the A/B test is created, click on "View" from the main A/B Testing page.

  2. Under the list of quizzes, click on "Publish A/B test".

  3. A window will appear with the embed code for this A/B test. From here, the steps to publishing an A/B test are the same as publishing an individual quiz.

  4. For next steps, check out this guide on adding a quiz to your Shopify store.

πŸ’‘ A/B tests must be published separately from individual quizzes.

Creating an A/B test will not replace any published quizzes on your website. A/B tests have their own unique embed code that will need to be added to your website to be visible.

To replace an existing quiz with an A/B test, you'll just need to replace the embed code of the original quiz with the A/B test's embed code.

Looking at A/B tested analytics

A/B tested analytics will be displayed inside of each A/B test page. Stats will be separated into 4 categories:

  • Graphs: comparison graphs for engagements, opt-ins, revenue & conversions.

  • Result page sharing stats: stats for result page URLs shared.

  • Top 5 products ordered: list of top 5 products ordered from a quiz result page.

  • Engagement stats: comparison stats for engagement metrics per question.

Analytics can be compared from before, during and after the A/B test using the date picker.

Ending an A/B test

Ending an A/B test will allow you to choose a winner, which will be the quiz that will be displayed after the test has been ended.

Any quizzes that are present inside of an A/B test will not be able to be added to another A/B test, even if the test has been ended.

To add a quiz inside of a current test (including ended tests), the current test must first be deleted. We recommend following the steps below to make sure your website quiz experience isn't disrupted.

πŸ“• Publishing an A/B tested quiz inside of another A/B test

  1. Click on "End test" to choose a winner of your A/B test.

  2. Replace the A/B test embed code on your website with the embed code of the winner's quiz. You can find instructions on how to do this here.

  3. Back in the A/B test, click on "Delete".

  4. You can now create your new A/B test.

Deleting the A/B test will not delete any stats, but it does remove the comparison view from the A/B test that was deleted.

You can recreate the same A/B test to compare stats between quizzes from a deleted A/B test.

Did this answer your question?