A/B testing quizzes can be done by linking each quiz through a pop-up, then A/B testing the pop-up.
Use this guide to learn all of the steps you'll need to know to set this up.
💡 What you'll learn
How A/B testing through pop-ups works (jump to section)
Setting up the pop-up (jump to section)
Testing pop-ups (jump to section)
Comparing quiz stats (jump to section)
Frequently asked questions (jump to section)
💡 Prerequisites
At least 2 quizzes must be published on a webpage (if you know the links to each page, this can be set up before the quizzes are published).
How A/B testing through pop-ups works
Octane AI currently offers 3 types of pop-ups:
Pop-ups with a discount: a standard, opt-in only pop-up
Pop-ups without a discount: a standard, opt-in only pop-up that can be converted with a custom call-to-action (CTA)
Quiz pop-ups: a pop-up containing a quiz
This A/B testing method will be done by creating a pop-up without a discount, since quiz pop-ups can't be A/B tested directly.
This pop-up will be able to link customers to a quiz page - which we can A/B test with 2 or more different links.
Setting up the pop-up
Before following the steps below, make sure you have at least 2 links ready. You can use placeholder links if your quizzes haven't been published yet.
Check out this guide to learn how to add quizzes to a Shopify page.
📕 Creating pop-ups with a custom call-to-action (CTA)
In Octane AI, go to the Pop-ups page.
Click on "New" and under
WELCOME POP-UP, select "Without a discount".Once inside the pop-up editor, add a name for the pop-up in the left menu.
Under
OPT-IN CHANNELS, expand Email.Select "Remove email opt-in".
A new menu called Pop-up states should appear under
BEHAVIOR.Under "Submit button behavior", select one of the following options:
Go to submitted state and open a link in a new tab
Open a link in the same tab (without submitted state)
Enter the link for the 1st page that you'd like to test.
Save and exit the pop-up editor.
📕 Setting up A/B testing
Once you save your pop-up and exit, you'll be taken to the analytics page of the pop-up you just created.
On the analytics page of your pop-up, scroll down until you find the A/B Testing section.
Click on "Set-up A/B test".
Click on "Edit" for the B variation once it's created.
In the pop-up editor, find Pop-up states under
BEHAVIORon the left menu.Under "Submit button behavior", select one of the following options:
Go to submitted state and open a link in a new tab
Open a link in the same tab (without submitted state)
Enter the link for the 2nd quiz you want to test.
Save and exit the pop-up.
On the pop-up analytics page, scroll down to the A/B Testing section and click on the Weight value for the B pop-up.
Change the weight to your desired value.
Testing pop-ups
Pop-ups can be tested by setting them to active, which will make the pop-up appear on your website.
By default, this will make the pop-up appear on all pages of your website. The steps below will explain how you can make the pop-up appear on specific pages only, which lets you test it out on a low traffic page (such as your website's terms & conditions page).
📕 Accessing pop-up targeting settings
Next to the A or B pop-up, click "Edit".
In the pop-up editor, click on Targeting under
BEHAVIORin the left menu.In the left menu, under SPECIFIC PAGES find the "Show pop-up on" dropdown menu.
Select the dropdown menu and choose "Only these pages".
Click New rule set.
Leave the condition as "contains" and enter part of the URL that you want to show your test pop-up on.
For example, if the page is "https://www.my-store.com/terms-and-conditions", you can enter "terms-and-conditions".
Under
END TRIGGERin the left menu, select "Completes this pop-up". This will let you take the pop-up repeatedly for testing.Save & exit the editor.
Repeat these steps for your other pop-up.
Once your pop-up is activated, you should see it trigger on your specified page(s).
Comparing quiz stats
Analytics between A/B tested quizzes can be compared from the Quizzes page by using the compare stats feature.
Use this guide to learn more about how this works.
If you have a quiz linked in your navigation bar, or any other source (like ads), we recommend creating copies of your quiz specifically for A/B testing through pop-ups.
This will allow you to compare stats not just between your A/B tested quizzes, but also with quizzes linked through other methods.
Frequently asked questions
Where do I view stats for A/B tested quizzes?
Analytics for quizzes will be accessible in the quiz's analytics page, which you can access through the Quizzes page.
This includes any quizzes that are embedded inside of a pop-up.
Can I A/B test quiz pop-ups?
Currently, quiz pop-ups are not available for A/B testing. A/B testing is available for opt-in pop-ups, and pop-ups that use a custom CTA like the one in this guide.
Which pop-ups can I use for this A/B testing method?
You can use a welcome or exit intent pop-up for this method. Welcome pop-ups are great for general traffic, while exit intent pop-ups are more targeted towards customers that are about to leave the website.
Exit intent pop-ups trigger when someone moves their mouse off of the browser window or interacts with the browser toolbar on mobile.
How do I reset my quiz stats?
Resetting the analytics of a quiz isn't directly possible. If you're launching your quiz and want to erase your test metrics, you can create a new copy of your quiz.
Since the fresh copy will have no analytics, this can be used for the launch version of your quiz instead..