A/B testing your quiz

Learn how to set up A/B testing for your quizzes, and how to monitor performance.

A/B testing is a method used to compare two different versions of a webpage or app to determine which one performs better.

It involves presenting two versions (A and B) to different segments of your audience and analyzing which version yields better outcomes.

With Quiz Kit, your A/B test involves using two different versions of your quiz and then your customers are presented with one of version when they visit the quiz. The quiz they are presented is randomized with a 50/50 chance of either quiz being seen.

In this section

• Setting up a new A/B test

• Publishing an A/B test

• Editing an A/B test

• A/B test analytics

  • A/B test metrics

---

Setting up a new A/B test

A/B testing is a valuable tool in refining and enhancing user experiences based on empirical data and user behaviour analysis.

Using A/B testing helps you assess quiz changes like design modifications or content variations to see which version leads to increased engagement or conversion rates.

Steps:

1. From the Quiz Kit app menu, click A/B testing.

2. In the Name text field, enter a test name.

3. In the Quizzes text field, select two of your quizzes that you want to compare in the test.

4. Click Create.

---

Publishing an A/B test

When you're ready, you need to make the A/B test live to customers.

Steps:

1. From the A/B testing main page, click Publish.

2. Follow the instructions on publishing if you want to add your quiz to another location in your store.

By default, the A/B test quizzes are published on their dedicated page, following the theme you selected earlier.

You also have the same options to publish the A/B test to a section of your store, as you did with your quizzes originally. Learn more about publishing options.

---

Editing an A/B test

After an A/B test has been created and published, you can edit certain settings while the test is ongoing.

Steps:

1. From the A/B testing main page, click the ellipsis button (...) beside your quiz.

2. Do one of the following:

   1. Edit name: You can edit your A/B test's name

   2. Publish settings: Review the publication options for your A/B test

   3. Finish testing: You can end the A/B test when you're done

   4. Delete: Delete the whole A/B test and its analytics.

---

A/B test analytics

The View analytics button provides a snapshot of your test results. It displays the two quizzes that you selected to A/B test, and their key metrics.

Understanding these metrics helps you make informed decisions about which quiz to keep active and which one to unpublish.

A/B test metrics

• Engagements: The number of times a new or returning customer initiates interaction (ie the first click) with your quiz

• Completions: The number of times a customer finishes the quiz (ie gets their results)

• Abandonment: The percentage of customers that didn't complete the quiz

• Emails/phones captured: The number of email addresses and phone numbers that were input into the quiz if completed

• Revenue conversion rate: The calculation for how effectively your quizzes turn completions into orders/revenue