Riddle Lab – Personality Quiz Question Format

Riddle Lab – Personality Test Questions

Experiment: Do simple Likert scale questions perform better or worse than situational questions and answer options in a personality quiz?

Introduction

We have conducted this experiment with our customer MORE in Common – https://www.moreincommonent.com/

MORE in Common provides coaching services to clients like the San Francisco 49ers to promote diversity and inclusion.
They use Riddle personality tests for client pre-qualification and lead generation.

Personality tests are the best-performing interactive content formats for collecting audience insights. They also show high sharing rates, and readers love to take personality tests to learn more about themselves and compare themselves to their friends and co-workers.

Unlike quizzes, personality tests do not have right or wrong answers and are, therefore, easier to complete from an audience perspective since no subject-specific knowledge or skill is required.

Creating a personality test, on the other hand – coming up with meaningful questions, answer options, and scoring – is more complex than writing a quiz. We have conducted this experiment to support your writing efforts and give you science-based recommendations on formulating your answer options to get to your test’s highest possible completion rate.

Background

Likert scale answer options are usually answer options that rank from very low to very high. All questions have the same answer choices when using Likert scale answer options.

The Likert scale is a popular rating system used in surveys to measure people’s opinions or attitudes. It typically presents a statement and asks respondents to rate their level of agreement or disagreement on a scale, usually with options such as:

1. Strongly disagree
2. Disagree
3. Neutral
4. Agree
5. Strongly agree

This scale allows individuals to express how much they agree or disagree with a particular statement, providing a simple way to quantify subjective opinions.

Assumption

Situational answer options should have higher completion rates than using the same Likert scale answer options for each question. 

Key metrics to capture

1. Completion rate for the entire personality test for each version
2. Capture the lead form completion rate in each version as a secondary metric

Experiment setup

While we typically use the same Riddle with slight variations for our experiments, this setup is not practical for this particular experiment. We created personality tests for the same psychological topic but with a slight variation in the test title and outcomes to facilitate the different styles of answer options.

The scientific version of the personality test is called: What is emotional intelligence, and why does it matter?

It was created by the expert psychologists from MORE in Common and uses these answer options for each question:

• Very untrue
• Moderately untrue
• Somewhat untrue
• Somewhat true
• Moderately true
• Very true

The personality test with the situational answer options that differ for each question was developed by the Riddle staff writers in conjunction with MORE in Common. This version of the test is called: “How rich is your emotional wealth?”
Both tests contained the same number of questions, and a form asking the participants for their name and email was shown prior to displaying the results.

We drove traffic to both versions of the test using ad copy generated by Google based on the topic. 

The headlines:

Emotional Intelligence Quiz (Likert scale answers): Unlock EI Mastery – Lead with Emotional IQ – Discover your potential

Emotional Wealth Quiz (Situational answers): Explore Emotional Wealth – Unlock your EQ Riches – Discover your potential

Ad Metrics:

Assumption and key metrics

Key metrics

The most important metric for this experiment is the percentage of people who answer all questions and complete the test.

We also measured the percentage of people who completed the lead form and the quality of these leads. Quality was defined by the email being a valid email and the contact having high relevance for our partner’s business goals (people who are interested in professional development consulting).

Assumptions

Before starting the test, we assumed that the situational answer choices would perform better and lead to a higher completion rate. We also assumed the percentage of people filling out the lead form would be about the same in both versions.

Results

Our assumptions proved wrong. The completion rate for the quiz with the Likert scale questions outperformed the situational questions.

The completion rate measured by how many people started the quiz and answered the last question was:

• Likert scale questions quiz completion rate: 68.97%
• Situational questions quiz completion rate: 35.14%

Based on the higher amounts of test finishes, the lead completion rate measured by the number of people starting the quiz and the completed lead forms is also higher in the Likert scale version:

• Likert scale questions lead completion rate: 32.75%
• Situational questions lead completion rate: 17.57%

The only metric where the situation question quiz outperformed the Likert scale quiz was the rate of people who saw the lead form and completed it.

Lead completion rate was measured as the number of people who saw the form and completed it:

• Likert scale questions form completion rate: 47.5%
• Situational questions form completion rate: 50.0%

Conclusion

Simple and easy-to-read answer options lead to higher test completion rates. Situational answers provide better answers and lead to more accurate results but adversely affect the test completion rate.

It is easier to quickly click through a test if all the answer options for all questions are always the same, as you remember the answer choices and can make a selection without reading the answer options.

The other noticeable difference was in the quality of the leads generated by the two versions of the quiz. Our partner, MORE in Common, offering professional development consulting, said that 36% of the leads from the Likert scale quiz were related to professional development and can be used to generate new business. From the leads collected for the situational answers test, which focused on emotional wealth vs. emotional intelligence, only 7% focused on professional development, and 93% focused on personal development. 

While our intention with this experiment was not to capture the quality of leads based on test subject, this is a valuable insight. The topic of your Riddle highly affects the quality of the leads you collect. 

The ads for the non-scientific quiz performed significantly better in terms of CTR and cost per click. But as the completion rate of the scientific quiz was twice as high, the cost per lead was lower for that version. 

About “More In Common”:

More In Common helps leaders and teams understand themselves better and become better leaders, thus driving better business results. We have worked with Microsoft, Google, The San Francisco 49ers, and many individual leaders across industries.  Our main goal is to help organizations find success through true inclusion and human connection and we achieve it by coaching and consulting with teams and leaders to initiate growth and productive connections. 

www.moreincommonent.com