Experiment: AI quiz writing can generate a quiz in seconds, but an editor writing by hand gets more engagement. How much more – and is the extra effort worth it for a publisher?
Quizzes are some of the best-performing interactive content you can put on a page:
- More time on site
- More clicks per session
- A genuine reason for a reader to stick around
So the question isn’t whether to run them. For most publishers now, the real question is how to use a quiz maker to produce them at the volume modern publishing demands – and whether AI can carry that load without hurting the numbers that matter.
AI quiz writing can spin up a full quiz from a URL in seconds. A skilled editor writes a noticeably better one, but it takes real time. That’s the trade-off every content team is weighing right now, so we put actual data behind it.
The setup
We built one seven-question history quiz and produced two versions of it that were identical in every structural way:
- Same seven questions on the same topics
- Same graphics
- Same number of answer choices per question
- Same results and scoring
The only thing we changed was the writing:
- AI version – intro, question wording, answer text, and answer explanations all generated by AI
- Human version – the same elements written and polished by hand: warmer, chattier, with a bit of personality in each question stem (“Okay smarty pants, which ancient civilization built the Great Pyramids of Giza?” instead of the flat version)
Our human writer was our co-founder, Mike – 20+ years of marketing experience and, fittingly for a history quiz, a former history teacher.
We drove traffic to both and let them run.
One note on reading the data fairly – the two versions didn’t get identical traffic:
- AI version: 1,443 views
- Human version: 1,167 views
Raw finisher counts would be misleading, so every number below is normalized to a rate. That keeps the comparison like-for-like.
The results
The human-written quiz won on every engagement metric we track. (But there’s a catch…)
| Metric | AI-written | Human-written | Human advantage |
|---|---|---|---|
| Start rate (started / viewed) | 64.0% | 69.8% | +5.8 pts |
| Completion (finished / started) | 71.0% | 73.7% | +2.7 pts |
| Finish rate (finished / viewed) | 45.5% | 51.5% | +6.0 pts |
| Active time per finisher | 48.8s | 60.3s | +11.5s (+24%) |
The headline is the finish rate:
- Human version got 51.5% of viewers all the way to the end, versus 45.5% for AI
- That’s a 6-point gap, or about 13% more finishers from the same audience
- The people who took the human version also stayed engaged about a quarter longer
The pattern is consistent all the way down the funnel. The human copy:
- Pulled more people in to start
- Kept more of them through to the finish
- Held their attention longer once they were in
One detail stands out. The biggest single drop in any quiz is between question one and question two – and the human version leaked less there too (down 14% versus AI quiz writing’s 19%). The chattier, personality-led intros look like they’re doing real work: giving readers a reason to keep tapping.
A quick word on the numbers themselves. These completion rates sit below what Riddles normally reach, and that’s by design:
- We drove cold traffic to both quizzes – readers arriving standalone, with no article around them and no prior interest in the topic. That’s the toughest audience there is.
- In their natural habitat – embedded in a relevant article, reaching readers who came for the subject – quizzes complete far higher. Our 2025 Quiz Marketing Report, drawn from 3.13 billion answered questions, puts the cross-format completion average at 73.4% and the typical session at 2 minutes 48 seconds.
- The low baseline doesn’t weaken the comparison – both versions faced the same cold traffic, so the gap between them is what matters, and you’d expect the human edge to carry over on warmer, in-context traffic too.
So on a pure quality basis, the human wins clearly. But that’s only half the story for a publisher.
The AI quiz writing trade-off: time vs. quality
Here’s what the engagement numbers leave out:
- The human quiz took about 60 minutes to write and polish
- The AI version was effectively instant – paste a URL, generate, quick sanity check, done in a minute or two
That’s a roughly 30-to-1 difference in production time. Which changes the question from “which is better?” to “better at what, and what’s your bottleneck?”
Let’s run the math both ways. (These are illustrative calculations to show the shape of the decision, not platform benchmarks.)
When your bottleneck is traffic – one great slot to fill
Say you have a flagship article and one quiz slot on it, pulling 10,000 views. You’re not short on editorial time; you’re short on places to put the quiz. Here, quality is everything:
- Human quiz: 5,150 finishers, and at 60.3s each, about 86 hours of total engaged reader-time
- AI quiz: 4,550 finishers, and about 62 hours of total engaged time
For one extra hour of an editor’s time, the hand-crafted version delivers about 600 more finishers and roughly 40% more total engaged reader-time – from the exact same audience. When you have the traffic and only one shot at it, writing it by hand pays for itself easily.
When your bottleneck is editorial time – a quiz on everything
Now say you want a quiz or other interactive content (polls, mini-games like Sudoku, etc.) on every article, and you publish 50 a week:
- Hand-crafting all 50: about 50 editorial hours a week – it won’t happen, so most articles get no quiz at all
- AI: the same 50 quizzes in under two hours of setup
Even though each AI quiz converts a bit lower, the volume math isn’t close. If each of those 50 articles pulls even 1,000 quiz views:
- AI approach: about 22,750 finishers a week, from under two hours of work
- Hand-writing in that same two hours: maybe two quizzes, about 1,030 finishers
The 13% per-quiz quality edge is real, but it’s swamped by the 30-to-1 speed difference the moment your constraint is time rather than audience.
The short version:
- Hand-write your highest-value, highest-traffic quizzes, where every point of completion counts
- Let AI cover the long tail you’d otherwise never staff
- The two aren’t really competing – they solve different problems
Why a wider range of quizzes pays off
The volume argument isn’t just about filling more slots. It’s about what each extra quiz does to the page it sits on.
Most readers leave an article fast:
- Attention is getting scarcer, not more plentiful. Contentsquare’s 2026 Digital Experience Benchmark, built from 99 billion sessions across 6,500+ sites, found engagement down 10% year on year, with visitors spending less time on site and scrolling less per visit.
- A quiz interrupts that. A reader who starts a quiz tends to stay for the whole thing.
- Our 2025 Quiz Marketing Report shows interactive content holding attention roughly three times longer than a standard page.
Put a quiz on one article and you lift one page. Put a relevant quiz on every article and you lift the whole site.
That extra dwell time is where the commercial return comes from:
- More ad impressions. Every question is a chance to serve or refresh an ad on inventory you already have. A reader engaged for two or three minutes sees more than one who bounces in fifteen seconds.
- Better-quality impressions. Time in view matters, not just the impression count. IAB UK’s work on attention found attention is a stronger signal of ad impact than viewability alone, and the IAB and MRC attention guidelines note that an ad next to genuinely engaging content earns more real attention. Engaged readers are worth more per impression.
- First-party data on every topic. Each quiz answer is a declared signal – interest, opinion, knowledge level. Spread across your whole content range, that becomes a far richer audience picture than any one flagship quiz can give.
This is the case for range:
- A single great quiz lifts one story.
- A library of quizzes across your whole output turns interactive content from a special project into a standard part of every page – and compounds the dwell time, ad revenue, and data behind it.
Two of our white papers go deeper here:
- The Publisher’s Guide to Content Monetization walks through the revenue side in detail.
- The Evergreen Interactive Content white paper covers how to build quizzes that keep earning for years rather than weeks.
This is exactly where the AI-plus-human split earns its keep: humans on the flagship pieces where quality compounds most, AI filling out the range so no article goes without.
Three takeaways
- Human-written quizzes engage better, by a meaningful margin. A skilled writer’s wording lifted the finish rate by 6 points (about 13% more finishers) and drove roughly 24% more active time per reader, on an otherwise identical quiz. Words matter.
- AI quiz writing closes most of the gap for a tiny fraction of the effort. A 45.5% finish rate isn’t far off 51.5%, and it arrives in seconds instead of an hour. For the volume of content publishers need today, that speed is the whole point.
- The right call depends on your bottleneck. Traffic-limited and filling one prime slot? Hand-craft it. Time-limited and trying to cover everything you publish? AI wins on reach, and a 45.5% finish rate beats no quiz at all.
- And remember, 45.5% is a cold-traffic floor. That number comes from a test audience landing on a quiz with no article around it. Embedded in your site’s content, where readers arrive engaged, expect far higher – many publishers see 65-75% or more.
What we’re testing next: the hybrid ‘AI creates / humans polish’ model
The most interesting result here isn’t AI or human winning. It’s how close AI got – which points straight at a third option we’re running in an upcoming Lab:
Let the AI create the quiz, then have an editor spend two or three minutes polishing it, rather than writing from scratch.
The hypothesis is simple. Most of the human advantage seemed to come from a handful of small touches:
- A warmer intro
- A bit of personality in the question stems
- A sharper answer explanation
Those are quick edits on top of a working draft, not an hour of original writing. If a two-to-three-minute polish recovers even half the finish-rate gap, you’d be shipping near-human-quality quizzes at roughly 20 an hour instead of one.
We’re also testing a second route to the same goal: a saved project in Claude built on a library of our best human-written quizzes. Instead of polishing after the fact, we give the AI the expert examples up front – the tone of voice, the question-writing patterns, the intro style, the best practices our writers already use – so the first draft comes out closer to human quality. If the model learns from the people who write the best quizzes, the gap it needs closing should be smaller before an editor ever touches it.
That would be the sweet spot: most of the quality, almost none of the cost. We’ll report back with the numbers.
Try it yourself
Every format in this experiment is built into Riddle:
- Our AI quiz writing tool creates a full quiz, poll, or personality test from a URL or a topic in seconds
- Everything stays fully editable, so your team can add that two-minute human polish whenever a quiz is worth it
- It’s a full-featured free trial, no credit card needed
Create your account at riddle.com and run your own test.
Frequently asked questions
Is AI quiz writing as good as human quiz writing?
Close, but not quite. In our head-to-head test, the human-written quiz finished at 51.5% versus 45.5% for the AI version – about 13% more finishers – and held readers roughly 24% longer. AI quiz writing was still strong, just a step behind on engagement.
How much faster is AI quiz writing than writing by hand?
Dramatically. The human quiz took about 60 minutes to write and polish. AI quiz writing produced the same quiz from a URL in seconds, reviewed in a minute or two – roughly a 30-to-1 difference in production time.
Should publishers use AI quiz writing or human writers?
It depends on your bottleneck. For a flagship, high-traffic slot where every point of completion counts, hand-writing pays off. When you need a quiz on every article and editorial time is the constraint, AI quiz writing wins on reach – a good AI quiz on 50 articles beats a perfect one on the three you had time to write.
Do quizzes actually increase dwell time and ad revenue?
Yes. Readers who start a quiz tend to stay for the whole thing, and interactive content holds attention roughly three times longer than a standard page. That extra dwell time means more ad impressions on existing inventory, higher-quality impressions, and first-party data on every topic.
Why were the completion rates in this test lower than Riddle’s averages?
We drove cold traffic to both quizzes – standalone, with no article context and no prior interest in the topic. Embedded in a relevant article, Riddle quizzes complete far higher, averaging 73.4% across formats. Because both versions faced the same cold traffic, the gap between them still holds.
What is the hybrid approach to AI quiz writing?
Instead of writing from scratch or shipping raw AI output, an editor spends two or three minutes polishing an AI-generated draft. We’re also testing a Claude project trained on a library of expert human quizzes, so AI quiz writing arrives closer to human quality before any editing.
Recommended Riddle Lab experiments
- Do Quiz Timers Boost Completions and Dwell Time? – a per-question timer lifted completion by about 41%. The single setting most publishers overlook.
- How Much Do In-Quiz Ads Cost You in Completions? – the toll a single ad takes on completions, and whether placement changes it.
- Pop-up vs. Quiz Conversion Rates – why a form inside a quiz out-converts an interruptive pop-up.
- Personality Quiz Question Format – how question and answer wording changes completion, a close cousin of this study.
See the full library of tests on the Riddle Lab blog.
