When people get bored taking a survey, expect to see an impact on your data in addition to early exits.
This is an important fact to be aware of. What’s equally vital is understanding what you can do in the way of survey design to reduce respondent dropout caused by boredom and increase respondent engagement.
What's the the impact of respondent fatigue on data collection?
Inconsistent answers
As respondents get bored completing surveys, their overall engagement starts to drop. They start to click less and it can worsen throughout the survey duration. This results in a lack of attention and leads to different answers at the beginning versus the end of the survey.
Neutralized responses
Answers in Likert scales and sliders become more neutralized as people get bored. Kantar ran an experiment by moving the same questions from the start of a survey to the end and compared the answers. The neutral answer or “I don’t know” responses rose by 18%.
We also saw very positive and very negative responses falling significantly, by an average of 25%. A reduction in the variation of answers also dropped as respondents got bored. This is reflected in reductions in the levels of Standard Deviation, delivering less clarity in results as less consideration is given by the respondent in their answers on a scale.
High dropout levels
Repetitive questioning can trigger significant levels of dropout. A list of 14 statements compared to six sees a difference of 10% in dropout rate (12% to two percent respectively). Furthermore, the survey enjoyment score decreases by nine percent, suggesting that fewer repetitive lists is more engaging for respondents to complete.
The longer the survey the higher the dropout rate as well. Kantar has found that a survey over 25 minutes loses more than three times as many respondents as one under five minutes. We see that more people complete shorter surveys because it’s easier to keep them engaged.
How do you reduce online survey dropout and improve overall data quality?
1. Think like a respondent
You may be reading the examples above and thinking they are common sense. Well in essence, they are.
Write your questionnaire from the perspective of person taking it. Try completing it yourself or ask your colleagues to. Taking the survey yourself will give you the understanding of how a respondent will navigate the survey and what needs to be addressed for an easier, clearer and more engaging experience.
2. Hunt out the redundant questions
One of the first things to look for in a survey that needs trimming is redundant questions: questions that give roughly the same or similar answers to other questions in the survey.
These redundancies are commonly found in banks of attitudinal statements and word-association lists. There are some sophisticated statistical ways to analyse this, but the simplest test is to run is a basic Pearson correlation. A Pearson correlation is a standard Excel function that compares the answers of every question with every other in the survey and eliminates those with high levels of correlation.
3. Make use of iconography
Generally, respondents don’t like processing long lists because it requires either memory or reading the list several times. Using iconography can alleviate those issues.
A good icon, once read, acts as a quick visual prompt that can be processed in a fraction of a second. The icons reduce the memory load of choices for the respondent, making the question much more straightforward to answer.
4. Switch to shorter, more effective question techniques
You can reduce the amount of work involved in answering questions by simply changing the way you ask the question. For example,
- Ask pre-shortening or filter questions to create less work for the respondent and to more efficiently identify the key issues. Preliminary filter question will also illuminate the most important factors.
- Use selective ranking by asking respondents to “rank the top 5 factors”. This question type saves 15 seconds on average and reduces the minimum sample requirements to roughly 300 to gain statistical clarity.
- Use choice based max differentiation by presenting small random sub-sets of say five choices and asking them to pick the most and least important factor.
5. Ask more rewarding questions
Shorter surveys will yield more engaging experiences, but so will smarter, rewarding questions. Here are a few examples;
- Switch a rating scale to a binary question. They can deliver more reliable, better differentiated answers.
- Ask respondents to participate in a gamified scenario to achieve higher concentration and thought out answers
- Use predictive techniques that ask respondents to solve a problem, requiring more brainpower that can reach far more considered and accurate answers.
Learn more
Speak to our expert team about improving your surveys using the 'contact us' button.
Find more on this topic in our Online Survey Training modules or download 12 tips for writing stronger survey questions.
If you're interested in more research tips like these, subscribe using the form below. We'll email you on the first Thursday of every month with an online survey research tip on topics like as design, engagement, sampling and data enhancement. You'll also be the first to hear when new Online Survey Training modules are released.