Key takeaways
- Keep response lists concise and easy to scan, especially on mobile devices.
- Use randomization or rotation strategically to reduce order bias.
- Avoid long, repetitive lists that encourage satisficing and respondent fatigue.
- Ensure response options are mutually exclusive, exhaustive, and include an appropriate "out" when needed.
- Design response lists that help respondents find and select the answer that best reflects their views.
Response lists are one of the most common tools in survey design. Whether you're measuring brand perceptions, advertising reactions, behaviors, or attitudes, respondents are often asked to select from a predefined set of answer options.
But poorly designed response lists can introduce bias, increase respondent burden, and reduce data quality. Long lists encourage respondents to take shortcuts, while confusing or incomplete options can lead to inaccurate responses.
In this article, we'll explore five best practices for creating effective response lists that improve both the respondent experience and the quality of your data.
What is a response list?
Response lists present respondents with a set of predefined answer options from which they can select one or more responses.
For example:
Thinking about the ad you have just seen, which of the following describes it? Select all that apply.
- I enjoyed it
- It was memorable
- It was aimed at me
- I found it irritating
- I got bored with it
- It was difficult to understand
- I did not like the people in it
- It was relevant to me
- It improved my opinion
- None of these
Depending on the question type, respondents may be asked to select a single answer or choose all options that apply.
Common issues when writing response lists
Primacy effects and order bias
When respondents are presented with a list of response options, they tend to pay more attention to the options near the top of the list. This is known as the primacy effect.
Research shows that even relatively short lists can be affected by order bias, particularly when the response options are complex or require careful consideration. In longer lists, respondents are more likely to select options that appear first, while options in the middle or at the end may receive less attention or be overlooked entirely.
Satisficing: choosing a “good enough” answer
Think about buying a new laptop. Some people spend hours researching every available option to find the best fit for their needs. Others stop searching once they find something that's "good enough."
Respondents often behave the same way when answering surveys. Rather than carefully reading every response option, some will stop once they find an answer that reasonably reflects their view. This behavior is known as satisficing.
Satisficing becomes more likely when response lists are long, repetitive, or difficult to process. As a result, respondents may select options that appear earlier in the list simply because they encounter them first, reinforcing the primacy effect.
Fatigue effects in long statement batteries
Long batteries of statements can lead to respondent fatigue, particularly when respondents are asked to rate numerous attitudes or options using the same scale.
As fatigue sets in, respondents may pay less attention to later statements, speed through questions, or fall into repetitive answering patterns such as selecting the same response across an entire battery. This can reduce data quality and introduce bias into the results.
While the exact point at which fatigue occurs varies by respondent and topic, it's common to see engagement drop when batteries exceed around 10 to 15 statements.
Poor mobile experience
Today, most surveys are completed on smartphones, making mobile-friendly questionnaire design more important than ever. Long response lists that might seem manageable on a desktop can become frustrating on a small screen.
When respondents are forced to scroll through lengthy lists, read responses that wrap across multiple lines, or navigate crowded screens, the experience becomes more demanding. As the effort required to answer increases, respondents are more likely to skim options, miss information, satisfice, or abandon the survey altogether.
Fortunately, by following a few best practices, you can make it easier for respondents to answer consistently while improving the quality of the insights you collect.
5 tips for writing effective questionnaire response lists
Below are some practical tips for writing effective response lists in questionnaires:
1. Minimize order bias through randomization and rotation
One of the most effective ways to reduce primacy bias (respondents paying more attention to items at the top of a list) is to randomize the order of response options between respondents.
To reduce the primacy effect consider the following:
- Randomize lists of statements, feelings, brands, products, and attributes where possible.
- Keep randomized lists in the same order throughout the survey when respondents encounter them multiple times.
- Consider rotating alphabetical lists when navigation is important but order effects are still a concern.
- Avoid randomizing rating scales, demographic categories, or lists with a natural logical order.
Randomization won't eliminate order bias entirely, but it helps distribute it more evenly across answer options.
2. Keep response lists short and focused
Short and focused lists help reduce cognitive effort and make it easier for respondents to process every option carefully.
When faced with too many choices, many respondents stop searching for the best answer and instead select one that feels "good enough." To reduce this behavior consider the following:
- Remove redundant or overlapping options.
- Combine similar responses where appropriate.
- Focus on the options that are most relevant to your research objectives.
- Consider whether every answer choice is necessary.
A shorter, more focused list typically produces better-quality data than a lengthy list filled with marginally different options.
3. Design response lists for mobile devices
Most surveys are now completed on smartphones, making mobile optimization essential.
To improve usability:
- Aim for no more than 8 response options where possible.
- Treat 15 options as an upper limit before considering alternative formats.
- Avoid horizontal scrolling.
- Keep answer options concise.
- Limit statements to a maximum of two lines on mobile screens.
- Remove unnecessary words, conjunctions, and repetition.
Simple, scannable answer lists help respondents answer quickly and accurately.
4. Make response options mutually exclusive and exhaustive
Respondents should be able to find an answer that accurately reflects their situation without feeling forced into a choice.
A well-designed response list should:
- Cover the vast majority of likely answers.
- Avoid overlapping categories.
- Include appropriate escape options when needed.
Depending on the context, consider including:
- Don't know
- Not applicable
- None of these
- Other (please specify)
For questions about opinions, preferences, or behaviors, an "Other" or "None" option can help capture answers that might otherwise be missed.
5. Watch for fatigue in statement batteries
Large batteries of statements can quickly become repetitive and tiring, causing respondents to disengage. To minimize fatigue:
- Limit batteries to essential statements only.
- Break long batteries into logical topic groups.
- Randomize or rotate statement order where appropriate.
- Keep the overall number of screens manageable.
- Review whether every statement is needed to answer the research question.
The goal is to collect high-quality data, not simply to ask every question that might be interesting.
The bottom line
Response lists may seem like a small part of questionnaire design, but they have a significant impact on respondent behavior and data quality. By minimizing order bias, reducing cognitive burden, optimizing for mobile devices, providing complete answer options, and avoiding fatigue, researchers can create surveys that are easier to complete and more likely to produce reliable insights.
Want to learn more?
Looking for more in-depth guidance and survey design examples for mobile devices? Download "What does good survey design look like" using the form below.
