Why are Quality Survey Designs So Important?
Quality survey designs play a vital role in obtaining accurate and reliable information from respondents. A survey that is well-structured can yield valuable insights into customer satisfaction, employee engagement, product feedback, and other metrics that are essential for making informed business decisions.
Higher Response Rates
Well-designed surveys have a higher response rate and minimise the chances of non-response bias. This results in more representative survey results, which are more reliable and accurate.
Minimise Response Errors
Quality survey designs minimise response errors, such as inaccurate responses or misinterpretation of questions. Randomising questions, using skip patterns, and pretesting the survey with a small sample of respondents are effective ways to minimize these errors.
Improve Overall Data Quality
Quality survey designs improve the overall quality of data collected, which can inform business strategies and decision-making. By gathering reliable information, organizations can gain insights into their customers, employees, and stakeholders. This information can help improve products and services, increase customer satisfaction, and drive business success.
Conducting effective online research with today’s consumers requires surveys that respondents can take when they want, where they want, and on their device of choice. This means your online surveys must work well on smartphones. But how do you go about this effectively?
Online Survey Designs Best Practices
1. Design mobile first
2. Keep the survey short: 12 minutes or less
3. Avoid double-barreled questions
4. Limit the length of list options
5. Reduce repetition
6. Consider balanced scales
7. Ask meaningful questions
8. Use clear and concise language
9. Leverage open-ended questions wisely
10. Include opt-out options
11. Don't overuse mandatory fields
12. Respect respondents
13. Use responsive survey designs
14. Make it fun!
15. Critically test the survey
1. Design mobile first
To access a representative audience online, it’s important to use survey designs that are accessible to people where they are – and that’s often on smartphones. Designing for mobile devices is more than getting a question to fit on their screen and overall compatibility, it’s making the entire survey-experience enjoyable on mobile devices, so you receive the best possible results.
The best practices laid out below account for length, language, complexity and more – all of which can impact the rate at which people complete surveys, and the quality levels of data you receive back.
2. Keep the survey short: 12 minutes or less
Keeping your survey short can reduce survey dropouts. While smartphone and PC respondents alike might take surveys at home or in their office, a smartphone user’s attention is likely more limited than others. They may be interrupted by notifications from other apps, a TV programme starting, or being next in line at the checkout if they are out shopping. Whatever the reason, it is important to get respondents through the survey before they run out of time or patience.
There isn’t an exact time limit, but it is recommended to keep all surveys, especially those running on smartphones, under 12 minutes – ideally below 10. As you will see throughout our recommendations, survey design length has a large impact on keeping respondents engaged.
3. Avoid double-barreled questions
Survey questions should be focused and unambiguous. A common mistake in survey design is combining two ideas into a single question—such as, “How satisfied are you with the product’s quality and pricing?” While quality and pricing are both important, they are distinct attributes. A respondent may feel differently about each but be forced to give a single answer, resulting in unclear or misleading data.
Double-barreled questions introduce noise and reduce clarity in your results. Instead, break them apart: ask about each concept separately to give respondents the opportunity to provide more precise feedback. This small change can significantly improve the quality and usefulness of your data.
4. Limit the length of list options
Scrolling on a small screen can make it challenging for respondents to focus and follow. To avoid this, reduce the number of items shown in any list. This can be done by splitting them into logical groupings or only expanding an option list if the overarching group is selected.
Consider the purpose of your research and only include items that are relevant to your objectives. Perhaps group unlikely, low-incidence choices together into a single item.
Many survey questions include an “other, please specify” option. These can be useful, but only give the option if you will use the data collected from the open response.
5. Reduce repetition
Having to repeat yourself in a conversation is never fun. And the same goes for surveys. Many survey designs contain banks of agreement or approval scales, which can be particularly tiresome for respondents and can lead to speeding, straight-lining or random pattern answering.
This will be compounded if a respondent doesn’t have much experience with the subject (see point 6 below) or if some statements seem to overlap or duplicate each other. For example, asking a respondent if they consider a brand as high quality, then asking if they feel it is a premium brand, can be seen as overlap and be frustrating for respondents.
Try to place limits on the number of iterations of a repetitive bank of statements in survey designs and remove anything that is either not important to the research or is duplicated by other statements asked. A limit of 12 statements is a good target to aim for and will reduce noise in the data.
6. Consider balanced scales
Rating scales can be an effective way to collect feedback, but they should be used with intention. Long grids or repeated scale-based questions often feel tedious to respondents, particularly when viewed on smaller screens. This can lead to disengagement, rushed answers, or straight-lining.
If scales are used, ensure they are balanced, with an equal number of positive and negative options and a clearly defined midpoint. For example, a five-point scale ranging from “Very satisfied” to “Very dissatisfied” allows for more objective interpretation than one that leans heavily positive.
Consider how responses will be interpreted. A 7 out of 10 may represent different sentiments for different respondents. If you’re only planning to report on top-box or bottom-box scores, a simpler question format—such as a yes/no or multiple-choice response—may yield more actionable data.
Where possible, avoid overloading surveys with lengthy grids. Keep scale usage purposeful, clear, and limited to where it adds real value.
7. Ask meaningful questions
Only ask questions that respondents are qualified to answer. For example, asking detailed questions based on brand awareness might include several questions that the respondent cannot answer. Being aware of a brand does not mean they have had any exposure to that brand's customer service or delivery options, and being asked questions on these topics may stall respondents, cause them to drop out or answer without having a relevant answer – all of which will impact your final dataset.
Consider whether a familiarity or usage question could be used to filter the follow up questions or whether the question is necessary to include. This helps ensure respondents are only asked questions that make sense for them and improves the overall quality of the data collected. Avoiding assumptions about a respondent’s experience is a key part of designing thoughtful, effective surveys.
8. Use clear and concise language
Consider who is taking your survey and how best to speak to them. We recommend simplifying the language and avoiding research jargon as fundamentals to aid engaged feedback. Keeping question text worded in a short and concise manner is also important, especially on small screen devices, as shortening the text will reduce scrolling and improve your data.
It’s also important to ensure respondents can navigate through questions with ease. Add instructional text only when it’s really needed to make a task clear.
9. Leverage open-ended questions wisely
Although open-ended questions can be used to provide valuable insights, these questions are more difficult to answer on smartphones due to the small screen size and on-screen keyboards. Ask such questions only as necessary and avoid using them consecutively, as they can lead to frustration and prompt respondents to drop out of the survey.
When used strategically, open-ended questions can reveal nuances that closed-ended formats may miss. However, to avoid increasing response burden, limit their use to moments where detailed feedback is truly needed. In many cases, including a single optional open-ended question at the end such as “Is there anything else you’d like to share?”, can offer space for additional thoughts without slowing down the experience.
10. Include opt-out options
Even with careful question design, there will be times when none of the listed answers apply to a respondent’s situation. In these cases, it’s important to offer opt-out choices such as “Not applicable,” “I don’t know,” or “Prefer not to answer.”
Forcing respondents to choose from a list that doesn’t reflect their experience can lead to inaccurate or incomplete data. This is especially true for questions related to behavior, preferences, or sensitive topics.
Including an appropriate opt-out option helps respondents stay engaged and ensures the integrity of your dataset. Avoid combining opt-outs into a single, vague response, for example, “I don’t know / Not applicable”, as this can blur the meaning and limit your ability to interpret the results clearly.
11. Don't overuse mandatory fields
Making every question required might seem like a way to get complete data, but it often backfires. If someone isn’t sure how to answer or doesn’t feel comfortable with a question, forcing a response can lead to guesswork—or cause them to leave the survey entirely.
Not all questions apply to every respondent. Some may simply not have an answer, and that’s okay. Focus on making only the most important questions mandatory. The rest should allow respondents to skip if needed.
Giving people a way to move forward without answering every single question helps reduce drop-off and improves the overall quality of your data.
12. Respect your respondents
Good survey design considers more than just the data. It also considers the people providing it. Be thoughtful about how questions are asked, especially when dealing with sensitive topics like health, income, identity, or personal experiences.
Use language that is inclusive and neutral. Avoid assumptions, and make sure response options reflect a range of identities and situations. For demographic questions, this might mean offering more than two gender choices or including a "Prefer not to say" option.
Respect also means not overloading respondents. Keep the survey focused, make instructions clear, and avoid asking for information you don’t actually need. When respondents feel respected, they are more likely to stay engaged and give honest answers, which leads to stronger results.
13. Use responsive survey designs
Use scripting tools that will intelligently adapt the layout of content based on the type of question being presented and the respondent’s screen size and device orientation. Intelligent layouts will not simply shrink a question designed for a laptop to fit on a smaller device – when viewed on a smartphone, intelligent tools will respond based on the device and screen orientation to provide respondents with best possible layout. This creates a better and easier experience for respondents – one where they can be considerate with their responses rather than focus on making sense of a question or locating the “next” button.
14. Make it fun!
Ask questions in a way that connects with respondents and makes them feel at ease to answer your survey openly. Explain the purpose of the research and thank them for their consideration and involvement, especially on serious subject matter. Utilise light-hearted or humorous language, images and even memes when appropriate. These can get respondents thinking and drive engaged, honest responses.
Compelling tasks are also a valuable tool. Mini personality tests or games are not only a fun format to complete, they are also a great way to encourage respondents to give detailed and meaningful responses. You’re collecting data, but they are also benefiting by learning a little about themselves and others.
15. Critically test the survey
Finally, take the survey yourself. It’s a simple recommendation but extremely effective. When proof-reading or testing a survey link, put yourself in the shoes of the respondent. Ask yourself the following: Do all the questions make sense? Have you avoided being repetitive or boring? Are you motivated to finish the survey and provide considered answers?
If the answer to any of the above questions is a “no”, then seriously consider what could be done to make the survey more enjoyable and encourage respondents to give better answers. Rewrite or remove questions or sections that don’t work.
Improve Your Survey Designs with Kantar
Kantar is committed to working on best practice recommendations for device-agnostic survey designs, and we collaborate with groups across the industry, sharing research-on-research findings and data trends. We’ve pulled these tips together after thorough testing and years of experience conducting online research with clients. Although the focus is on engaging respondents using smartphones, broadly adopting these techniques will improve the effectiveness of your surveys and the quality of your data returns across all devices.
Keen to know more? Speak to our award-winning survey design team to learn how you can enhance your data collection from all devices. For more learning opportunities, try our Survey Design Training Modules.
Like these research tips?
Subscribe to receive a monthly research tip on online survey design, sampling, data integration and more - in addition to updates when new Survey Design Training Modules are released. Click here. if the form doesn't appear.
