A low completion rate means that respondents are not filling out all the information that you’ve included in the survey. If a respondent fails to complete a survey, ALL of their answers are thrown out; clients want “completes.” While some research firms may downplay the value of obtaining a high completion rate, PhiPower believes it is important and carries significant implications for the accuracy of your data and overall reliability of your research. Beyond the implications related to data, non-sampling error, and higher sampling costs, a low completion rate is a clear indicator from respondents of a poor survey experience. While there is no way to know definitively, in general, it is fair to say that when you have a low completion rate it indicates that respondents have a problem with the survey, or that they are just plain bored.
In some cases with online surveys, there may be a technical glitch that is causing respondents to drop out; be sure to check and ask (for PhiPower surveys, less than 1% answer that they encountered some sort of technical problem, and some of those use it as an opportunity to speak out) . In most cases, there is likely to be an issue with a particular question or set of questions – perhaps the organization and construction of questions is poor, creating confusion; maybe the question is too personal or sensitive; or the question may be seen as irrelevant, or even worse misleading. Typically, low completion rates can be linked to a long, boring, and tedious survey experience. Repetitious lists or grids, especially if they are text-based, are very likely to get respondents to bail (or mentally tune out) before completing.
PhiPower’s average completion rate of 95-98% can be attributed to several things.
Respondents are people — While this may sound trivial, research often tends to forget that just because a respondent has opted into the survey, doesn’t mean they’ll complete it. They lead full and busy lives and just like the rest of us, they have limited bandwidth and attention. Subsequently, capturing their attention and maintaining their focus with a neutral survey is a delicate balancing act that requires not only skill and experience, but also empathy and imagination.
Use of interactivity and rich media – PhiPower transforms linear, silent, text-based surveys into interactive experiences that integrate images, video, graphics, sound, and animation to vividly and realistically present organizations, products, and services. When respondents are engaged with rich media, interactive instruments, and behavioral simulations, they focus, enabling PhiPower surveys to elicit much deeper levels of concentration and more authentic experiences. This focus is reflected in industry-leading completion rates (95-98%), high-quality data, and, subsequently, rich guidance.
Quality assurance – Every survey should be rigorously checked in advance. Make sure the survey functions properly with a variety of browsers (old and new) and operating systems, make sure respondents are using the correct type of device, and format the survey properly so it serves correctly despite their screen setting. Avoid, for example, a need for scrolling to see material “below the fold,” and if it is absolutely necessary to present a tall grid that requires scrolling, make sure the column headings are always visible (and never scroll completely off the screen).
Deep commitment to respecting respondents – It’s not just about respecting their time and making surveys shorter and more concise. That is part of it, but it’s really about a fundamental commitment to creating and delivering a better survey experience. This is where PhiPower excels. Here are ten suggestions that stem from PhiPower’s approach that will help you increase your completion rates:
• Craft questionnaires with respect. For example, when designing a questionnaire, thank respondents for participating, be polite (“please” and “thank you” can go a long way), and generally don’t force people to answer questions; by providing “I am not sure” response options, you will probably get closer to the truth, and it can set uncertain respondents at ease.
• Respect people’s time by keeping surveys short; 10 minutes OR LESS should be considered the gold standard.
• Use a survey engine (PhiPower developed and programmed its own from scratch) so respondents do not need to wait for images and videos; Phi’s images are served immediately with a simple move of the mouse (art files are loaded in the background).
• Scales should be as simple, easy to understand, and respondent-appropriate as possible, especially when dealing with kids. (With the help of parents, PhiPower has successfully interviewed children as young as three years of age.)
• As much as possible, before designing a quantitative questionnaire, conduct qualitative research so you know the survey is tapping into key dimensions and is using appropriate language.
• Avoid, avoid, avoid repetition (seriously). While it might be confidence-instilling to ask the same question three different ways (to make sure the results are consistent), that could be costly if respondents break off, or become “careless speeders.” (To the latter, include “trap” questions, e.g., don’t only present positive attributes, to identify – and remove – “careless speeders,” and over-sample every cell to accommodate their removal.)
• Include visuals and navigation aids so stimuli are vivid, engaging, realistic, and easy to understand; PhiPower has in-house graphic artists, animators, and voice-over talent.
• Keep surveys anonymous to promote candor, and allay fears of being Spammed. Figuratively “hold respondents’ hands” if you need to ask sensitive questions; as much as possible, pose them near the end (to first gain trust and to minimize their reactivity); and, again, don’t require an answer (provide an “I would rather not say” option).
• When recruiting, tell respondents how long the survey will take, provide them with a progress bar (not all suppliers do), and provide an estimate that is longer than the median interview. When closing, thank respondents, and point out that their opinions may influence what is made available to them in the future; they should feel good about themselves and the experience.
We hope this is helpful. Please feel free to share it with your colleagues, etc. If you have questions, or would like more detail on anything here — or if you are interested in learning more about our approach, please feel free to contact us.