Survey Question Types
This article is rated as:
We love a survey at Eval Academy! It’s one of the most commonly used data collection tools in an evaluator’s toolbox. There are lots of benefits to using a survey: you can collect lots of responses for a low cost, you can collect over time and space, you can be sure participants are asked the same question in the same way for comparisons, and more!
Have you ever stopped to think about how many question-type choices you have in creating your survey? I think most of us turn to the good ol’ Likert questions but there are so many more. Here’s a quick rundown of some options:
1. Multiple choice (or drop-down menus if online)
“Multiple choice” questions can be used when you have discrete option choices and when you want to limit write-in answers that are more resource-intensive to analyze. “Multiple choice” is also a great option when you want to force a choice, like understanding preferences.
In a “multiple choice” question, the key is that respondents choose one answer, not multiple (despite the name!). In creating a survey, if you think your choice list may not be exhaustive, you can always add an “other” category. Most online survey platforms have built-in features for this, and you can ask respondents to type in their “other” response.
In an academic world, “multiple choice” questions might be called “nominal” and probably include dichotomous questions, that is, questions that only include two answer options like “yes/no”.
Examples:
[forced choice] Which of the following activities is most important to achieving your program outcomes?
Offering the drop-in program
Operating 24/7
Doing community outreach
Partnering with schools
[other category] In which country do you reside?
Canada
Australia
United States
Other, please write in: ___________
How many times did you access this program in the last 6 months?
1 – 3 times
4 – 6 times
7+ times
I have not accessed the program in the last 6 months
2. Check all that apply
“Check all that apply” is similar to “multiple choice”. It has the same format, but respondents can select more than one answer. This is used when the answer options are not mutually exclusive.
It is important to be clear on your surveys which questions allow for multiple selections, often by adding “select all that apply” after your question.
Both “check all that apply” and “multiple choice” questions should still have response options that reflect all possibilities, which may include things like “None”, not “Not applicable”. The example below includes an exclusive answer option of indicating that services have not been used.
Example:
[with exclusive answer option] Which services have you used in the last months? (select all that apply)
Drop-in program
Lunch program
After school tutoring
I have not used any services in the last 12 months (exclusive)
3. Likert Scales
We’ve written about Likert scales before. Likert scales are used when you want to capture a spectrum of responses to assess how strongly a respondent feels about what you are measuring, like satisfaction, acceptability, etc. Likert scales can also be used to assess how frequently something happens.
The Likert scale may be viewed as an alternative to Yes/No questions by changing a dichotomous response into a spectrum, often with a middle or neutral point.
Examples:
[frequency] How often do you access the program?
Never
Rarely
Sometimes
Often
Always
[satisfaction] How satisfied are you with this program?
Very Satisfied
Somewhat Satisfied
Neither Satisfied nor Dissatisfied
Somewhat Dissatisfied
Very Dissatisfied
4. Rating scales
Ratings scales can be used in similar situations as a Likert scale, but often have more answer options (e.g., on a scale of 1 – 10 or even 0 – 100). Rating scales don’t have labels clearly attached to each marker. Without these labels it is impossible to say what the difference between a rating of 6/10 and a rating of 7/10 means but a rating scale (usually) creates more specificity and sensitivity than Likert scales because they offer more option choices. However, the interpretations of those differences are tricky. A rating scale also offers the respondent more freedom in responding without being tied to the anchors of a Likert scale.
When using a rating scale, make sure you are clear which end of the scale is positive! Respondents could circle a response, as in the first example below, or write in their rating, as in the second example.
In an academic world, this question type might be called an “ordinal” question.
Examples
[circle an option] On a scale of 1 to 10, indicate to what extent do you agree that the keynote speaker was engaging.
[write in your rating] On a scale of 0 to 100, how would you rate your health, where 0 is very ill and 100 is the best health?_______
5. Visual Analogue Scale (VAS)
Another way to get a distribution of responses is to use a VAS. In this question type you ask respondents to make a mark on a line (horizontal or vertical) that indicates how closely they feel to each end of the line, which is labelled by you. A VAS is similar to ranking, but removes the numeric labels (e.g., 1 – 10 or even 0 – 100). In this way, respondents aren’t limited to number options. The major limitation with a VAS is that it is difficult to analyze. In the past, I’ve had to find a ruler to literally measure from the end of the line to the mark, however some online platforms can do this for you. On online platforms this might be called a sliding scale, where the respondent slides the mark along a line.
A VAS is probably most frequently used in health care to measure mood, pain, health, or quality of life. They are highly sensitive to change, which means they’re great to use over multiple time points. They can be used in some populations that may have literacy difficulties, or when you are looking to minimize respondent burden because the respondent only needs to make one mark.
Again, be sure to label the ends clearly. Because a VAS is less common, many respondents won’t have encountered it before. It might be smart to include a completed example at the top of the page.
Example
Place an X on the line about how likely you are to attend this program again:
6. Ranking
Ranking questions can also be used to prioritize lists and help you to understand what has highest importance to your audience.
While potentially very informative for evaluation, I have had feedback from respondents that these are difficult questions. Ranking forces a respondent to really think about what matters most, and in some cases that can be a hard or even impossible ask. Be careful when using ranking questions that you aren’t forcing a choice that will make your respondents uncomfortable. It’s also best to keep the list as short as possible, a rule of thumb is six or fewer items. Imagine being ask to rank order 20 items!
Online platforms are fairly useful for ranking questions. Paper formats are trickier – you could ask respondents to assign a number value to each item (first example), or to draw a line between the item and a pre-ordered list (second example). If you are using this latter format, a completed example might be helpful.
Ranking is another example of an ordinal question type.
Examples
Please rank these services from most important to least important by assigning a number, where 1 is most important and 4 is least important:
Please rank these services from most important to least important, by connecting them to the list, where 1 is the most important and 4 is you’re the least important.
7. Allocation
Allocation questions are another way that you can evaluate preference or priority of items. In your question you tell your respondent that they have a resource (most commonly this would be points or virtual dollars) and you ask them to distribute, or allocate, that resource among the answer options. This mitigates the problem of the ranking question where respondents may have trouble assigning something the lowest rank if they still find it valuable or important. Using allocation, the respondent can allow for ties between items.
On online platforms you can ensure that totals add up to the allocated amount.
Examples
Imagine you have $100 distribute. How much you distribute it amongst the following charities:
You have 10 points. Distribute those points to the following school services to indicate how valuable they are to you:
8. Image choice
Image choice is when your response options aren’t words or numbers, but images. Image choice is a great option to give respondents a refreshing way to answer, or if there are literacy or cognition concerns. Images may also be good for youth and children (faces icons can also be used as part of a Likert scale). However, images can be more difficult to analyze and interpret. One way to mitigate this difficulty is by adding words with the images, but this of course negates the benefit of being word-free.
Example
[wordless] Select the image that best represents how you felt about the session today:
[with labels] Select the image that best represents where you would like to live:
9. Open ended (long or short answer)
Open ended questions are when the respondent writes in their answer. Generally, these are described as “short answer”, which is just that: the respondent may answer in just one word or maybe one sentence. “Long answer” is when you are asking for more information, perhaps an explanation of something or detailing an experience.
Technically, when you include an option in a “multiple choice” question that says “Other, please specify” that is a short answer question type. Short answer question types are good options when you don’t know how to create a list that is exhaustive, or when your exhaustive list might be too long.
Example (Short answer)
What other programs have you participated in in the last 12 months? ____________________________________________________________________________________________________________________________________________________________
Long answers, as described above, ask for detail. I often use these when I want to probe deeper into a survey question. I’ll use branching in online survey platforms so that, depending on what option a respondent chose for a previous question, a long-answer question will appear asking “Why did you select “Strongly agree” to the previous question?”.
Survey respondents get tired of long answer questions quickly. They add considerable time to completing a survey and to analyzing it. It’s best to limit these to areas that you really want to probe, where an interview or focus group isn’t an option. I usually make long answer questions on my online surveys optional.
Example (Long answer)
Please describe your main takeaway from the session today.
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
10. Matrix questions
Ok, so matrix questions aren’t actually a distinct question type. Most often, they are a series of Likert questions with a distinct format. When you have a series of questions that all use the same response option you can format them into a table (see example below).
Matrix question tables definitely save room on your paper or digital page, and they indicate to the respondent that the next set of questions are all similar, thus reducing some cognitive load in responding. However, Matrix formatting can also be overwhelming if your question list is long. It’s a general rule of thumb to limit the number of matrix questions on a survey.
One of my pet peeves is when I see a matrix question on a survey but the survey creator hasn’t offered any instruction. Just because it may seem intuitive or easy to respond, it’s still important to offer your respondents some guidance.
Example
Please place a check or X next to each statement to indicate your level of agreement.
11. Form fields
Technically form fields probably aren’t used on surveys, but on forms. However, it is possible that if you need to collect identifying information from respondents that form fields will be used. I’m not sure these are their own question type per se, as they are more likely short answer questions, but the formatting of the field provides some guidance – things like First Name, Last Name, Address etc. Most online platforms offer assistance with form fields.
12. Others!
There are other options out there that I haven’t explored, particularly around the use of mapping. Sometimes it’s fun and informative to play around with online tools to see what options they give and consider how you can use them in your survey.
Some final considerations:
Survey question type choice should be based firstly on what format best suits the information you are seeking to gain. Where several options are available to you, consider selecting the one that makes your survey shorter and/or easier to complete. Aim for simplicity.
As I’ve mentioned a few times, it is important in any question type in your survey to offer clear instructions, and clear labels.
Question type is just one decision to make in creating your survey. The wording of your questions is also important! Take a look at this infographic for more help.
I’ve presented 10+ question types here, but that doesn’t mean you should use them all in one survey. Switching question types frequently can be stressful for the respondent. Try to limit yourself to three or fewer different question types.