Survey items
Write survey items to represent the content areas of the construct specification
A foundational principle of writing good survey items is that there is a logical flow between the question and the response set. Another principle of writing survey items is that a survey should be written to measure for only ONE construct. Each survey item in a survey should assess only ONE particular topic.
When it comes to the structure of a survey item, there are two primary components: The item stem and the response set.
1. The item stem is the question or stimulus that is presented to participants which elicits a response.
2. The response set constitutes the slate or subjective continuum that participants use to respond to the question or stimulus.
1. The item stem is the question or stimulus that is presented to participants which elicits a response.
2. The response set constitutes the slate or subjective continuum that participants use to respond to the question or stimulus.
General tips for writing survey items
1. Survey items should be written to reflect the citations and allocated percentages presented in the construct specification. The items should be written so that they represent the specific content areas.
2. A good rule of thumb is to write TWICE as many items as you think you will need to cover all of the content areas in the construct specification. The underlying statistical and psychometric reasoning associated with creating surveys requires many observations to yield reliable and valid inferences. With that being said, it is also important not to write spurious or "fishing" questions that are not grounded in the literature.
3. The items should be written in general language utilized within the targeted population. If the survey is being given to the general population, use third-grade reading level language. Define or operationalize any complex terminology used in the survey. And, of course, write culturally appropriate questions and response sets.
4. Again, each item should cover only ONE topic. When an item presents several different questions or stimuli, respondents cannot be sure which one the respondent is responding to in the survey. This is known as "double-barreling" an item.
5. Avoid using language that biases items in either a positive or negative fashion. Also, use the same tense in all items. If at all possible, use indirect language that will allow for varied responses across the population of interest.
6. Researchers must have a working mastery of the literature area so that they can write appropriate questions that will effectively assess the construct and its content areas. This mastery will allow for researchers to state your items in relative terms reflecting the literature and population of interest.
7. Give instructions for each section of a survey that uses different styles of response sets. Researchers may surmise a logical flow from using Likert scales in one section of a survey and then switching to checking boxes in another section, but respondents may not be able to make the same "leap." It is also pertinent to remind participants what response set is being used on the page. Truncated instructions can also be included at the top of each page as needed. Finally, if there are multiple pages in the survey, put a reminder at the bottom that tells the respondent to "turn the page over" or "continue to the next page."
8. Include a "not appropriate," "I don't know," "neutral," or "choose not to answer" option as needed within the context of the construct and research question. Interesting serendipitous findings can be yielded when accounting for this unique variance within a population.
9. Items can be written in the form of statements that act as catalysts for eliciting a response from participants. Participants then give a subjective rating related to the statement in a response set. Items can also be written in a contextual or situational manner. Lastly, some items should be written to assess the frequency, duration, and intensity of the construct. These are contexts that are easily applicable and interpretable when conducting survey research.
10. Choose a cognitive, emotional, situational, or perceptual context to frame the item stems. Use words and phrases that will act as catalysts for realistic reactions from respondents:
2. A good rule of thumb is to write TWICE as many items as you think you will need to cover all of the content areas in the construct specification. The underlying statistical and psychometric reasoning associated with creating surveys requires many observations to yield reliable and valid inferences. With that being said, it is also important not to write spurious or "fishing" questions that are not grounded in the literature.
3. The items should be written in general language utilized within the targeted population. If the survey is being given to the general population, use third-grade reading level language. Define or operationalize any complex terminology used in the survey. And, of course, write culturally appropriate questions and response sets.
4. Again, each item should cover only ONE topic. When an item presents several different questions or stimuli, respondents cannot be sure which one the respondent is responding to in the survey. This is known as "double-barreling" an item.
5. Avoid using language that biases items in either a positive or negative fashion. Also, use the same tense in all items. If at all possible, use indirect language that will allow for varied responses across the population of interest.
6. Researchers must have a working mastery of the literature area so that they can write appropriate questions that will effectively assess the construct and its content areas. This mastery will allow for researchers to state your items in relative terms reflecting the literature and population of interest.
7. Give instructions for each section of a survey that uses different styles of response sets. Researchers may surmise a logical flow from using Likert scales in one section of a survey and then switching to checking boxes in another section, but respondents may not be able to make the same "leap." It is also pertinent to remind participants what response set is being used on the page. Truncated instructions can also be included at the top of each page as needed. Finally, if there are multiple pages in the survey, put a reminder at the bottom that tells the respondent to "turn the page over" or "continue to the next page."
8. Include a "not appropriate," "I don't know," "neutral," or "choose not to answer" option as needed within the context of the construct and research question. Interesting serendipitous findings can be yielded when accounting for this unique variance within a population.
9. Items can be written in the form of statements that act as catalysts for eliciting a response from participants. Participants then give a subjective rating related to the statement in a response set. Items can also be written in a contextual or situational manner. Lastly, some items should be written to assess the frequency, duration, and intensity of the construct. These are contexts that are easily applicable and interpretable when conducting survey research.
10. Choose a cognitive, emotional, situational, or perceptual context to frame the item stems. Use words and phrases that will act as catalysts for realistic reactions from respondents:
11. Choose a response set that logically flows with the item stem. Here are some popular 5-point Likert scales:
Likert scales
Click on the Step 5: Survey Pretest button to continue.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics