Surveys represent one of the most common types of social scientific research. Because survey data is usually coded in terms of numbers, it is usually considered as a type of quantitative research. Most often, survey research is conducted using a sample rather than the entire population of interest. The survey instrument is often called a questionnaire. A closely related data gathering technique commonly used by social scientists is the interview.
Interviews
Interviews consist simply of asking people questions. Interviews can be categorized by how structured they are. Social scientists often break interviews down into structured, semistructured, and unstructured. These categories describe how the interviews are conducted.
In a structured interview, the researcher decides which questions will be asked prior to the interview. In semistructured interviews, the researcher creates general categories of questions to ask during the interview. These questions are usually open-ended, and the researcher is free to probe deeper into an interesting line of questioning. Unstructured interviews are akin to a conversation. Both parties understand that the interviewer is trying to elicit information for a study, but there are no prewritten questions.
Questionnaires
Mailed questionnaires are a popular research tool. The digital analog of putting questionnaires online has also become very popular among researchers, especially when the target population is very likely to consist of technology users. In practice research such as program evaluation, agencies often use questionnaires to elicit information from clients.
Face-to-face questionnaires blur the lines between questionnaires and structured interviews. They are essentially the same thing. Group questionnaires are often used when the researcher wishes to collect information from a captive audience. These are often used to evaluate an entire group’s opinion of a training session or other group-based activity.
Designing Survey Instruments
As with most research methods, careful consideration should be given to survey design long before implementation begins. Because data analysis methods require stable datasets containing the same items for every participant, survey instruments cannot be changed once data collection has begun. Several items should be considered in order to maximize the success of a survey research project.
Participant Attitudes
When conducting survey research, it is important for the researcher to examine the probable attitudes of potential participants toward participating in the research process. For example, mailed surveys are convenient for the researcher and relatively cost effective, but consider how easily a survey can be tossed in the garbage can along with other “junk” mail. This speaks to the important issue of how many completed set of responses the researcher can obtain versus the number attempted. This is known as the response rate. A persistent problem with mailed surveys is a very low response rate. While the general population may be reluctant to fill out a survey and place it back in the mail, participants who feel a professional obligation to participate may yield a much higher response rate.
The Nature of the Questions
Different questions need to be asked in different ways. In other words, the researcher must consider the suitability of the method to the questions being asked of participants. For example, open-ended questions that require a lengthy response may not yield acceptable results with a mailed survey where the respondent must write copiously. This sort of research seems to lend itself to interviews where the respondent reflects on the answer, while the researcher records the response.
Cost
Survey research can quickly become expensive. With modern technology, the cost of getting a survey into the hands of respondents has become cheap. There is still a high cost in terms of human resources. Collecting, organizing, and analyzing data is a time consuming process. It is important to be realistic about the time required to gather a suitable amount of data for a research project. Additional interviewers can speed up the process, but more people often translate into more money.
Suitability of the Instrument
Survey research has proven to be an extremely valuable research tool for the social scientific community. Still, the method has limitations that must be recognized. The critical point to examine is whether survey responses can adequately answer the research question. The data gathered from surveys usually represents the attitudes, beliefs, and opinions of the respondents. If the research question involves these elements, then survey instruments may be appropriate. Researchers must be cautious when using survey items that represent perceptions when facts are the real concern of the study. For example, many researchers have found that people’s fear of crime is only marginally related to actual occurrences of crime in a neighborhood. Public perceptions of a crime problem seem to have more to do with disorder in the neighborhood than the rates of serious crime.
Crafting Questions
Well-constructed questionnaires are critical to the success of a survey research project. The type of questions, the content of the questions, the wording of questions, and the order of presentation of the questions should all be carefully considered. One of the first considerations for each question is whether the question will be open or closed-ended. It is worthy of note that many researchers choose a combination of the two major methods to gather information.
Closed-Ended Questions
Closed-ended questions have several advantages for the researcher because they limit the range of participants’ answers to the survey questions. In these types of questions, the respondent must choose from a set of predefined categories. These categories can be discrete such that they are easy to code into a dataset later. Closed-ended questions can take on several forms. Respondents can be asked simple yes or no questions, or the questions can be phrased in terms of true or false. Multiple choice questions are also very common. Also very common are ranking scale questions called Likert scales. Often, respondents are asked to rank their agreement with a particular statement along a continuum ranging from strongly disagree to strongly agree.
Open-Ended Questions
Open-ended questions do not provide any answer choices for the participant. They are encouraged to construct their own responses. These responses can be as simple as yes or no answers, or as complex as personal narratives describing life events. The major advantage of open-ended questions is that they do not force responses to fall into what may be artificial divisions. It is very difficult to predict all possible answers. This is especially true with nuanced and complex questions.
Question Formats
There are many potential ways to format survey questions. While response rates and data entry concerns are always present, the driving force should always be data quality. Several common questions are briefly considered below:
Rating Scales. This type of questions asks respondents to rank something on a scale. Often Likert-type scales are used for this purpose.
Ranking Scales. These types of questions ask the respondents to rank order a list of items. These types of questions are often used to gauge public perceptions about priorities (e.g., what is the biggest problem facing America today?) Because respondents will generally need to see the list to rank order the list items, these questions lend themselves to online or print survey methods.
Magnitude Estimation Scales. These types of questions ask respondents to estimate the magnitude of something (often a problem) on a particular scale. The extreme ends of the scale are often defined by the researcher to provide benchmarks for the respondent. For example, physicians often measure pain by asking patients to rate their pain with zero being no pain at all, and ten being the worst pain they have ever felt.
Split Questions. These are multipart questions where a general question is asked first, and then the researcher moves on to questions designed to clarify the response.
Funneling Questions. These are multipart questions that move from general questions (often open-ended) to very specific (often closed-ended) questions.
Wording Questions
If a researcher is engaged in crafting survey questions, the odds are that an extensive literature review has already been completed and that the researcher is somewhat of an expert in the topic being studied. This means that the researcher is intimately familiar with the vocabulary and jargon of the field. Often, respondents will not be aware of this special jargon and terminology. This disconnect between the researcher’s choice of words and the respondents’ understanding of those words can cause major problems. Some general rules can be used to aid in the process of question writing:
-
-
- Questions should be written in plain English with no technical jargon.
- Questions should be tailored to the respondents whenever possible.
- Questions should be kept short, simple, and direct.
- Questions should be kept as specific as possible to narrow the range of interpretation.
- Questions should not be unnecessarily personal when they concern sensitive issues.
-
Problems with Surveys
Surveys are only representative of a population of interest if the sample is reflective of the population. This suggests that survey researchers must pay careful attention to the rules of good sampling techniques. Another threat to the representativeness of the sample is known as nonresponse bias. Nonresponse bias refers to errors in the data caused by the systematic bias introduced by patterns of not participating in the survey. It is likely that people that did not participate in the survey will have different attitudes, opinions, and beliefs than those that did choose to participate in the survey. This is especially true when surveys depend on volunteerism and not random sampling techniques.
Another potential problem is known as response set bias. This type of bias enters the data when respondents choose a particular answer for every question, such as “strongly agree.” A partial solution to this problem is to write questions in such a way that about half the questions are phrased in a positive treatment of the topic and the other half are phrased as a negative treatment of the topic.
Modification History File Created: 07/25/2018 Last Modified: 07/25/2018
APA Citation McKee, A. J. (2019). Fundamentals of Social Research. Forma Pauperis Press. https://www.docmckee.com/WP/oer/research-contents/
This work is licensed under an Open Educational Resource-Quality Master Source (OER-QMS) License.
