- Even changing one word can have a trigger different responses
- Should avoid ambiguity, but it’s near impossible to because people interpret every single word differently. 
- Ambiguity can come from having “contradictory grammatical and semantic elements” and can cause cognitive overload, therefore respondent doesn’t answer question fully or misinterprets


Question Wording

- Open-ended questions
- Close-ended questions 

Response Categories & Scales

Providing Other Response Options

An item-order effect occurs in survey research when the specific sequence in which questions are presented unintentionally influences how participants respond. Earlier items can alter a respondent's interpretation of subsequent questions or prime certain information in their memory, creating a context bias. Researchers often randomize or counterbalance the presentation of items to mitigate this effect.

Item-Order Effect

In survey research, context effects are unintended influences on a respondent's answers that arise not from the specific content of the question itself, but from the surrounding context in which the item is presented. These biases can occur when previous questions, response options, or the overall survey format alter a participant's interpretation of an item or the information they retrieve from memory.

Survey Context Effects

Survey formatting is the structural organization of a questionnaire designed to maintain respondent engagement and minimize fatigue. Beyond simply writing clear items, effective formatting involves crafting an introduction to secure informed consent, providing explicit instructions with examples, and strategically sequencing the items. Best practices include placing the most critical questions early in the survey, grouping similar items together, reserving demographic questions for the end, and always concluding with an expression of appreciation to the respondent.

Survey Formatting

Survey item order refers to the strategic sequencing of questions to maximize respondent engagement and minimize fatigue. Because participants are most attentive and least fatigued at the beginning of the survey (immediately following the introduction), researchers should place the most important substantive items early on and position less critical items later in the questionnaire.

Survey Item Order

What are common factors in survey design that can unintentionally influence participants' answers and introduce systematic biases?

Designing an effective survey is difficult because participants' answers can be unintentionally influenced by factors such as question wording, the order of items, and the available response options. These unintended influences can introduce statistical noise or create systematic biases that lead to misleading results. To maximize the reliability and validity of the data, researchers must apply specific construction principles to minimize these negative effects.

University of Michigan - Ann Arbor

Google

The goal is to create standardized questions that allow for response comparisons within or across groups. 

Constructing Survey Questionnaires

A survey is a versatile data collection tool used to gather meaningful answers about topics such as voting intentions, consumer preferences, social attitudes, or health. These self-report measures can vary in length and can be administered through multiple formats, including in-person interviews, telephone calls, mail questionnaires, or online over the Internet.

Survey

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

The mode of administration refers to how a survey is delivered to participants, which significantly impacts the overall response rate. In general, in-person interviews achieve the highest response rates, followed by telephone surveys, and then mail and internet surveys.

Survey Mode of Administration

Survey Construction Challenges

In research, a population is the entire group of individuals that the researchers seek to draw conclusions about based on their findings from the sample. 

Population

Cons of using surveys include:

- Surveys rely on self-reported data, which limits the types of information that can be collected.
- People may lie on surveys as they want to be viewed favorably.

Cons of Using Surveys

A key benefit of survey research is its efficiency in data collection. Surveys allow researchers to gather a substantial volume of information from a large sample of participants in a relatively short amount of time.

Advantages of Survey Research

An example of an online survey invitation typically begins with a polite greeting to the visitor and emphasizes the value of their opinion. It then states the survey's purpose, such as gathering feedback on a specific topic like news consumption habits, and provides an estimated completion time, for instance, 10-15 minutes. The invitation concludes with a clear call to action, instructing the user on how to start the survey, often by clicking a "Yes" or "No" button to indicate their willingness to participate.

Example of an Online Survey Invitation

When compared to case studies, surveys have the distinct advantage of collecting data from much larger samples. A large and diverse sample is more likely to accurately represent the diversity of the overall population, which significantly enhances the generalizability of the research findings. Conversely, the strength of case studies lies in their depth; surveys cannot gather the same level of detailed information on each participant as is possible through an in-depth case study.

Comparison of Surveys and Case Studies

A notable example of a comprehensive survey is the doctoral research by Ruth W. Howard. In 1934, she conducted a study by surveying 229 sets of triplets, which was the most extensive research on triplets at the time.

Ruth W. Howard's Triplet Survey

A primary advantage of the survey method is its efficiency in data collection. It allows researchers to gather large quantities of information from a substantial number of participants in a relatively short period.

Advantage of Surveys: Efficient Data Collection

A significant limitation of the survey method is that it relies on self-reporting, meaning participants do not always provide accurate responses. Individuals might intentionally lie, unintentionally misremember details, or answer questions in a way they believe makes them look good, which can compromise the reliability of the data.

Weakness of Survey Research: Reliance on Honest Self-Reporting

A drawback of surveys is that the data they yield is often relatively shallow, especially when contrasted with the in-depth information gathered through methods like case studies.

Weakness of Survey Research: Shallow Data

A team of public health researchers wants to quickly gather data on the dietary habits and attitudes towards nutrition from a large, geographically diverse sample of 5,000 adults. Which of the following data collection strategies would be the most practical and effective for achieving this specific research goal?

A major advantage of survey research is that it enables researchers to collect data from relatively large samples with ease. This capacity for widespread data collection makes it significantly easier to generalize the research findings to the broader population, especially when compared to methods that rely on smaller groups.

Advantage of Survey Research: Generalizability

In a 2012 study by Jenkins and colleagues, researchers used a survey to investigate the persistence of negative attitudes toward the US Arab-American community following the September 11, 2001 terrorist attacks. The survey included direct questions about overt prejudice and indirect questions assessing participants' willingness to socially interact with various ethnicities. Although participants directly denied holding prejudicial attitudes, their indirect responses revealed a significantly lower willingness to interact with Arab-Americans compared to other groups. This research illustrates how surveys can detect subtle forms of prejudice, even when direct self-reports are inaccurate or biased.

Example of Survey Research: Uncovering Subtle Prejudice

A sample is a smaller subset of individuals selected from a larger population of interest. Because it is usually impossible to study every member of a population, researchers measure variables within a sample with the intention of generalizing their findings back to the broader group, ideally using a representative sample that closely resembles the population.

Sample

In the context of survey research, a respondent is the term used to describe a study participant. These individuals provide the primary data for the study by directly self-reporting on their own behaviors, thoughts, beliefs, or feelings, typically through structured interviews or questionnaires.

Respondent

Survey research is a versatile methodology utilized to investigate a wide range of basic and applied research questions. It is predominantly used as a non-experimental research design to describe single variables, such as estimating the prevalence of mental disorders, or to identify statistical relationships among naturally occurring variables. Additionally, surveys can function as a data collection method within experimental research to test specific hypotheses concerning causal relationships between variables. When deployed with large and diverse samples, these survey-based experiments provide a valuable complement to traditional laboratory studies.

Applications of Surveys

Survey research is a flexible methodology that incorporates both quantitative and qualitative approaches, defined by two primary characteristics. First, it relies on self-report measures, where researchers ask participants to directly report on their thoughts, feelings, and behaviors using tools like questionnaires or interviews. Second, it emphasizes rigorous sampling techniques, typically favoring large random samples to acquire accurate estimates of population characteristics. This makes survey research one of the few approaches in psychology that routinely relies on random sampling.

Characteristics of Survey Research

The methodological roots of survey research can be traced back to English and American social surveys from the turn of the $$20$$th century, which aimed to systematically document widespread social issues such as poverty. In the $$1930$$s, the methodology progressed significantly as the United States government initiated nationwide surveys to assess social and economic conditions. The necessity to draw accurate inferences about the entire population from these surveys spurred crucial advancements in sampling techniques.

Origins of Survey Research

Following the September 11 terrorist attacks, researchers demonstrated the utility of survey research in psychology by conducting a large-scale internet survey on risk perception. The survey results indicated that participants generally tended to overestimate most risks, that females overestimated risks more than males, and that there were no differences in risk perception between teens and adults. To test their specific hypotheses about emotion, the researchers also incorporated an experimental manipulation within the survey by priming participants to feel either anger or fear. This revealed that risk perceptions are strongly tied to specific emotions; notably, individuals primed to feel anger perceived significantly less risk than those primed to feel fear.

Example of Survey Research: Emotion and Risk Perception

Researchers primarily conduct surveys using four main methods: in-person interviews, telephone surveys, mail surveys, and internet surveys. The decision of which mode to use depends on the specific goals of the research and the available budget.

Learn Before

Related

Learn After