Although there is a general hierarchy of internal validity among research designs, there is also some overlap. For example, a poorly designed experiment that contains many confounding variables can actually have lower internal validity than a well-designed quasi-experiment that has no obvious confounding variables.

Overlap in Internal Validity Between Research Designs

Experimental, quasi-experimental, and non-experimental (correlational) research vary systematically along a spectrum of internal validity. True experimental research generally possesses the highest internal validity because it utilizes the manipulation of an independent variable and strict control over extraneous variables to confidently rule out alternative explanations. In contrast, non-experimental research ranks lowest in internal validity because it fundamentally lacks variable manipulation and control. Quasi-experimental research occupies the middle ground; it incorporates some structural features of an experiment but lacks others, such as the random assignment of participants.

Google

Internal validity indicates the degree to which an empirical study's design justifies the conclusion that changes in the independent variable directly caused the observed differences in the dependent variable. Experimental research designs typically achieve high internal validity because they involve deliberately manipulating an independent variable while rigorously controlling for extraneous variables, thereby eliminating alternative causal explanations. For instance, in the Darley and Latané experiment, the rigorous manipulation of the number of perceived bystanders isolated this independent variable as the direct cause of differences in helping behavior. In contrast, non-experimental correlational designs exhibit low internal validity; for example, a statistical relationship between regular exercise and happiness cannot confirm that exercise causes happiness, as other factors or reverse causation might explain the link.

Internal Validity

Experimental research is a method that involves a tremendous amount of control over variables of interest. It utilizes randomized assignment of participants to conditions and systematic manipulation of an independent variable to observe its effects on a measured dependent variable, with the objective of drawing causal inferences.

Experimental Research

Quasi-experimental research is a methodological design that incorporates some, but not all, of the essential features of a true experiment. While researchers using this approach may introduce a treatment or comparison condition, they frequently fail to use random assignment to place participants into distinct groups or omit counterbalancing to control for potential order effects. Consequently, while it offers more control than purely correlational studies, quasi-experimental research generally possesses lower internal validity than true experimental research.

Quasi-Experimental Research

Non-experimental research is a broad category of research methodologies defined by the lack of manipulation of an independent variable. Instead of controlling factors, researchers simply measure variables as they naturally occur in real-world or laboratory settings. Although experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, non-experimental research generally cannot. Despite this inability to draw causal conclusions, non-experimental research is still considered extremely important and is utilized in cases where experimental research cannot be carried out.

Non-Experimental Research

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

A double-blind procedure is when neither the participant nor experimenter knows what treatment or procedure is being used. This technique is utilized to void the effects of experimenter and participant expectations as well strengthen internal validity in experimental studies.


Double-Blind Procedure 

In experimental research, an extraneous variable is any factor that varies within the context of a study other than the specific independent and dependent variables being investigated. These variables pose a significant problem because many of them are likely to influence the dependent variable. If these outside factors fluctuate, they introduce competing influences that make it difficult to separate the true effect of the independent variable from the effects of the extraneous variables.

Extraneous Variable

The placebo effect refers to a positive response or improvement in an individual's condition resulting from the administration of an inert, simulated treatment. This effect is primarily driven by the individual's psychological expectation that they will get better, which can autonomously reduce stress, anxiety, and depression, subsequently altering perceptions and even improving physiological functions like the immune system.

Placebo Effect

When designing or evaluating psychological research, it is often impossible to achieve high validity across all four major domains: internal, external, construct, and statistical validity. Because these validities can share an inverse relationship, researchers must strategically prioritize some types of validity over others based on the specific goals of their study. For example, Morling (2014) notes that psychology experiments frequently prioritize high internal and construct validity by using highly controlled laboratory settings, which can inadvertently sacrifice external validity. Recognizing these necessary trade-offs is crucial, as a study with modest validity in one area is not inherently invalid, but rather presents opportunities for improvement in future follow-up research.

Prioritizing Validities

Comparison of Internal Validity Across Research Designs

* Double-blind study
* Single-blind study

Types of Experimental Research 

* Experimental hypothesis
* Experimental group
* Control group
* Confederate
* Operational definition
* Independent variable
* Dependent variable
* Random sample
* Random assignment

Basic Experimental Design Components

After an experiment's data has been gathered from both the experimental and control groups, a statistical analysis is performed to interpret the results. The purpose of this analysis is to ascertain whether any observed differences between the groups are meaningful or simply the result of random chance.

Interpreting Experimental Findings in Psychology

Many researchers in psychology want to share their findings with their peers and other scholars in the field. Generally, researchers will publish peer-reviewed journal articles. 

Reporting Research in Psychology

A significant limitation of experimental research stems from its reliance on a controlled, laboratory environment. This artificial setting raises questions about ecological validity, meaning researchers must be cautious when generalizing findings, as behaviors observed in the lab may not accurately reflect how individuals act in their natural, real-world environments.

Weakness of Experimental Research: Artificial Settings

A major limitation of the experimental method is that ethical considerations can prohibit researchers from investigating many important psychological questions. If manipulating a variable could cause harm or distress, an experiment cannot be ethically conducted, forcing researchers to use other methods.

Ethical Constraints in Experimental Research

The primary objective of an experimental study is to determine if a causal relationship exists between different factors. This involves investigating whether intentionally changing one variable results in a measurable change in another variable.

The Core Aim of Experimental Research

The main advantage of experimental research is its unique ability to determine cause-and-effect relationships. By systematically manipulating an independent variable while controlling for extraneous factors, researchers, including those in developmental psychology, can confidently make causal statements about the impact of specific variables on a given process.

Primary Strength of Experimental Research: Establishing Causality

A researcher wants to test if a new note-taking strategy improves exam performance. They teach the new strategy to their morning class and the traditional strategy to their afternoon class. At the end of the unit, the morning class scores significantly higher on the exam. The researcher concludes that the new strategy *causes* better exam performance. Which of the following statements best analyzes the validity of this conclusion?

Falsifiability is a defining characteristic of any scientific claim, meaning it must be expressed in a way that allows for potential observations that would count as evidence against it. As argued by philosopher Karl Popper, if a claim cannot be tested and potentially proven wrong by systematic observation, it does not address an empirical question and is therefore considered pseudoscientific rather than scientific.

Falsifiability

An example of an experimental design involves manipulating room lighting to examine its effects on workers' productivity. In this scenario, the lighting conditions (bright versus dim lights) represent the independent variable, while the workers' productivity is the measured dependent variable. If loud noise is present only when the lights are bright, that noise becomes a confounding variable because it varies systematically with the lighting, providing an alternative explanation for any productivity changes. However, if the noise is present equally under both lighting conditions, it is merely an extraneous variable that does not disrupt the ability to make causal conclusions about the lighting.

Example of an Experiment: Lighting and Worker Productivity

A field experiment is a type of field study where the researcher actively manipulates an independent variable in a natural setting while controlling extraneous variables as much as possible. Depending on the level of control achieved, these studies can yield high levels of both internal and external validity.

Field Experiment

An experiment is a specific type of research study designed to determine if a meaningful relationship between two variables is causal in nature. By utilizing statistical analysis to support these causal conclusions, experiments serve as one of the most common and valuable tools in psychological research.

Experiment

There are many situations where a researcher cannot actively manipulate an independent variable for practical or ethical reasons, making a true experiment impossible. For example, a researcher cannot ethically manipulate whether individuals suffer a significant early illness to study its effect on hypochondriasis. When active manipulation is not possible, researchers must investigate the relationship using nonexperimental approaches.

Inability to Manipulate Variables

In experimental research, maintaining an organized and detailed record log is crucial. Researchers typically prepare a written sequence of experimental conditions in advance. As each subject is tested in their corresponding condition, researchers should record basic demographic details, the testing date, time, and location, as well as the presiding experimenter's name in this log. It is also important to document any unusual events—such as a participant becoming confused or uncooperative—or any spontaneous questions that emerge during the session. Keeping such comprehensive notes enables researchers to accurately address future inquiries about specific sessions and to statistically analyze potential effects, such as demographic differences or variations caused by different experimenters.

Experimental Record Keeping

Survey research is a versatile methodology utilized to investigate a wide range of basic and applied research questions. It is predominantly used as a non-experimental research design to describe single variables, such as estimating the prevalence of mental disorders, or to identify statistical relationships among naturally occurring variables. Additionally, surveys can function as a data collection method within experimental research to test specific hypotheses concerning causal relationships between variables. When deployed with large and diverse samples, these survey-based experiments provide a valuable complement to traditional laboratory studies.

Applications of Surveys

A laboratory experiment is a research study conducted in a highly controlled environment. Because researchers can manipulate variables and minimize outside influences, laboratory experiments typically possess high internal validity, enabling researchers to draw strong causal conclusions.

Laboratory Experiment

Single-subject research, frequently referred to as small-$$n$$ design (where $$n$$ represents the statistical symbol for sample size), is a quantitative methodology that involves studying a small number of participants and focusing closely on each individual's behavior. Despite its name, this approach typically investigates between two and ten participants rather than just one. Serving as a crucial alternative to group research, particularly in applied areas of psychology, single-subject research seeks to comprehend objective behavior by utilizing strict experimental manipulation and control to collect highly structured data that is analyzed using quantitative techniques.

Single-Subject Research

Match each component of experimental research with its specific role or function in the study design.

An example of a quasi-experimental study is investigating the effect of a person's sex on their spatial memory. In this case, sex is the independent variable, and spatial memory is the dependent variable. Researchers can observe and compare differences between males and females, but they cannot manipulate or randomly assign a participant's sex, which prevents them from making a causal claim.

Example of a Quasi-Experimental Study: Sex and Spatial Memory

A one-group posttest only design is a quasi-experimental approach where a researcher applies a treatment or manipulates an independent variable, and then measures the dependent variable just once after the intervention. It is considered the weakest quasi-experimental design because it lacks a control or comparison group. Without this comparison, there is no way to determine what the participants' outcomes would have been had they not completed the treatment.

One-Group Posttest Only Design

A one-group pretest-posttest design is a quasi-experimental research method where a single group of participants is measured on a dependent variable both before and after a treatment is administered. Unlike a within-subjects experiment, this design does not counterbalance the order of conditions because the treatment must inherently occur between the initial baseline measurement (pretest) and the subsequent measurement (posttest). While an improvement in posttest scores compared to pretest scores might intuitively suggest that the treatment was effective, such conclusions are difficult to make with certainty because the absence of a control group leaves the study highly vulnerable to alternative explanations and threats to internal validity.

One-Group Pretest-Posttest Design

A nonequivalent groups design is a specific type of between-subjects design and a quasi-experimental approach in which researchers do not use random assignment to allocate participants to their respective conditions. Due to this absence of random assignment, the resulting comparison groups are nonequivalent, meaning they are likely to have inherent, pre-existing differences. These initial dissimilarities introduce potential confounding variables that can significantly threaten the internal validity of the research.

Nonequivalent Groups Design

An example of a quasi-experimental study lacking random assignment involves evaluating an anti-bullying program across two similar schools. A researcher implements the program in one school (the treatment group) and compares the bullying incidents to the other school (the control group). Although this design uses a comparison group, the inability to randomly assign individual students to the schools means there may be a selection effect; preexisting differences between the students in the two schools could alternatively explain any observed differences in bullying.

Example of a Quasi-Experimental Study: Anti-Bullying Program

A selection effect is a type of confounding variable that occurs when the participants in a treatment group differ from those in a control group in some systematic, pre-existing way. This issue frequently arises in quasi-experimental designs where researchers cannot use random assignment to place participants into conditions. When a selection effect is present, it provides an alternative explanation for the results, making it difficult to confidently determine whether the observed differences in the dependent variable were caused by the experimental treatment or by the inherent differences between the groups.

Selection Effect

The Tuskegee syphilis study, conducted by the US Public Health Service from 1932 to 1972, is a tragic historical example of extreme injustice in scientific research. The study involved poor African American men in Alabama who were misled into believing they were receiving treatment for 'bad blood' while actually being observed for the untreated progression of syphilis. The researchers deliberately denied the men treatment even after penicillin became the standard cure, clearly violating the ethical mandate to treat participants fairly and highlighting the urgent need to protect marginalized groups from disproportionate research risks. The profound betrayal of these men was formally acknowledged in 1997 when President Bill Clinton issued a public apology on behalf of the US government, emphasizing how the participants had been exploited without their knowledge or consent.

Tuskegee Syphilis Study

In quasi-experimental research, researchers manipulate an independent variable prior to measuring the dependent variable. This chronological sequence effectively resolves the directionality problem—a common issue in purely non-experimental studies—because the established timeline clarifies which variable is the potential cause and which is the effect.

Elimination of Directionality Problem in Quasi-Experiments

Quasi-experiments are frequently employed to assess the effectiveness of specific treatments, such as educational interventions or psychotherapeutic approaches. They are particularly valuable and common in real-world field settings where practical, logistical, or ethical constraints make it challenging or impossible to randomly assign participants to different conditions.

Applications of Quasi-Experimental Research

While quasi-experimental research manipulates an independent variable before measuring a dependent variable, it still cannot definitively establish cause-and-effect relationships. The absence of random assignment or counterbalancing means researchers cannot confidently rule out confounding variables—pre-existing differences between conditions—as the true cause of any observed outcomes.

Causal Limitations of Quasi-Experimental Research

Which of the following is a key characteristic that distinguishes quasi-experimental research from a true experiment?

Non-experimental research is the appropriate and often necessary choice when specific characteristics of a research question prevent the use of an experiment. These situations typically occur when the research focuses on describing a single variable, examines non-causal statistical associations, explores broad and subjective experiences, or investigates potential causal links where manipulating the independent variable is practically or ethically impossible.

Conditions for Non-Experimental Research

Correlational research is a widespread type of non-experimental research in which an investigator measures two variables—either continuous or categorical—and evaluates the statistical relationship (correlation) between them. Crucially, the researcher does this with little to no effort to control extraneous variables or manipulate an independent variable. It is often chosen when a relationship is not believed to be causal, or when exploring causal relationships where manipulating the independent variable is impossible, impractical, or unethical. By utilizing this strategy, researchers can achieve two fundamental scientific goals: describing the strength and direction of the relationship between the variables, and predicting scores on one variable based on the other using a statistical technique called regression.

Correlational Research

Longitudinal research is a non-experimental design in which a single group of people is followed and repeatedly assessed over an extended period as they age. While this method provides a superior means of directly studying developmental changes by avoiding cohort effects, it is heavily time-consuming and demands a substantial long-term investment from both researchers and participants.

Longitudinal Research

Cross-sectional research is a non-experimental design used to investigate changes over time by simultaneously comparing two or more pre-existing groups of people, such as individuals in different age brackets. It is classified as non-experimental because it relies on naturally occurring group differences rather than the manipulation of an independent variable or random assignment.

Cross-Sectional Research

An example of non-experimental research is measuring the number of traffic fatalities involving cell phone use over a given period, such as the past year. In this scenario, researchers merely record naturally occurring data and do not actively manipulate whether drivers use cell phones. While this observational approach allows researchers to describe patterns and make predictions about traffic accidents, the lack of manipulation means they cannot definitively conclude that cell phone use causes traffic fatalities.

Example of Non-Experimental Research: Traffic Fatalities

An example of non-experimental research involves observing a busy intersection to record drivers' genders and whether they are using a cell phone. By measuring these variables as they naturally occur without any manipulation, researchers can describe relationships—such as whether men or women are more likely to use a phone while driving—and make predictions. However, because there is no experimental manipulation, they cannot establish a causal link between gender and cell phone usage.

Example of Non-Experimental Research: Intersection Observation

Experimental and non-experimental methods are not mutually exclusive and can be utilized complementarily to investigate different aspects of the same broader topic. A researcher might initially use a non-experimental design to broadly describe a variable or behavior as it naturally occurs. Subsequently, they can employ experimental designs to systematically manipulate related factors to uncover the specific causal mechanisms driving the observed variations.

Complementary Use of Experimental and Non-Experimental Research

Observational research encompasses a variety of non-experimental studies in which behavior is systematically observed and recorded. This approach involves examining subjects without any manipulation or control of variables.

Learn Before

Related

Learn After