In 1980, Mary Lee Smith, Gene Glass, and Thomas Miller published a comprehensive summary of hundreds of experiments that utilized random assignment to compare treatment and control conditions in psychotherapy. Their findings demonstrated that psychotherapy is generally highly effective, revealing that approximately $$80\%$$ of participants receiving treatment showed greater improvement than the average participant in a control group.

Smith, Glass, and Miller's 1980 Psychotherapy Effectiveness Study

What is the primary purpose of adding a control group to a pretest-posttest design?

To effectively rule out threats to internal validity—such as history, maturation, testing, instrumentation, regression to the mean, and spontaneous remission—researchers can add a control group to a pretest-posttest design. A control group does not receive the experimental treatment but is subjected to the same external events, passage of time, and repeated measurements as the treatment group. By comparing the posttest changes between the two groups, researchers can isolate the actual effect of the treatment from these alternative explanations, although doing so means the study is no longer a one-group design.

Google

A one-group pretest-posttest design is a quasi-experimental research method where a single group of participants is measured on a dependent variable both before and after a treatment is administered. Unlike a within-subjects experiment, this design does not counterbalance the order of conditions because the treatment must inherently occur between the initial baseline measurement (pretest) and the subsequent measurement (posttest). While an improvement in posttest scores compared to pretest scores might intuitively suggest that the treatment was effective, such conclusions are difficult to make with certainty because the absence of a control group leaves the study highly vulnerable to alternative explanations and threats to internal validity.

One-Group Pretest-Posttest Design

In research design, history serves as an alternative explanation for changes in a dependent variable, representing a significant threat to internal validity. This threat occurs when extraneous external events happen between the pretest and posttest measurements, unintentionally causing a shift in the participants' scores. For instance, if an anti-drug education program is being evaluated, an unexpected and highly publicized celebrity drug overdose occurring during the study period could independently influence participants' attitudes, confounding the results and making it impossible to determine if the education program was the true cause of the change.

History as a Threat to Internal Validity

Maturation is a threat to internal validity that occurs when participants undergo natural developmental or physiological changes—such as growing older, learning, or becoming fatigued—between the pretest and posttest measurements of a study. These natural, ongoing changes, rather than the experimental treatment itself, may be the actual cause of any observed differences in the dependent variable over time. For example, over the course of a year-long intervention, children naturally become better reasoners, which could be misconstrued as an effect of the intervention.

Maturation as a Threat to Internal Validity

The testing threat to internal validity occurs when the mere act of taking a pretest inadvertently influences a participant's responses on the subsequent posttest. Completing the initial measurement might inspire further thought, conversation, or practice regarding the topic being studied. This increased awareness or familiarity can independently cause a change in the posttest scores, regardless of whether the actual experimental treatment was effective.

Testing as a Threat to Internal Validity

Instrumentation threatens a study's internal validity when the fundamental characteristics of the measuring instrument or process change between the pretest and posttest. This issue frequently arises in observational studies where human observers are used to measure behavior, as they may gradually gain skill, become fatigued, or unconsciously alter their evaluation standards over time. Additionally, instrumentation effects can manifest when the participants themselves change their approach to the measurement tool, such as taking a survey very seriously during the novel pretest but becoming bored and less careful with their responses during the posttest.

Instrumentation as a Threat to Internal Validity

Regression to the mean acts as a severe threat to internal validity when participants are selected for a research study specifically because of their extreme initial scores. Statistically, individuals who achieve exceptionally high or low scores on a pretest are highly likely to score closer to the population average on a subsequent posttest simply due to natural, random variability. This makes it incredibly difficult to determine whether a shift in scores was genuinely caused by the experimental treatment or was merely the result of statistical regression.

Regression to the Mean as a Threat to Internal Validity

Spontaneous remission is the natural tendency for many medical conditions and psychological problems to gradually improve over time without any formal treatment or intervention. This phenomenon acts as a significant confounding variable in studies that lack a control group; researchers evaluating a treatment using a simple pretest-posttest design may falsely attribute a patient's recovery to their specific intervention, when in reality, the condition would have naturally improved on its own.

Spontaneous Remission

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

An interrupted time-series design is a quasi-experimental approach and a variant of the pretest-posttest design. In this methodology, a researcher takes a time series—multiple measurements of a dependent variable at regular intervals—both before and after a treatment is introduced. The treatment effectively 'interrupts' the ongoing measurements. Unlike the simpler pretest-posttest design, which relies on a single before-and-after measurement, the inclusion of multiple measurements allows researchers to better distinguish the actual effect of the intervention from normal variation.

Interrupted Time-Series Design

Control Group in Pretest-Posttest Designs

A practical example of a one-group pretest-posttest design involves evaluating an anti-drug education program for elementary school students. Researchers first measure the students' baseline attitudes toward illegal drugs (the pretest). Next, they implement the anti-drug curriculum over a specified period. Finally, they measure the students' attitudes again (the posttest). By comparing the pretest and posttest scores, researchers can see if attitudes changed, though the lack of a control group makes it difficult to definitively attribute any change solely to the educational program.

Example of a One-Group Pretest-Posttest Design

Why is it difficult to conclude with certainty that a treatment was effective when using a one-group pretest-posttest design?

In the context of research design, which of the following best defines 'history' as a threat to internal validity?

A major methodological advantage of the switching replication design is its built-in replication, which verifies a treatment's effectiveness across two distinct samples within a single experiment. The delayed rollout of the intervention also effectively rules out many internal validity threats, such as history effects, because it is highly unlikely that an extraneous event would happen to coincide perfectly with the treatment's introduction for the first group and later for the second. Similarly, it controls for maturation and instrumentation, as naturally occurring changes like spontaneous remission or shifts in measurement tools would likely appear simultaneously in both groups. Despite these robust controls, researchers must still account for other persistent threats, including demand characteristics, placebo effects, and experimenter expectancy.

Strengths of Switching Replication

Which of the following best describes maturation as a threat to internal validity?

Which of the following best describes the testing threat to internal validity?

Which of the following best describes how instrumentation threatens a study's internal validity?

An everyday example of regression to the mean involves an amateur bowler who typically averages a score of $$150$$. If she suddenly bowls an exceptionally high game of $$220$$, statistical probability suggests her next game will be significantly lower, regressing back toward her long-term average of $$150$$. This natural fluctuation demonstrates how extreme performances or scores are usually followed by more typical, average results, rather than indicating a permanent improvement or decline.

Example of Regression to the Mean

In a research study, what condition primarily makes regression to the mean a severe threat to internal validity?

In a 2001 meta-analysis of depression treatments, researchers Michael Posternak and Ivan Miller highlighted the powerful effect of spontaneous remission. They discovered that participants assigned to wait-list control conditions—who had not yet received any formal therapy—still showed an average improvement of $$10\%$$ to $$15\%$$ in their depression symptoms. This finding underscores the necessity of control groups to isolate true treatment effects from natural recovery.

Posternak and Miller's 2001 Spontaneous Remission Study

A wait-list control condition is an experimental setup where participants are informed they will receive the intervention, but must wait until those in the primary treatment condition have already completed it. This approach enables researchers to compare outcomes between participants who have received the treatment and those who are currently untreated but maintain the expectation that they will eventually improve.

Wait-List Control Condition

In a classic 1952 article, researcher Hans Eysenck challenged the assumed effectiveness of psychotherapy by evaluating the results of existing pretest-posttest designs. Although the original clinical studies showed patient improvement after receiving therapy, Eysenck compared these results to archival data from state hospitals and insurance companies for untreated patients. He found that both the treated and untreated groups recovered at similar rates. This highlighted that the improvements observed in the treated group were likely due to spontaneous remission rather than the psychotherapy itself, underscoring the absolute necessity of including a proper control group to accurately measure treatment efficacy.

Eysenck's 1952 Psychotherapy Effectiveness Study

A clear example of spontaneous remission is the natural recovery from the common cold. If researchers measure the symptom severity of $$100$$ people with colds, feed them chicken soup daily, and then re-evaluate them a week later, the patients will likely feel much better. However, because colds naturally run their course and improve over time without any medical intervention, this recovery is an instance of spontaneous remission rather than proof that the chicken soup cured the illness.

Learn Before

Related

Learn After