Conducting multiple independent-samples $$t$$-tests to compare every pair of group means in an experiment creates a statistical problem. While a single $$t$$-test has a $$5\%$$ chance of mistakenly rejecting a true null hypothesis (a Type I error), conducting several $$t$$-tests simultaneously causes this risk to compound. As more tests are run, the overall probability of mistakenly rejecting at least one true null hypothesis becomes unacceptably high, necessitating specialized statistical procedures to control the error rate.

Multiple Comparisons

In statistical analysis, what is the primary purpose of conducting post hoc comparisons after an initial test rejects the null hypothesis?

When a one-way ANOVA results in rejecting the null hypothesis, it indicates that not all population group means are equal, but it does not specify which ones differ. To pinpoint these specific differences, researchers conduct post hoc comparisons. These are follow-up statistical analyses that compare selected pairs of group means to determine exactly which groups are significantly different from each other.

Google

The one-way ANOVA is a specific type of analysis of variance used to compare the means of more than two samples ($$M_1, M_2 \ldots M_G$$) to determine if there are statistically significant differences among them. This statistical test is specifically applied within between-subjects designs where a single independent variable is manipulated across multiple independent groups.

One-Way ANOVA

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

In a one-way ANOVA, the null hypothesis asserts that the population means for all groups are perfectly equal ($$\mu_1 = \mu_2 = \ldots = \mu_G$$). In contrast, the alternative hypothesis posits that these population means are not all equal, indicating that at least one group's mean differs significantly from the rest.

Null and Alternative Hypotheses for One-Way ANOVA

The mean squares between groups ($$MS_B$$) is an estimate of population variance based on the differences among the sample means in an analysis of variance. It is calculated by dividing the sum of squares between groups by the between-groups degrees of freedom, and it serves as the numerator for the $$F$$ statistic.

Mean Squares Between Groups

The mean squares within groups ($$MS_W$$) is an estimate of population variance based on the differences among the individual scores within each group in an analysis of variance. It is calculated by dividing the sum of squares within groups by the within-groups degrees of freedom, and it serves as the denominator for the $$F$$ statistic.

Mean Squares Within Groups

In a one-way ANOVA, the shape of the $$F$$ distribution is determined by two distinct degrees of freedom ($$df$$) values. The between-groups degrees of freedom is calculated as the total number of groups minus one ($$df_B = G - 1$$). Simultaneously, the within-groups degrees of freedom is computed as the total sample size minus the number of groups ($$df_W = N - G$$).

Degrees of Freedom (One-Way ANOVA)

An ANOVA table is a standard format used to organize and report the intermediate steps and final results of an analysis of variance. This table typically displays the sources of variation (between groups and within groups), the sum of squares ($$SS$$), the degrees of freedom ($$df$$), the mean squares ($$MS$$), the calculated $$F$$ statistic, and the associated $$p$$-value.

ANOVA Table

The sum of squares between groups ($$SS_B$$) is an intermediate calculation in an analysis of variance that represents the total variation attributed to the differences among the group means. This value must be computed as a prerequisite step before finding the mean squares between groups.

Sum of Squares Between Groups

The sum of squares within groups ($$SS_W$$) is an intermediate calculation in an analysis of variance that represents the total variation based on the differences among the individual scores within each specific group. This value must be computed as a prerequisite step before calculating the mean squares within groups.

Sum of Squares Within Groups

To illustrate a one-way ANOVA, consider a health psychologist comparing the calorie estimates of three distinct groups: psychology majors, nutrition majors, and dieticians. The sample means are $$187.50$$, $$195.00$$, and $$238.13$$, respectively. Using statistical software or an ANOVA table, the researcher finds the between-groups mean square ($$MS_B$$) is $$5,971.88$$, the within-groups mean square ($$MS_W$$) is $$602.23$$, and the resulting $$F$$ ratio is $$9.92$$. With $$2$$ and $$21$$ degrees of freedom, the computed $$F$$ score ($$9.92$$) exceeds the critical value of $$3.467$$. Correspondingly, the $$p$$-value is $$.0009$$. Because this $$p$$-value is less than the standard $$.05$$ alpha level, the researcher rejects the null hypothesis and concludes that the population mean calorie estimates for the three groups are significantly different.

Example of a One-Way ANOVA

Post Hoc Comparisons

The repeated-measures ANOVA is a statistical test utilized for within-subjects designs, in which the means being compared are derived from the same participants who are tested under different conditions or at multiple times. While it shares the same fundamental logic as the one-way ANOVA, it employs a distinct mathematical approach specifically adapted to account for measuring the dependent variable multiple times for each individual.

Repeated-Measures ANOVA

The test statistic for the analysis of variance (ANOVA) is called the $$F$$ statistic. It represents a ratio of two distinct estimates of the population variance based on the sample data. Specifically, it is calculated by dividing the mean squares between groups ($$MS_B$$) by the mean squares within groups ($$MS_W$$). The mathematical formula is expressed as $$F = \frac{MS_B}{MS_W}$$. By computing this ratio, researchers can mathematically determine whether the differences observed among the sample means are larger than would be expected by random chance.

Learn Before

Related

Learn After