Google

Conducting multiple independent-samples $$t$$-tests to compare every pair of group means in an experiment creates a statistical problem. While a single $$t$$-test has a $$5\%$$ chance of mistakenly rejecting a true null hypothesis (a Type I error), conducting several $$t$$-tests simultaneously causes this risk to compound. As more tests are run, the overall probability of mistakenly rejecting at least one true null hypothesis becomes unacceptably high, necessitating specialized statistical procedures to control the error rate.

Multiple Comparisons

To safely conduct post hoc comparisons without inflating the risk of making a Type I error, researchers use modified $$t$$-test procedures rather than standard $$t$$-tests. Techniques such as the Bonferroni procedure, Fisher's least significant difference (LSD) test, and Tukey's honestly significant difference (HSD) test are designed specifically for this purpose. While their mathematical details differ, their shared goal is to keep the overall risk of mistakenly rejecting a true null hypothesis at an acceptable level (typically close to $$5\%$$) across multiple comparisons.

Modified t-Test Procedures

What is the primary statistical problem that arises when conducting multiple independent-samples t-tests to compare every pair of group means in an experiment?

A researcher conducts six separate t-tests to compare every pair of means among four experimental groups, using an alpha level of .05 for each test. Because each individual test maintains only a 5% chance of a Type I error, the overall probability of making at least one Type I error across all six tests also remains at 5%.

Learn Before

Related