Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of cognition, 1(1). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6646942/

University of Michigan - Ann Arbor

The statistical power of a research design is the probability that the study will reject the null hypothesis, given a specific sample size and an expected relationship strength. For instance, if a design has a power of $$.80$$, it means there is an $$80\%$$ chance of correctly rejecting the null hypothesis when the expected relationship exists in the population. It is essentially the complement of the probability of making a Type II error, so researchers strive for high statistical power to avoid missing a real effect.

Statistical Power

Power analysis and effect size in mixed effects models: A tutorial

A Type II error is a false negative in null hypothesis testing. It occurs when a researcher mistakenly concludes that their results are not statistically significant—thereby retaining the null hypothesis—when, in reality, the null hypothesis is false and a true relationship does exist in the population. In practical research, Type II errors typically happen because a study lacks adequate statistical power to detect the underlying relationship, often due to an insufficiently large sample size.

Type II Error

If a research design lacks adequate statistical power, researchers can increase it by either increasing the sample size or strengthening the relationship. Strengthening the relationship can be achieved by using a stronger experimental manipulation or by carefully controlling extraneous variables to reduce data noise, such as employing a within-subjects design instead of a between-subjects design. However, the most common and practical strategy to boost power is simply to increase the sample size.

Increasing Statistical Power

Because manual computation is complex, researchers typically compute the statistical power for various research designs using online tools and downloadable programs, such as Russ Lenth’s Power and Sample Size Page or G*Power. By inputting the expected relationship strength, the sample size, and the $$\alpha$$ level, these applications calculate the study's power or determine the exact sample size required to achieve a desired power level, such as $$.80$$.

Computing Statistical Power Online

Statistical power and the probability of committing a Type II error are perfectly complementary. A Type II error occurs when a researcher fails to reject a false null hypothesis (missing a real effect). Since statistical power is the probability of correctly rejecting that false null hypothesis, statistical power is calculated as $$1 - 	ext{probability of a Type II error}$$. Therefore, maximizing a study's statistical power is the primary method for minimizing the risk of making a Type II error.

Relationship Between Statistical Power and Type II Error

In psychological research, it is generally recommended that a study's design possess adequate power prior to data collection. A widely accepted guideline is that an adequate level of statistical power is $$.80$$. This indicates that if the expected effect or relationship truly exists in the population, the study has an $$80\%$$ chance of successfully detecting it and rejecting the null hypothesis.

Adequate Statistical Power

To achieve high statistical power in psychological research, sample sizes must be larger than often realized. For an independent-samples $$t$$-test with a large population effect, it requires approximately $$26$$ participants per sample to reach a power of $$0.80$$, while reaching a power of $$0.99$$ requires $$59$$ participants per sample.

Example of Sample Size Requirements for High Statistical Power

Statistical power is heavily dependent on the expected relationship strength (effect size). Weaker relationships are harder to detect and thus provide lower statistical power for a given sample size. To compensate and achieve adequate power when investigating a weak relationship, researchers must utilize very large sample sizes.

Relationship Between Statistical Power and Relationship Strength

Which of the following best defines the statistical power of a research design?

A researcher designs a study with a statistical power of .80. This means there is a 20% probability that the study will fail to detect a real effect that actually exists in the population.

Learn Before

Related