The statistical power of a research design is the probability that the study will reject the null hypothesis, given a specific sample size and an expected relationship strength. For instance, if a design has a power of $$.80$$, it means there is an $$80\%$$ chance of correctly rejecting the null hypothesis when the expected relationship exists in the population. It is essentially the complement of the probability of making a Type II error, so researchers strive for high statistical power to avoid missing a real effect.

Statistical Power

Critics of null hypothesis testing argue that the strict convention of rejecting the null hypothesis only when $$p < .05$$ is logically flawed. This rigid threshold creates an artificial divide where a result of $$p = .04$$ is deemed significant and worthy of publication, while a nearly identical result of $$p = .06$$ is dismissed as not significant. This arbitrariness can prevent valuable research from being published and exacerbates the file drawer problem by suppressing non-significant findings.

Arbitrariness of the p-value Threshold

Bayesian statistics is an approach to inferential statistics where a researcher specifies the prior probability that the null hypothesis and alternative hypotheses are true before conducting a study. Once the study is conducted, the probabilities are updated based on the newly collected data. This approach offers a potential alternative to traditional null hypothesis testing by providing a logical framework for quantifying uncertainty using probability distributions.

Bayesian Statistics

A Type I error is a false positive in null hypothesis testing. It occurs when a researcher mistakenly concludes that their results are statistically significant—thereby rejecting the null hypothesis—when, in reality, the null hypothesis is true and there is no actual relationship in the population. These errors happen because random sampling error can occasionally produce extreme results even when no true effect exists. The predetermined alpha level dictates the probability of committing this error when the null hypothesis is true.

Type I Error

A Type II error is a false negative in null hypothesis testing. It occurs when a researcher mistakenly concludes that their results are not statistically significant—thereby retaining the null hypothesis—when, in reality, the null hypothesis is false and a true relationship does exist in the population. In practical research, Type II errors typically happen because a study lacks adequate statistical power to detect the underlying relationship, often due to an insufficiently large sample size.

Type II Error

In the context of psychological research, which statement best defines a null hypothesis?

Statement that says there is no relationship between the two measured items

University of Michigan - Ann Arbor

Google

The one-sample $$t$$-test is a statistical procedure used to compare a single sample mean ($$M$$) against a specific, hypothetical population mean ($$\mu_0$$) that serves as an interesting standard of comparison. By defining a null hypothesis ($$\mu = \mu_0$$) and an alternative hypothesis ($$\mu \neq \mu_0$$), researchers use this test to determine if the sample data provide enough statistical evidence to conclude that the true population mean differs from the hypothesized value.

One-Sample t-Test

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

Null Hypothesis

Statement that says something is HAPPENING! New theory is true instead of an old one

Alternative Hypothesis

1. The dependent variable must be continuous (interval/ratio).
2. The observations are independent of one another.
3. The dependent variable should be approximately normally distributed.
4. The dependent variable should not contain any outliers.

Assumptions

To conduct a one-sample $$t$$-test, researchers compute the $$t$$ statistic using the exact formula: $$t = \frac{M - \mu_0}{\frac{SD}{\sqrt{N}}}$$. In this mathematical equation, $$M$$ represents the sample mean, $$\mu_0$$ is the hypothetical population mean serving as the standard of comparison, $$SD$$ is the standard deviation of the sample, and $$N$$ indicates the total sample size.

Formula for the One-Sample t-Test

The dependent-samples $$t$$-test, sometimes called the paired-samples $$t$$-test, is used to compare two means obtained from the same sample tested at two different times or under two distinct conditions. This statistical comparison is particularly appropriate for analyzing data from pretest-posttest designs or within-subjects experiments. Conceptually, it functions as a special case of the one-sample $$t$$-test.

Dependent-Samples t-Test

In a one-sample $$t$$-test, the null hypothesis posits that the true population mean ($$\mu$$) is identical to the hypothetical population mean ($$\mu_0$$), expressed formally as $$\mu = \mu_0$$. Conversely, the alternative hypothesis argues that the actual population mean significantly deviates from this expected standard of comparison, formalized as $$\mu 
eq \mu_0$$.

Null and Alternative Hypotheses for the One-Sample t-Test

When conducting a one-sample $$t$$-test, the degrees of freedom ($$df$$) are calculated by subtracting one from the total sample size ($$N - 1$$). This specific calculation dictates the exact shape of the $$t$$ distribution, enabling researchers to accurately identify the $$p$$ value associated with their test statistic.

Degrees of Freedom (One-Sample t-Test)

To demonstrate a one-sample $$t$$-test, imagine a health psychologist investigating whether university students accurately estimate the $$250$$ calories in a chocolate chip cookie. The null hypothesis ($$\mu = 250$$) assumes accurate estimation. The researcher asks a sample of $$10$$ students, resulting in a sample mean ($$M$$) of $$212.00$$ calories and a standard deviation ($$SD$$) of $$39.17$$. Plugging these statistics into the one-sample $$t$$-test formula yields a $$t$$ score of $$-3.07$$. For a two-tailed test with $$9$$ degrees of freedom, the critical value is $$\pm 2.262$$. Because the calculated $$t$$ score is more extreme than this critical value, the researcher rejects the null hypothesis and concludes that students significantly underestimate the cookie's calories. If the researcher had a strong theoretical reason to expect an underestimation, a one-tailed test with a less severe critical value of $$-1.833$$ could have been used, making it easier to reject the null hypothesis if the results trended in the expected direction.

Learn Before

Related

Learn After