The one-sample $$t$$-test is a statistical procedure used to compare a single sample mean ($$M$$) against a specific, hypothetical population mean ($$\mu_0$$) that serves as an interesting standard of comparison. By defining a null hypothesis ($$\mu = \mu_0$$) and an alternative hypothesis ($$\mu \neq \mu_0$$), researchers use this test to determine if the sample data provide enough statistical evidence to conclude that the true population mean differs from the hypothesized value.

One-Sample t-Test

The dependent-samples $$t$$-test, sometimes called the paired-samples $$t$$-test, is used to compare two means obtained from the same sample tested at two different times or under two distinct conditions. This statistical comparison is particularly appropriate for analyzing data from pretest-posttest designs or within-subjects experiments. Conceptually, it functions as a special case of the one-sample $$t$$-test.

Dependent-Samples t-Test

The independent-samples $$t$$-test is a statistical procedure utilized to determine if there is a significant difference between the means of two distinct groups ($$M_1$$ and $$M_2$$). This analytical method is appropriate when the two samples are tested under varying conditions within a between-subjects experimental design, or when researchers are comparing pre-existing groups (such as extraverts and introverts) in a cross-sectional research design.

Independent-Samples t-Test

When presenting the outcomes of a $$t$$-test in a research manuscript, specific APA style formatting conventions must be followed. The results are typically written in the format: $$t(df) = \text{t-score}, p = \text{p-value}$$. For instance, a result might be reported as $$t(9) = -3.07, p = .01$$. In this structure, the letters $$t$$ and $$p$$ must be italicized. The degrees of freedom ($$df$$) are placed in parentheses immediately following the $$t$$ symbol, and must be presented as a whole number without any decimal remainder. Finally, both the calculated $$t$$ score and the $$p$$-value are rounded to two decimal places.

Reporting t-Test Results in APA Style

The analysis of variance (ANOVA) is the most common null hypothesis test used when researchers need to compare the means of more than two groups or conditions. It serves as a necessary extension of the $$t$$-test, which is strictly limited to evaluating differences between exactly two means (such as a single sample mean against a population mean, or the means of two distinct groups).

Analysis of Variance

The $$t$$ distribution illustrates the spread of possible $$t$$ scores that researchers would expect to observe if the null hypothesis were entirely true. This distribution forms a unimodal and symmetrical curve centered precisely around a mean of $$0$$. By utilizing this known distribution curve, researchers can evaluate any computed $$t$$ score to find its exact $$p$$-value, which represents the proportion of expected scores that are at least as extreme as the observed sample result.

Distribution of t Scores

What is the primary statistical function of a t-test in psychological research?

The $$t$$-test is the most frequently used null hypothesis test for evaluating the statistical relationship between two means. There are three primary variations of this procedure—the one-sample $$t$$-test, the dependent-samples $$t$$-test, and the independent-samples $$t$$-test—each designed to accommodate slightly different research designs.

Google

University of Michigan - Ann Arbor

Method of fitting regression models in which the choice of predictive variables is carried out by an automatic procedure - adding or subtracting variables at each step depending on the results of predetermined tests 
Examples of tests that could be used:
1. F-tests
2. t-tests
3. R Squared

Stepwise Model Selection

The fundamental purpose of null hypothesis testing is to provide researchers with a formal framework to decide between two distinct interpretations of their sample data. It helps determine whether an observed statistical relationship accurately reflects a genuine relationship in the broader population or if it is simply the misleading result of random sampling error.

Purpose of Null Hypothesis Testing

The mean is a measure of central tendency representing the average of a distribution, symbolized as $$M$$. It is calculated by finding the sum of all the scores and dividing by the total number of scores. The mean is widely used because it provides a reliable indication of central tendency, is easy to understand, and has properties that are valuable for inferential statistics.

Mean

Null hypothesis testing is the most common approach to inferential statistics in psychological research. It provides researchers with a formal, structured process to evaluate whether an observed sample result likely occurred by random chance or whether it accurately reflects a genuine relationship or difference in the broader population.

Null Hypothesis Testing

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

Used for the purpose of comparing statistical models
Test statistic must have an f-distribution under the null hypothesis.
Example of application: finding the model that best fits the population from which the data were sampled.

F-test

Statistical measure of how close the data are to the fitted regression line.
r - correlation coefficient
r squared - coefficient of determination


R Squared

t-Test

A major criticism of null hypothesis testing is that it is fundamentally uninformative. Because the null hypothesis posits that there is precisely zero relationship between variables in the population, critics argue that it is never literally true in the real world. Consequently, rejecting the null hypothesis merely confirms that some nonzero relationship exists, which provides researchers with very little precise information about the actual strength or nature of the phenomenon being studied.

Informativeness of Null Hypothesis Testing

Despite widespread criticisms, some statisticians and researchers strongly defend null hypothesis testing when it is understood and applied correctly. A primary argument in its defense is that it provides a standardized, principled method for researchers to demonstrate that their findings are not merely the result of random chance. This framework is considered especially useful and necessary when investigating new or unestablished scientific phenomena.

Defense of Null Hypothesis Testing

The formal process of null hypothesis testing relies on a specific sequence of logical steps to decide between interpretations of a statistical relationship. First, researchers temporarily assume that the null hypothesis is true (i.e., there is no relationship in the population). Second, they determine the probability of obtaining their sample relationship under this assumption. Finally, if the sample result is extremely unlikely given a true null hypothesis, they reject the null hypothesis in favor of the alternative hypothesis; if the result is not extremely unlikely, they retain the null hypothesis.

General Logic of Null Hypothesis Testing

What is the fundamental purpose of null hypothesis testing?

A key characteristic of the mean is its high sensitivity to extreme scores, or outliers. In highly skewed distributions, the mean is pulled away from the median in the direction of the skew (the longer tail). Because extreme scores heavily influence its value, the mean may cease to be an accurate representation of the typical score in such datasets, leading researchers to often prefer the median for skewed distributions.

Sensitivity of the Mean to Outliers

Compared to other measures of central tendency, the mean possesses statistical properties that make it uniquely useful for conducting advanced inferential statistics. However, researchers must consider its sensitivity to outliers before relying on it exclusively.

Usefulness of the Mean in Statistical Analyses

The mathematical formula for the mean of a distribution is: $$M = \frac{\Sigma X}{N}$$. In this equation, $$M$$ represents the mean, $$\Sigma$$ (the Greek letter sigma) is the summation sign indicating to add all values of the variable $$X$$, and $$N$$ represents the total number of scores in the distribution.

Statistical Mean Formula

A hypothetical population mean, symbolized mathematically as $$\mu_0$$, serves as a specific, interesting standard of comparison in statistical research. It represents an expected baseline value for an entire population, against which researchers can compare an actually observed sample mean ($$M$$) to draw statistical conclusions.

Hypothetical Population Mean

The mathematical formula for the mean (symbolized as $$M$$) is $$M = \Sigma X / N$$. In this equation, $$\Sigma$$ (the Greek letter sigma) indicates the summation across all values of the variable $$X$$, and $$N$$ represents the total number of scores in the distribution.

Formula for the Mean

A $$z$$ score indicates how many standard deviations a particular raw score is above or below the mean of its distribution. It standardizes the distance of a score from the mean, providing a consistent way to describe a score's relative location.

z Score

The standard deviation is the most widely used measure of variability, representing the average distance that individual scores within a distribution deviate from the mean.

Standard Deviation

Which measure of central tendency is calculated by finding the sum of all scores in a distribution and dividing that sum by the total number of scores?

In null hypothesis testing, researchers make one of two formal decisions (to reject or retain the null hypothesis) concerning one of two possible states of reality (the null hypothesis is actually true or false). This intersection creates a matrix of four distinct possible outcomes. Two outcomes are correct decisions: rejecting a false null hypothesis and retaining a true one. The remaining two outcomes are statistical errors: rejecting a true null hypothesis (Type I error) and retaining a false null hypothesis (Type II error).

Outcomes of a Null Hypothesis Test

In 2015, the editors of the journal Basic and Applied Social Psychology took the unprecedented step of banning the use of null hypothesis testing and $$p$$-values in its publications. Instead of relying on traditional statistical significance thresholds to evaluate validity, the editors required authors to focus heavily on descriptive statistics and effect sizes. Although this ban was not widely adopted across the field, it significantly amplified the ongoing conversation in psychology about the limitations of the 'gold standard' of statistical validity.

The 2015 Ban on Null Hypothesis Testing

A quantitative measure, such as Cohen's $$d$$ or Pearson's $$r$$, used to estimate the strength of a statistical relationship within a population. In response to criticisms of null hypothesis testing, researchers are encouraged to report effect sizes alongside their tests. This practice is crucial because a $$p$$-value alone only indicates whether a relationship is statistically significant—which is heavily influenced by sample size—but fails to convey the actual magnitude or importance of that relationship.

Effect Size

A statistical range calculated around a sample statistic designed to contain the true population parameter a specified percentage of the time, most commonly $$95\%$$. Some researchers advocate using confidence intervals as an alternative to traditional null hypothesis testing because they are often more intuitive to interpret. Furthermore, a confidence interval inherently provides the data needed to perform a null hypothesis test; if a hypothetical population mean falls completely outside the calculated interval, the sample mean is considered statistically significantly different from that hypothetical value.

Confidence Interval

In null hypothesis testing, researchers reject the null hypothesis in favor of the alternative hypothesis if the observed sample relationship would be extremely unlikely to occur under the assumption that the null hypothesis is true. This decision indicates there is sufficient statistical evidence to conclude that a relationship exists in the population.

Reject the Null Hypothesis

The $$t$$-test is the most commonly used null hypothesis test for evaluating statistical relationships that focus on the difference between two means.

$$t$$-Test

Null hypothesis testing is subject to several major criticisms, including frequent misunderstandings of results by researchers, the arbitrary nature of the rigid $$0.05$$ significance threshold, and the argument that rejecting the null hypothesis provides very little information about the actual relationship between variables in a population.

Criticisms of Null Hypothesis Testing

To illustrate the logic of null hypothesis testing, consider Mehl and colleagues' investigation into whether men and women differ in talkativeness. Assuming the null hypothesis (that there is no difference in the population) was true, they evaluated the likelihood of finding their sample's small difference of $$d = 0.06$$. Finding this result fairly likely under the null hypothesis, they decided to retain the null hypothesis, concluding there was no evidence of a sex difference in the population.

Mehl's Study on Talkativeness

Kanner and colleagues provided an example of rejecting the null hypothesis by studying the relationship between daily hassles and psychological symptoms. They asked how likely it would be to find their sample's strong correlation of $$+.60$$ if the null hypothesis (that there is no correlation in the population) were true. Because such a strong sample relationship would be fairly unlikely under the null hypothesis, they rejected the null hypothesis in favor of the alternative hypothesis, concluding a positive correlation exists in the population.

Kanner's Study on Hassles and Symptoms

Null hypothesis testing follows a structured logic: researchers first assume the null hypothesis is true, calculate how likely their sample results would be under this assumption, and then decide to either reject or retain the null hypothesis based on this calculated probability.

Learn Before

Related

Learn After