The $$t$$ distribution illustrates the spread of possible $$t$$ scores that researchers would expect to observe if the null hypothesis were entirely true. This distribution forms a unimodal and symmetrical curve centered precisely around a mean of $$0$$. By utilizing this known distribution curve, researchers can evaluate any computed $$t$$ score to find its exact $$p$$-value, which represents the proportion of expected scores that are at least as extreme as the observed sample result.

Distribution of t Scores

Degrees of freedom is a core statistical concept that dictates the precise mathematical shape of a test statistic's distribution when the null hypothesis is true. By determining this exact structural shape, the degrees of freedom make it possible for researchers to accurately calculate the probability, or $$p$$ value, associated with any given test score.

Degrees of Freedom

When evaluating the relationship between quantitative variables using Pearson's $$r$$, the appropriate statistical procedure is a hypothesis test of the correlation coefficient. This test shares the fundamental logic of other null hypothesis tests but specifically assesses whether a true, non-zero correlation exists within the broader population.

Hypothesis Test of the Correlation Coefficient

Assuming the null hypothesis is true, the distribution of $$t$$ scores is symmetrical and unimodal, with a mean of $$0$$. This known distribution allows researchers to find the $$p$$-value associated with any computed $$t$$ score based on its degrees of freedom.

Distribution of $$t$$ Scores

During hypothesis testing, what is the term for the specialized numerical value computed from sample data that serves as an intermediate step to evaluate the probability of obtaining the observed results if the null hypothesis were true?

A test statistic is a specialized numerical value computed from sample data specifically to aid in determining the $$p$$-value during hypothesis testing. It serves as an intermediate step to evaluate the probability of obtaining the observed results if the null hypothesis were true.

Google

The p value is the probability of the experiment having given the result it did purely by chance. The threshold of significance is almost always 0.05 so that if the probability of a chance outcome is less than 1 in 20, the experimental result is declared to be significant. There are other important thresholds as summarized in the following table

p value

https://kpu.pressbooks.pub/psychmethods4e/

KPU Research Methods in Psychology - 4th American Edition

Because the thresholds are conventionalized, we need to recognize the apparent arbitrariness of the conventions and not make such black and white interpretations.

Don't make such black and white interpretations

Always report effect sizes where possible. Effect sizes are measures of importance because they indicate how much effect the experimental manipulation had on the differences seen in the data.

Report effect sizes

The $$p$$-value in null hypothesis testing depends on two primary factors: the strength of the relationship in the sample (effect size) and the size of the sample. A stronger relationship and a larger sample both decrease the likelihood that a result occurred by chance under the null hypothesis, thereby yielding a lower $$p$$-value.

Factors Determining the p-value

Test Statistic

A common criticism of null hypothesis testing stems from researchers misinterpreting the $$p$$-value. A frequent error is believing that the $$p$$-value represents the probability that the null hypothesis is true, rather than the probability of observing the sample data if the null hypothesis were true. Another closely related misconception is assuming that $$1 - p$$ equals the probability of replicating a statistically significant result, a claim that is overly optimistic and ignores the crucial role of statistical power in replication.

Misinterpretations of the p-value

In null hypothesis testing, alpha ($$\alpha$$) is the predetermined criterion used to decide how low a $$p$$ value must be before a sample result is considered unlikely enough to reject the null hypothesis. It is almost always set to $$.05$$. This means that if there is a $$5\%$$ chance or less of observing a result at least as extreme as the sample result when the null hypothesis is true, researchers will reject the null hypothesis.

Alpha (Level of Significance)

p-hacking is a problematic research practice where researchers make various arbitrary analytical decisions to artificially inflate their chances of obtaining a statistically significant result. By selectively reporting dependent variables, unjustified removal of outliers, or selectively presenting only significant findings, researchers continuously manipulate their data until it yields a desirable $$p$$-value. This practice severely compromises the reliability of published results by creating an unacceptably high rate of Type I errors in the psychological literature.

p-hacking

In 2015, the editors of the journal Basic and Applied Social Psychology took the unprecedented step of banning the use of null hypothesis testing and $$p$$-values in its publications. Instead of relying on traditional statistical significance thresholds to evaluate validity, the editors required authors to focus heavily on descriptive statistics and effect sizes. Although this ban was not widely adopted across the field, it significantly amplified the ongoing conversation in psychology about the limitations of the 'gold standard' of statistical validity.

The 2015 Ban on Null Hypothesis Testing

The alpha ($$\alpha$$) criterion is a predefined probability threshold—almost always set to $$.05$$ (or $$5\%$$)—that establishes how low a $$p$$ value must be before researchers consider the sample result unlikely enough to reject the null hypothesis.

Alpha Criterion

The most common misinterpretation of the $$p$$ value is mistakenly believing it represents the probability that the null hypothesis is true (i.e., the probability that the result occurred by chance). In reality, the $$p$$ value calculates the probability of obtaining the specific sample data assuming the null hypothesis is already true.

Misinterpretation of the p Value

The $$p$$ value in null hypothesis testing is determined by two main considerations: the strength of the sample relationship and the size of the sample. A stronger sample relationship and a larger sample size both decrease the probability of obtaining the observed result if the null hypothesis were true, resulting in a lower $$p$$ value.

Learn Before

Related

Learn After