Learn Before
p value
A p value is the probability, calculated under a specified statistical model, of getting a result at least as extreme as the one observed. The model combines the null hypothesis with the test's other assumptions, such as how the data were collected and how they are distributed. A p value is not the probability that the null hypothesis is true, and it is not the probability that the result occurred purely by chance. A small p value means the data would be unusual if the whole model were true; on its own it cannot say whether the null hypothesis or another assumption is at fault. By a common convention in psychology, a result is called statistically significant when p is .05 or less, that is, when p is at or below a significance level (alpha) of .05: under the model, results at least as extreme as the observed one would occur no more than 1 time in 20. A p value does not measure how large or important an effect is, and a conclusion should not rest only on whether p passes a threshold. The table below lists conventional descriptors for ranges of p; its third column shows typical but incautious interpretations that claim more than a p value can show, not correct readings.
Source: Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133; section 2 and principles 1–3, 5 and 6 (pp. 131–132). https://doi.org/10.1080/00031305.2016.1154108
0
2
Tags
User experience (UX) research
Design Science
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
Learn After
Don't make such black and white interpretations
Factors Determining the p-value
Test Statistic
Misinterpretations of the p-value
Alpha (Level of Significance)
p-hacking
The 2015 Ban on Null Hypothesis Testing
Alpha Criterion
What does a p value represent in the context of psychological research?
If a psychology researcher calculates a p-value of 0.08, they will typically declare the experimental result to be statistically significant.
A social psychologist is analyzing data from four different studies. Based on the standard thresholds for statistical significance provided in the table, match each study's obtained p value with its correct interpretation.
Misinterpretation of the p Value
Determinants of the Value
A researcher obtains a value of and labels the result 'highly significant' (). According to standard significance thresholds, where and , this labeling is _____.
A cognitive psychologist conducts an experiment on learning styles and calculates a value of . Applying the standard threshold of significance () described in the text, the researcher should declare this result to be statistically significant.
Define what a value represents in the context of an experiment and state the standard threshold for declaring an experimental result to be statistically significant.
Reporting Effect Sizes in Null Hypothesis Significance Testing
Using the convention stated in this question for marking results with asterisks (a cut-off value itself counts as meeting it), match each label with the range of values it marks.
By a common convention in psychology, a result is declared statistically significant when . For a result exactly at this threshold (), results at least as extreme would occur, under the test's model (the null hypothesis plus the test's assumptions), time in how many?
By a common convention in psychology, a result is declared statistically significant when . For a result exactly at this threshold (), if the null hypothesis and the test's assumptions were true, results at least as extreme would occur 1 time in _____.
Arrange the following three values in ascending numerical order, from the smallest (1) to the largest (3).
A psychologist has values from four different studies. Arrange the values in ascending numerical order, from the smallest to the largest. (This orders the numbers only: values from different studies, each calculated under its own model, do not by themselves rank how strong the evidence is.)
A psychology student runs an analysis on their study's data and obtains a value of . Using the common convention of a significance level of (a result is significant when ), what conclusion should the student draw about their experimental result, and why?
Based on the definition of a value and a significance level of set in advance, explain in your own words what it means if, assuming the null hypothesis and the test's assumptions were true, a result at least as extreme as the researchers' would occur less than 1 time in 20, and what this does not establish.