Learn Before
Arbitrariness of the p-value Threshold
Critics of null hypothesis testing argue that the strict convention of rejecting the null hypothesis only when is logically flawed. This rigid threshold creates an artificial divide where a result of is deemed significant and worthy of publication, while a nearly identical result of is dismissed as not significant. This arbitrariness can prevent valuable research from being published and exacerbates the file drawer problem by suppressing non-significant findings.
0
1
Tags
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
Statistical Power
Arbitrariness of the p-value Threshold
Bayesian Statistics
Type I Error
Type II Error
In the context of psychological research, which statement best defines a null hypothesis?
If a researcher predicts that caffeine consumption increases alertness, the null hypothesis for this study would state that there is no relationship between caffeine consumption and alertness.
A team of researchers is designing several psychological studies. Match each specific research objective to the corresponding Null Hypothesis (H₀) that the researchers must assume before collecting data.
In psychological research, evaluating a new theory requires a disciplined logical process starting from the Null Hypothesis (). Using the logic of the null as the 'default' position (as seen in the provided image), arrange the steps below to reflect the logical sequence of evaluating a scientific claim.
You are designing a psychological study to investigate the relationship between 'Sleep Quality' and 'Academic Performance' in college students. To establish the necessary logical starting point for your statistical analysis (as illustrated in the provided diagram), which of the following null hypotheses () would you create for your study?
A researcher is investigating whether 'Classical Music' affects 'Concentration Scores' by comparing a group that listens to music with a group that sits in silence. According to the analytical logic of the null hypothesis (H₀), if the researcher assumes there is no relationship between music and concentration, then the numerical value of the difference between the population mean of the music group and the population mean of the silent group is assumed to be _____.
In psychological research, the _____ hypothesis is a formal statement asserting that no relationship exists between the two measured items.
A clinical psychologist is evaluating a new therapy for depression by measuring therapy attendance (number of sessions) and depression severity scores. The null hypothesis for this study would state that there is no relationship between therapy attendance and depression severity.
Match each statistical concept related to hypothesis testing with its correct definition or description based on the logic of the null hypothesis.
Order the logical steps a researcher takes when evaluating a study's results using null hypothesis testing, from the initial assumption to the final decision.
Learn After
What is a primary criticism regarding the strict convention of using a p < .05 threshold to reject the null hypothesis?
Critics of the conventional p < .05 significance threshold argue that it is logically flawed because it creates an artificial divide, leading researchers to dismiss a finding of p = .06 while publishing a nearly identical finding of p = .04.
Place the following steps in the correct order to illustrate how the arbitrariness of the p-value threshold () leads to the 'file drawer problem' in psychological research.
Analyze the consequences of the standard significance threshold by matching each research scenario to the specific logical or systemic critique of the convention it illustrates.
When a researcher dismisses a result of as 'not significant' while accepting as 'significant,' they are relying on a threshold that critics describe as ______ because there is no inherent logical or mathematical reason to prefer as the definitive boundary for scientific truth.
Imagine you are tasked with creating a new 'Continuum-Based Reporting' policy for a psychological journal to address the logical flaws inherent in the arbitrariness of the -value threshold. Which of the following submission requirements would best achieve the goal of eliminating the 'artificial divide' mentioned by critics?
Match each term related to the critique of p-value thresholds with its corresponding definition.
A psychologist proposes changing the significance threshold from to , arguing that this stricter cutoff would resolve the logical problems that critics raise about null hypothesis testing. According to the critique of the arbitrariness of the -value threshold, this proposed change would fully address the core concern.
Critics of null hypothesis testing argue that the convention misrepresents statistical evidence because it treats probability, which is inherently _____, as though it were a binary, all-or-nothing judgment about whether a result is significant.
A statistics professor wants to present the critique of the threshold as a logical chain, ordering arguments from the most fundamental flaw to the most far-reaching practical consequence. Evaluate the arguments below and arrange them from most fundamental (1) to most consequential (4).
Explain the primary criticism that critics raise regarding the standard practice of null hypothesis significance testing with a strict threshold of . In your explanation, recall how this threshold affects publication decisions and identify what systemic issue in psychological research is exacerbated by this convention.
Explain how Dr. Smith's decision demonstrates the logical flaws of the rigid threshold, and describe the systemic impact of this choice on the broader psychological literature.
Suppose a psychology journal implements a new policy that allows researchers to publish studies with -values between and as 'potentially significant' instead of dismissing them. Apply the critics' arguments about the arbitrariness of the threshold to explain how this policy change could help mitigate the file drawer problem.