Learn Before
Explain how Dr. Smith's decision demonstrates the logical flaws of the rigid threshold, and describe the systemic impact of this choice on the broader psychological literature.
Case context: A researcher, Dr. Smith, conducts an experiment to test if a new mindfulness technique reduces anxiety. The statistical analysis yields a p-value of . Because the result does not meet the standard threshold, Dr. Smith decides to shelve the project, assuming the study failed and is not worth submitting to a journal.
Question: Explain how Dr. Smith's decision demonstrates the logical flaws of the rigid threshold, and describe the systemic impact of this choice on the broader psychological literature.
Sample answer: Dr. Smith's decision illustrates the logical flaw of the rigid threshold because a result of is nearly identical to a result of , yet the rigid cutoff leads Dr. Smith to dismiss the finding as a failure. Systemically, shelving this project prevents valuable research from being shared, which directly contributes to the file drawer problem by ensuring that only statistically significant findings are published.
Key points:
- Identifies that Dr. Smith is relying on an artificial divide created by the standard threshold.
- Recognizes that provides nearly identical evidence to a significant result like .
- Explains that shelving the study prevents valuable findings from entering the scientific literature.
- Connects Dr. Smith's decision to the file drawer problem.
Rubric: To earn full credit, the student must explain: 1) that Dr. Smith is treating the boundary between and as an artificial divide; 2) that this is logically flawed because the statistical evidence is nearly identical to a significant result; and 3) that shelving the study suppresses valuable research and contributes directly to the file drawer problem.
0
1
Tags
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
What is a primary criticism regarding the strict convention of using a p < .05 threshold to reject the null hypothesis?
Critics of the conventional p < .05 significance threshold argue that it is logically flawed because it creates an artificial divide, leading researchers to dismiss a finding of p = .06 while publishing a nearly identical finding of p = .04.
Place the following steps in the correct order to illustrate how the arbitrariness of the p-value threshold () leads to the 'file drawer problem' in psychological research.
Analyze the consequences of the standard significance threshold by matching each research scenario to the specific logical or systemic critique of the convention it illustrates.
When a researcher dismisses a result of as 'not significant' while accepting as 'significant,' they are relying on a threshold that critics describe as ______ because there is no inherent logical or mathematical reason to prefer as the definitive boundary for scientific truth.
Imagine you are tasked with creating a new 'Continuum-Based Reporting' policy for a psychological journal to address the logical flaws inherent in the arbitrariness of the -value threshold. Which of the following submission requirements would best achieve the goal of eliminating the 'artificial divide' mentioned by critics?
Match each term related to the critique of p-value thresholds with its corresponding definition.
A psychologist proposes changing the significance threshold from to , arguing that this stricter cutoff would resolve the logical problems that critics raise about null hypothesis testing. According to the critique of the arbitrariness of the -value threshold, this proposed change would fully address the core concern.
Critics of null hypothesis testing argue that the convention misrepresents statistical evidence because it treats probability, which is inherently _____, as though it were a binary, all-or-nothing judgment about whether a result is significant.
A statistics professor wants to present the critique of the threshold as a logical chain, ordering arguments from the most fundamental flaw to the most far-reaching practical consequence. Evaluate the arguments below and arrange them from most fundamental (1) to most consequential (4).
Explain the primary criticism that critics raise regarding the standard practice of null hypothesis significance testing with a strict threshold of . In your explanation, recall how this threshold affects publication decisions and identify what systemic issue in psychological research is exacerbated by this convention.
Explain how Dr. Smith's decision demonstrates the logical flaws of the rigid threshold, and describe the systemic impact of this choice on the broader psychological literature.
Suppose a psychology journal implements a new policy that allows researchers to publish studies with -values between and as 'potentially significant' instead of dismissing them. Apply the critics' arguments about the arbitrariness of the threshold to explain how this policy change could help mitigate the file drawer problem.