Learn Before
Based on this scenario, identify and explain how Dr. Aris's decisions illustrate the file drawer problem. In your explanation, describe the specific consequences these decisions have on the validity of the published literature regarding this mindfulness application.
Case context: Dr. Aris spends three years conducting five independent, well-designed experiments testing a new mindfulness application's effect on test anxiety in college students. In four of these experiments, the application shows no statistically significant reduction in anxiety compared to a control group. In the fifth experiment, a statistically significant reduction is found (). Dr. Aris decides to write up and submit only the fifth study for publication, and a major journal accepts it. The other four studies are left in Dr. Aris's digital archives.
Question: Based on this scenario, identify and explain how Dr. Aris's decisions illustrate the file drawer problem. In your explanation, describe the specific consequences these decisions have on the validity of the published literature regarding this mindfulness application.
Sample answer: Dr. Aris's decisions illustrate the file drawer problem because she selectively submitted only the single experiment with statistically significant results while leaving the four experiments with non-significant results unpublished in her archives. This creates a skewed representation of the application's effectiveness in the published literature. The published study likely represents a Type I error (a false positive), and any reader of the published literature will be misled into overstating the true strength and prevalence of the application's effectiveness in the population.
Key points:
- Identifies that only the significant result was published and the four non-significant studies were kept 'in the drawer'.
- Diagnoses this selective reporting as an instance of the file drawer problem.
- Explains that the published study likely represents a Type I error (false positive).
- Explains that the published literature now artificially overstates the true strength of the mindfulness app's effect.
Rubric: To receive full credit, the response must: 1) Identify that only the single significant study was published while the four non-significant studies were hidden/archived. 2) Connect this behavior directly to the file drawer problem. 3) Explain that the published study is likely a Type I error (false positive). 4) Explain that this selectively published literature artificially overstates the mindfulness application's effectiveness.
0
1
Tags
KPU
Research Methods in Psychology - 4th American Edition @ KPU
Related
Registered Reports
Journal of Articles in Support of the Null Hypothesis
Which of the following best describes the 'file drawer problem' in scientific research?
The file drawer problem implies that published scientific research may artificially overstate the true strength and prevalence of relationships between variables.
A research team is investigating whether a new study technique improves test scores. Match each specific scenario to its role or consequence within the file drawer problem.
Arrange the following steps in the correct order to illustrate how the file drawer problem leads to a biased understanding of a psychological phenomenon within the scientific literature.
Match each term related to publication bias to the definition that best describes its role in the scientific process.
Which of the following best describes the likely consequence for a researcher who relies solely on published literature in a field affected by the 'file drawer problem'?
When evaluating the credibility of a psychological claim based on published research, a scientist must account for the fact that the 'file drawer problem' causes the available evidence to _____ the true strength of the relationship between variables.
A research team conducts ten independent studies testing a new memory drug. Nine of these studies find no statistically significant difference and are left unpublished in the lab's records, while the single study that yields a statistically significant improvement is published in a journal. True or False: According to the concept of the file drawer problem, this selective publishing will lead the published scientific literature to artificially overstate the drug's true effectiveness in the population.
When researchers and journal editors selectively publish studies with statistically significant results while discarding those with non-significant findings, the resulting bias in the published literature leads to a disproportionately high rate of _____ errors.
Order the steps a researcher takes to evaluate whether a published psychological effect is overstated due to the file drawer problem.
Define the 'file drawer problem' in scientific research and explain how it influences the proportion of Type I errors and the reported strength of relationships in published literature.
Based on this scenario, identify and explain how Dr. Aris's decisions illustrate the file drawer problem. In your explanation, describe the specific consequences these decisions have on the validity of the published literature regarding this mindfulness application.
Imagine you are conducting a literature review to evaluate the effectiveness of a new cognitive behavioral therapy technique. Knowing that the psychological literature is affected by the file drawer problem, how should you apply this knowledge when interpreting the overall strength of the therapy's effects as reported in published journal articles?