Intuitive Judgment of Statistical Significance

When determining statistical significance in null hypothesis testing, how do sample size and relationship strength interact?

A researcher finds only a very weak correlation between sleep quality and exam performance, yet the result is statistically significant. This outcome is plausible if the study used a very large sample of participants.

In psychological research, statistical significance depends on a trade-off between sample size and relationship strength. Match each hypothetical research scenario with the statement that best explains its likely statistical outcome.

In null hypothesis testing, the statistical significance of a result is determined by the trade-off between the number of participants and the strength of the relationship. Rank the following hypothetical research studies in order from the study that requires the strongest observed relationship to reach statistical significance (1) to the study that requires the weakest observed relationship to reach statistical significance (4).

Imagine you are developing a research proposal to investigate a rare psychological condition. Because the condition is so rare, you can only construct a sample of $10$ participants ($N = 10$). To formulate a design that is still capable of reaching a statistically significant result ($p < .05$), which of the following strategies should you prioritize in your plan to address the trade-off between sample size and relationship strength?

In null hypothesis testing, a weak statistical relationship can never reach statistical significance, even if the sample size is extremely large.

A researcher evaluates a study and concludes that its significant $p$-value ($p < .05$) proves a 'powerful and highly impactful' effect, despite the study having $30,000$ participants and a relationship strength of only $r = .02$. This conclusion is likely an overestimation because it fails to account for the _____ between sample size and relationship strength in null hypothesis testing.