Trade-off Between Sample Size and Relationship Strength

Which two primary factors determine the $p$-value in null hypothesis testing?

If two separate studies observe the exact same strength of relationship, the study with the larger sample size will generally yield a lower p-value.

A researcher is conducting three different studies on how 'study breaks' affect test scores. Based on the factors that determine a $p$-value, rank the following study results from the lowest $p$-value (1) to the highest $p$-value (3).

A team of psychologists is reviewing multiple independent studies on the relationship between sleep duration and cognitive performance. Match each study scenario with the specific configuration of factors that best explains the observed outcome of the $p$-value.

You are designing a study to investigate the impact of a new mindfulness technique on stress reduction. Previous research indicates the effect is likely to be subtle. Which of the following research plans demonstrates the most effective synthesis of the factors that determine a $p$-value to maximize the likelihood of finding a statistically significant result ($p < .05$)?

In null hypothesis testing, both a larger sample size and a stronger relationship in the sample contribute to a _____ $p$-value.

Two independent psychology experiments both result in an identical statistical significance level of $p = .03$. Experiment A used a very large sample of $N = 800$ participants, whereas Experiment B used a much smaller sample of $N = 30$ participants. Based on the factors that determine the $p$-value, the experiment that must have observed a stronger relationship (larger effect size) between its variables is Experiment _____.

A clinical psychologist is comparing two independent studies on the effectiveness of a new cognitive therapy. Study A has a sample size of $N = 300$ and finds a strong relationship between therapy completion and symptom reduction. Study B has a sample size of $N = 40$ and finds a weak relationship. True or False: Based on the factors that determine a $p$-value, Study A is expected to yield a lower $p$-value than Study B.

Match each statistical concept from null hypothesis testing with the description of its relationship to the resulting $p$-value.

Evaluate the design parameters of the following three proposed psychology studies. Order them from the study configuration expected to yield the lowest $p$-value (1) to the study configuration expected to yield the highest $p$-value (3).

Probabilistic Nature of Statistics

Trade-off Between Sample Size and Relationship Strength

Practical Significance

In the context of inferential statistics, what does it mean when a research finding is described as statistically significant?

In psychological research, if a finding is described as statistically significant, it implies that the observed effect is likely to be a genuine phenomenon rather than a product of random variation.

A psychological researcher is evaluating the results of several different studies. Match each specific research scenario with the correct statistical interpretation regarding significance and the nature of the findings.

A psychologist is evaluating whether a new study on social interaction reveals a real phenomenon or just a fluke. Arrange the logical stages of the researcher's analytical process to correctly conclude that a finding is statistically significant.

Match each term related to the interpretation of research results with its correct defining characteristic.

Example of Sample Size and Relationship Strength

A researcher reports that a new study-skills training program led to a statistically significant increase in exam scores compared to a control group. What does it mean for this finding to be 'statistically significant'?

A researcher claims that a research finding is 'proven' because the probability of the results occurring by random chance alone is only $0.001$. A peer reviewer evaluates this claim and suggests a more scientifically accurate term, noting that the finding should instead be described as _____ to indicate that it is highly unlikely to have occurred due to random variation while avoiding the overstatement of absolute proof.