Trade-off Between Sample Size and Relationship Strength

Which two primary factors determine the $p$-value in null hypothesis testing?

If two separate studies observe the exact same strength of relationship, the study with the larger sample size will generally yield a lower p-value.

A researcher is conducting three different studies on how 'study breaks' affect test scores. Based on the factors that determine a $p$-value, rank the following study results from the lowest $p$-value (1) to the highest $p$-value (3).

A team of psychologists is reviewing multiple independent studies on the relationship between sleep duration and cognitive performance. Match each study scenario with the specific configuration of factors that best explains the observed outcome of the $p$-value.

You are designing a study to investigate the impact of a new mindfulness technique on stress reduction. Previous research indicates the effect is likely to be subtle. Which of the following research plans demonstrates the most effective synthesis of the factors that determine a $p$-value to maximize the likelihood of finding a statistically significant result ($p < .05$)?

In null hypothesis testing, both a larger sample size and a stronger relationship in the sample contribute to a _____ $p$-value.

Two independent psychology experiments both result in an identical statistical significance level of $p = .03$. Experiment A used a very large sample of $N = 800$ participants, whereas Experiment B used a much smaller sample of $N = 30$ participants. Based on the factors that determine the $p$-value, the experiment that must have observed a stronger relationship (larger effect size) between its variables is Experiment _____.

A clinical psychologist is comparing two independent studies on the effectiveness of a new cognitive therapy. Study A has a sample size of $N = 300$ and finds a strong relationship between therapy completion and symptom reduction. Study B has a sample size of $N = 40$ and finds a weak relationship. True or False: Based on the factors that determine a $p$-value, Study A is expected to yield a lower $p$-value than Study B.

Match each statistical concept from null hypothesis testing with the description of its relationship to the resulting $p$-value.

Evaluate the design parameters of the following three proposed psychology studies. Order them from the study configuration expected to yield the lowest $p$-value (1) to the study configuration expected to yield the highest $p$-value (3).

State the two primary factors that determine the $p$-value in null hypothesis testing, and describe how changes in each of these factors affect the resulting $p$-value.

Explain why Experiment B achieves a lower $p$-value than Experiment A, even though the observed relationship strength is identical in both studies. Your response should demonstrate how sample size influences the likelihood of the null hypothesis.

A researcher conducts a study with $N = 15$ participants and finds a strong relationship between exercise and mood score, but the resulting $p$-value is not statistically significant ($p = .12$). Apply your knowledge of the factors determining the $p$-value to suggest a specific change the researcher should make in a replication study to decrease the $p$-value, and explain the statistical reasoning behind this change.