A university implements a mandatory academic support program for students who score below 50 on a 100-point placement test. To evaluate the program's effectiveness, researchers compare the final course grades of students who scored 49 (and entered the program) with those who scored 50 (and did not). Which of the following potential issues represents the most significant threat to the validity of the conclusion that the program caused any observed difference in grades?

Evaluating a Customer Loyalty Program

Evaluating Scholarship Effectiveness

The Logic of Discontinuity Design

A researcher is evaluating a scholarship program where students with a GPA of 3.5 or higher are eligible to apply. To measure the scholarship's impact, the researcher compares the post-graduation salaries of students with a 3.51 GPA who successfully applied and received the scholarship to the salaries of students with a 3.49 GPA who were ineligible. Based on the principles of using a sharp cutoff to establish causality, any observed salary difference between these two groups can be confidently attributed solely to the effect of receiving the scholarship.

A research study aims to measure the impact of a free financial literacy course. The course is automatically offered to all individuals with a credit score of 620 or lower. To assess the course's effectiveness, analysts compare the financial health of individuals just below this score to those just above it one year later. Match each component of the study design to its correct description in this scenario.

A study is designed to measure the impact of a new tutoring program offered to students with a GPA below 2.5. To ensure the comparison between students just above and just below the 2.5 GPA threshold is valid, the most critical assumption is that students cannot precisely ________ their GPA in order to gain or avoid access to the program.

A research team wants to evaluate the causal impact of a new, intensive math tutoring program. The program is automatically assigned to all high school students who score below 600 on a standardized math assessment. Arrange the following steps in the correct logical order for conducting a study based on this sharp cutoff.

Critiquing a Small Business Grant Study

A city wants to evaluate the impact of a new summer reading program on students' vocabulary. The program is offered for free to all students who score less than 70 (out of 100) on a standardized pre-test. Two methods are proposed to measure the program's effect:

Method 1: Compare the average vocabulary of all students who participated (pre-test score < 70) with the average vocabulary of all students who did not (pre-test score ≥ 70).

Method 2: Compare the average vocabulary of students who scored just below 70 (e.g., 68-69) with the average vocabulary of students who scored just above 70 (e.g., 70-71).

Which method provides a more reliable estimate of the program's true causal effect, and why?