An evaluation corpus should cover common, rare, ambiguous, adversarial, and protected-user situations across important graph paths. A random sample of easy successes cannot establish system reliability.

Evaluation Corpora Must Represent Real Routes

A counterfactual test changes one relevant or irrelevant feature and observes whether routing changes appropriately. It can reveal brittle shortcuts and unfair proxy use.

Counterfactual Tests Probe Decision Sensitivity

In this situation—testing an adaptive tutor for beginners, experts, disabilities, and conflicting goals—which choice best applies “Evaluation Corpora Must Represent Real Routes”?

A correct final answer can hide unsafe tool calls, leaked data, wasted loops, or unsupported intermediate claims. Trajectory evaluation inspects state transitions and actions as well as outcome.

Learn Before

Related