Case context: A product team keeps updating a ranking model for an e-commerce app and checks the dev set after each revision to see whether the latest change helped. This is part of day-to-day development, not a formal research publication.

Question: Should the team make statistical significance testing a standard step every time it compares these intermediate dev-set results? Explain your choice.

Sample answer: No. Statistical testing can be done in principle, but it should not be treated as a required step for every intermediate comparison. For this kind of development workflow, teams usually do not rely on it, and it is often not very helpful for judging incremental progress. The situation where such testing is more relevant is when preparing a research paper, which is not the case here.

Key points:

• Statistical tests are possible.
• They are not usually part of routine intermediate comparisons.
• The scenario is about iterative development progress.
• The team is not preparing a research paper.
• The paper-preparation exception does not apply.

Rubric: The response should clearly say whether routine significance testing is recommended, distinguish possibility from normal practice, connect the decision to iterative dev-set evaluation, and note that the research-publication context is absent.