Case Study

Choosing rapid experiments for a product recommendation model

Case context: A machine learning team is improving a shopping app’s recommendation system. They can either spend four weeks building one large, carefully engineered model change, or they can create a process for running several smaller experiments, checking results quickly, and making another round of changes within that same four-week period.

Question: Using the idea that development speed matters in machine learning, which option should the team select to make the most progress, and why?

Sample answer: The team should select the option that supports several short experiments and repeated check-and-adjust cycles. The reason is that making more passes through the develop-test-learn cycle in the same amount of time usually leads to faster improvement. A process that allows quick feedback and frequent updates will move the project forward more effectively than one lengthy, high-effort change.

Key points:

  • The better choice is the one that allows more frequent experiment cycles.
  • More completed develop-test-learn rounds in the same time usually means faster progress.
  • One slow, highly elaborate change is less effective than several quicker, smaller changes.

Rubric: The response must recommend the option that enables multiple faster experiment cycles. It must explain that shorter cycles through the development process lead to faster overall progress.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI