Google

Machine learning is a highly iterative process. When building a machine learning system, the author often follows a loop: start with an idea about how to build the system, implement the idea in code, and carry out an experiment that tells how well the idea worked. Based on what is learned, generate more ideas and keep iterating; many dozens of ideas may be tried before finding one that is satisfactory.

Iterative Loop of Machine Learning Development

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

The faster one can go around the iterative machine learning development loop, the faster one will make progress.

Faster Iteration Cycles Accelerate ML Progress

When a team has several ideas for improving a cat detector, it can efficiently evaluate those ideas in parallel.

Evaluating Multiple Cat Detector Improvement Ideas in Parallel

After finishing the book, the author encourages readers to share it with friends and teammates so that more people on the team can apply the same machine learning strategy ideas.

Sharing Machine Learning Strategy Knowledge with Teammates

What are the three core steps in the machine learning iterative loop according to the author?

Does a machine learning system builder's first idea usually work successfully?

You may try many _____ of ideas before finding one you are satisfied with.

Match each step of the iterative loop with its primary function.

Order the steps of the machine learning iterative loop.

Question:
Based on the text, explain why building a machine learning system is fundamentally an iterative loop rather than a straightforward linear process. What typical steps are involved in this loop?

Sample answer:
Building a machine learning system is highly iterative because initial ideas rarely work perfectly on the first try. The development process requires a constant feedback loop. First, a developer starts with an idea on how to build or improve the system. Next, they implement that idea in code. After that, they carry out an experiment to measure how well the idea worked. Since the first few ideas usually fail, the developer must use the learnings from that experiment to generate new ideas. This cycle repeats continuously, often requiring dozens of attempts before a satisfactory solution is found.

Key points:
- The core steps are Idea, Code, and Experiment.
- The first few ideas typically do not work.
- Learnings from experiments inform the generation of new ideas.
- The cycle repeats dozens of times until a satisfactory outcome is achieved.

Rubric:
A full-credit response must identify the Idea -> Code -> Experiment cycle, explicitly state that initial ideas usually do not work, and explain that learnings from experiments are used to generate new ideas in a continuous loop.

Explain why machine learning development is considered a highly iterative process.

Case context:
A machine learning team implements their first idea for improving a system. The initial experiment shows poor performance, and the team is highly discouraged. They are considering abandoning the project because their approach failed.

Question:
What fundamental principle of machine learning development is this team ignoring, and what should their immediate next step be?

Sample answer:
The team is ignoring the principle that machine learning is a highly iterative process and that it is completely normal for the first few ideas to fail. Instead of giving up, their immediate next step should be to analyze the learnings from their failed experiment. Based on those learnings, they need to generate new ideas, implement them in code, and run another experiment, understanding that it may take dozens of iterations to succeed.

Key points:
- Machine learning is inherently a highly iterative process.
- It is expected that the first few ideas will not work.
- The team should extract learnings from the failed experiment.
- They must generate new ideas and restart the Idea-Code-Experiment loop.

Rubric:
The answer should point out that early failure is expected in ML. It must advise the team to use the results from the failed experiment to generate new ideas and continue the iterative loop.

Diagnose a team's failure to improve their ML system after one attempt.

Question:
In the machine learning development loop, after implementing an idea in code and carrying out an experiment, what is the next action you should take based on the results?

Sample answer:
Based on the learnings from the experiment, you should go back to generate more ideas and keep iterating.

Key points:
- Go back to generate more ideas.
- Keep on iterating.

Rubric:
The student must state that the developer should use the experimental results to generate new ideas and continue iterating.

Learn Before

Related