Assume each of the steps below would take about an equal amount of time (a few days). What is the first thing you do?

Letting the Data Speak: Getting started on an end-to-end deep learning project

- **Lets the data speak:** For a large enough data set, the neural network can figure out the appropriate function mapping from X (input) to Y (output). Moreover, by relying on a pure machine learning approach, the data and its underlying statistics are the main factors for classifying/predicting the output, and it does not have to reflect on human preconceptions.
- **Less hand-designing of components:** This simplifies the design workflow and saves a lot of time designing individual features of the pipeline. This may contribute to a more ubiquitous adoption of this learning method.

University of Michigan - Ann Arbor

End-to-end learning means training a single model (specifically a deep neural network) that represents the complete target system, bypassing intermediate layers usually present in traditional pipeline designs. So, instead of training separate models for preprocessing, feature selection, ... all of them will be done in the same model.

The model learns all the steps between the initial input phase and the final output result. This is a deep learning process where all of the different parts are simultaneously trained instead of sequentially.

End-To-End Deep Learning

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

https://www.coursera.org/learn/machine-learning-projects?specialization=deep-learning

Structuring Machine Learning Projects

https://towardsdatascience.com/e2e-the-every-purpose-ml-method-5d4f20dafee4

End-to-end learning, the (almost) every purpose ML method

Advantages of End-to-End Deep Learning

- **Large amount of data is necessary:** incorporating some prior knowledge into the training improves performance and is considered a key element. For end-to-end learning more training examples must be provided because we don't incorporate this prior knowledge.
- **Difficult to improve or modify the system:** if some structural change must be applied, the old model is of no use and the whole model has to be replaced and trained all over again.
- **Potentially useful hand-designed modules cannot be used:** many hand-designed modules hold a lot of human knowledge and are efficient in solving some tasks, especially when the available data is limited.
- **Difficult to validate:** due to the complex architecture, the potential number of input/output pairs can be big enough to make high level validation impossible.

Limitations of End-to-End Deep Learning

- Speech Recognition
- Autonomous Driving

Learn Before

Related

Learn After