Rather than the traditional entire pipeline, we can use a deep neural network to execute the intermediate multi-branch steps:

$Camera\:Images-\left(Deep\:Learning\right)->Steering\:direction$

Autonomous driving relies heavily on machine learning to detect objects and take action accordingly. The pipeline for this is not linear and is more complex. The three components of this pipeline detect cars, detect pedestrians, and then plot a path for the car to take to avoid both of these. 

End-to-End Deel Learning in Autonomous Driving

The traditional approach to speech recognition relies on a pipeline with several intermediate components:

$$\text{Audio (input)} \rightarrow \text{feature extraction} \rightarrow \text{phoneme detection} \rightarrow \text{word composition} \rightarrow \text{text transcript (output)}$$

The end-to-end approach replaces this multi-step chain with a single deep neural network, allowing the system to be optimized directly for the final output using a single criterion:

$$\text{Audio (input)} \rightarrow \text{Deep Neural Network} \rightarrow \text{text transcript (output)}$$

Traditional vs. End-to-End Speech Recognition Pipelines

- Speech Recognition
- Autonomous Driving

University of Michigan - Ann Arbor

End-to-end training is a core methodology in deep learning where a complete system is built and tuned jointly to map raw inputs to final outputs. This approach contrasts with traditional machine learning pipelines that assemble individually optimized components, such as a separate, manually engineered feature extractor followed by a shallow classifier. By simultaneously training all layers of computation, the model automatically discovers the optimal representations needed for the task.

End-to-End Training

https://towardsdatascience.com/e2e-the-every-purpose-ml-method-5d4f20dafee4

End-to-end learning, the (almost) every purpose ML method

https://www.coursera.org/learn/machine-learning-projects?specialization=deep-learning

Structuring Machine Learning Projects

- **Lets the data speak:** For a large enough data set, the neural network can figure out the appropriate function mapping from X (input) to Y (output). Moreover, by relying on a pure machine learning approach, the data and its underlying statistics are the main factors for classifying/predicting the output, and it does not have to reflect on human preconceptions.
- **Less hand-designing of components:** This simplifies the design workflow and saves a lot of time designing individual features of the pipeline. This may contribute to a more ubiquitous adoption of this learning method.

Advantages of End-to-End Deep Learning

- **Large amount of data is necessary:** incorporating some prior knowledge into the training improves performance and is considered a key element. For end-to-end learning more training examples must be provided because we don't incorporate this prior knowledge.
- **Difficult to improve or modify the system:** if some structural change must be applied, the old model is of no use and the whole model has to be replaced and trained all over again.
- **Potentially useful hand-designed modules cannot be used:** many hand-designed modules hold a lot of human knowledge and are efficient in solving some tasks, especially when the available data is limited.
- **Difficult to validate:** due to the complex architecture, the potential number of input/output pairs can be big enough to make high level validation impossible.

Learn Before

Related

Learn After