In a small data regime, traditional algorithms may or may not do better depending on how features are hand-engineered. With only about 20 training examples, feature hand-engineering can matter more than choosing logistic regression or a neural network.

Google

Recent progress in deep learning has been driven by data availability and computational scale.

Scale Drives Machine Learning Progress

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Data availability is a major driver of recent deep learning progress because digital-device activity creates large amounts of data that can be fed to learning algorithms.

Data Availability as a Driver of Deep Learning Progress

Computational scale is a major driver of recent deep learning progress because it makes it possible to train neural networks large enough to take advantage of huge datasets.

Computational Scale as a Driver of Deep Learning Progress

Older learning algorithms such as logistic regression can plateau as more data is added, meaning their learning curves flatten and the algorithms stop improving even with more data.

Older Learning Algorithms Can Plateau with More Data

Larger neural networks can obtain better performance, and the best performance comes from training a very large neural network while also having a huge amount of data.

Learn Before

Related