One reliable way to improve an algorithm's performance today is to train a bigger network and get more data.

Bigger Networks and More Data as Reliable Improvement Levers

With one million training examples, Machine Learning Yearning favors using a neural network.

One Million Examples Can Favor a Neural Network

Larger neural networks can obtain better performance, and the best performance comes from training a very large neural network while also having a huge amount of data.

Google

Recent progress in deep learning has been driven by data availability and computational scale.

Scale Drives Machine Learning Progress

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Data availability is a major driver of recent deep learning progress because digital-device activity creates large amounts of data that can be fed to learning algorithms.

Data Availability as a Driver of Deep Learning Progress

Computational scale is a major driver of recent deep learning progress because it makes it possible to train neural networks large enough to take advantage of huge datasets.

Computational Scale as a Driver of Deep Learning Progress

Older learning algorithms such as logistic regression can plateau as more data is added, meaning their learning curves flatten and the algorithms stop improving even with more data.

Older Learning Algorithms Can Plateau with More Data

Large Neural Networks Benefit from Huge Data

In a small data regime, traditional algorithms may or may not do better depending on how features are hand-engineered. With only about 20 training examples, feature hand-engineering can matter more than choosing logistic regression or a neural network.

Learn Before

Related

Learn After