Learn Before
Concept

Mini-batch size variations

if mini-batch size = m --> batch gradient descent: takes too much time, too long per iteration (blue line)

if mini-batch size = 1 --> stochastic gradient descent: every example is its own mini-batch, lose speed up from vectorization (purple line)

in practice: want to choose somewhere in between 1 & m for batch size --> fastest learning, vectorization, makes progress without processing entire training set (green line)

Image 0

0

2

Updated 2021-03-14

Tags

Data Science