Learn Before
Concept
Mini-batch size variations
if mini-batch size = m --> batch gradient descent: takes too much time, too long per iteration (blue line)
if mini-batch size = 1 --> stochastic gradient descent: every example is its own mini-batch, lose speed up from vectorization (purple line)
in practice: want to choose somewhere in between 1 & m for batch size --> fastest learning, vectorization, makes progress without processing entire training set (green line)
0
2
Updated 2021-03-14
Tags
Data Science