Most optimization techniques are constructed with the idea that one has access to the exact gradient; however, we often find ourselves making rough approximations or even biased estimates for the gradient, especially when the objective function we want to minimize is intractable the only option is to make an approximation. Thankfully, many neural network algorithms have some method in their design to accommodate for these imperfections.