When training a neural language model minimizing loss will result in a algorithm for language modeling, and a new set of embeddings that can be used as word representations for other tasks. With that in priority, training proceeds by taking an input as very long text, concatenating all the sentences, starting random weights, and iteratively moving through the text predicting each word. The architecture consists of setting the parameters θ = E, W, U, b and via grading descent, using error backpropagation on the computation graph to compute the gradient. Thus, effectively setting the weights (W, U), and learning embeddings (E) to predict upcoming words.