Examples of burn-in

Burn-in is an initial phase before normally running your model. The epochs of burn-in is number of epochs to ramp learning rate from either 0 or 1 to your final learning rate. It is helpful to find good embeddings in some cases.

University of Michigan - Ann Arbor

The author tested 15, 50, and 100 epochs for Poincaré embeddings and chose 100 as the best number to train the model, with epochs of burn-in = 10.

Improved Concept Embeddings for Learning Prerequisite Chains: Epochs

- A learning rate scheduler: https://github.com/ultralytics/yolov3/issues/238
- Löffel, E. P. (2019). Improved Concept Embeddings for Learning Prerequisite Chains.

Examples of burn-in: Positive examples

- Markov chain Monte Carlo (MCMC): http://users.stat.umn.edu/~geyer/mcmc/burn.html

Examples of burn-in: Negative examples

When you try to train a huge amount of data with limited computer memory, you can separate the whole training set to several batches that can fit into your computer memory. Then you feed these batches to your model one by one. After feeding all batches once, you complete one epoch. To successfully train your model, you need multiple epochs.

Learn Before

Related

Learn After