Pseudo-rehearsal methods are a sub-group of rehearsal methods. Instead of using training samples from memory, pseudo-rehearsal models generate examples by knowing the probability distributions of previous task samples.

University of Michigan - Ann Arbor

Rehearsal methods rely on retaining some training examples from prior tasks, so that they can later be shown to a task at hand. As training samples are kept per each task and are periodically replayed while learning the model, the computing and memory requirements of the model increase proportionally to the number of tasks.

Rehearsal Methods

Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020. Continual Lifelong Learning in Natural Language Processing: A Survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6523–6541.
https://aclanthology.org/2020.coling-main.574/

Learn Before

Related