Synaptic regularization methods are focused on reducing interference with the consolidated knowledge by adjusting learning rates in a way that prevents changes to previously learned model parameters.

Synaptic Regularization Methods

Episodic memory methods store training samples from previously seen data, which are later rehearsed to allow learning new classes.

Episodic Memory Methods

Memory methods are a special case of regularization methods that can be divided into two groups: 
- Synaptic regularization methods
- Episodic memory methods

University of Michigan - Ann Arbor

Regularization methods are single-model approaches that rely on a fixed model capacity with an additional loss term that aids knowledge consolidation while learning subsequent tasks or data distributions.

Regularization Methods

Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020. Continual Lifelong Learning in Natural Language Processing: A Survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6523–6541.
https://aclanthology.org/2020.coling-main.574/

Learn Before

Related

Learn After