Synaptic regularization methods are focused on reducing interference with the consolidated knowledge by adjusting learning rates in a way that prevents changes to previously learned model parameters.

University of Michigan - Ann Arbor

Memory methods are a special case of regularization methods that can be divided into two groups: 
- Synaptic regularization methods
- Episodic memory methods

Memory Methods

Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020. Continual Lifelong Learning in Natural Language Processing: A Survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6523–6541.
https://aclanthology.org/2020.coling-main.574/

Continual Lifelong Learning in Natural Language Processing: A Survey

Synaptic Regularization Methods

Episodic memory methods store training samples from previously seen data, which are later rehearsed to allow learning new classes.

Learn Before

Related