Inspired by the Complementary Learning Systems theory in neuroscience, CLS-ER maintains two semantic memory models updated at different rates alongside an episodic memory buffer. The fast and slow learners work together to balance stability and plasticity in continual learning.
2020
A surprisingly effective baseline that greedily stores samples in a memory buffer using class-balanced reservoir sampling, then trains a model from scratch on only the buffer contents at evaluation time. Despite its simplicity, GDumb matches or outperforms many sophisticated continual learning methods.
2021
Combines replay with prototype-based learning by maintaining evolving class prototypes in a learned embedding space. Prototypes are updated using a momentum mechanism as new data arrives, enabling efficient nearest-prototype classification in continual learning settings.
2017
Trains a separate expert network for each task and uses an autoencoder-based gating mechanism to select the appropriate expert at test time. By combining task-specific experts with a learned routing mechanism, Expert Gate achieves strong performance without requiring task identity at inference.