Continual Learning Landscape

ReplayVisionBenchmark: CIFAR100+1

Stores a subset of past training samples in a memory buffer and replays them alongside new task data during training. This interleaving of old and new examples helps the model retain previously learned representations while adapting to new tasks, effectively combating catastrophic forgetting through data rehearsal.

Generative Replay (DGR)

ReplayGenerativePrivacy+1

Uses a generative model (e.g., GAN or VAE) to synthesize pseudo-samples from previous tasks instead of storing raw data. The generator learns the distribution of past tasks, enabling unlimited replay without requiring a memory buffer, making it particularly suitable for privacy-sensitive applications.

Gradient Episodic Memory (GEM)

ReplayGradient ConstraintVision+1

Stores a small episodic memory of past examples and uses them to constrain gradient updates during training on new tasks. GEM projects the gradient onto a feasible region that does not increase the loss on previous tasks, providing formal guarantees against forgetting.

Averaged GEM (A-GEM)

ReplayEfficientGradient Constraint+1

An efficient variant of GEM that replaces the per-task gradient constraints with a single averaged constraint computed from a random subset of the episodic memory. This dramatically reduces computational overhead while maintaining comparable performance to GEM.

Dark Experience Replay (DER/DER++)

ReplayKnowledge DistillationVision+1

Extends experience replay by storing not just raw input-output pairs but also the model's logits (dark knowledge) at the time of storage. By matching both hard labels and soft logits during replay, DER++ achieves superior knowledge retention and consistently outperforms vanilla experience replay.

Maximally Interfered Retrieval (MIR)

ReplaySample SelectionOnline+1

Proposes a controlled memory retrieval strategy that selects the samples from the buffer that would be most interfered with (i.e., suffer the largest increase in loss) by the incoming parameter update, thereby focusing replay on the most vulnerable memories.

Regularization-based

Elastic Weight Consolidation (EWC)

RegularizationFisher InformationNeuroscience-inspired+1

Adds a quadratic penalty term to the loss function that discourages changes to parameters important for previous tasks. Importance is estimated using the diagonal of the Fisher Information Matrix, computed after training on each task. This allows the model to learn new tasks while preserving critical weights.

Synaptic Intelligence (SI)

RegularizationOnlineNeuroscience-inspired+1

Computes parameter importance online during training by accumulating the contribution of each weight to the reduction in loss along the optimization trajectory. Unlike EWC which requires a separate importance computation phase, SI tracks importance in real-time, making it more computationally efficient.

Learning without Forgetting (LwF)

RegularizationDistillationVision+1

Uses knowledge distillation to preserve the predictions of the old model on new task data. Before training on a new task, the current model's outputs on the new data are recorded as soft targets. A distillation loss then ensures the updated model stays close to these original predictions.

Memory Aware Synapses (MAS)

RegularizationUnsupervisedOnline+1

Estimates parameter importance based on how sensitive the learned function output is to parameter changes, using the gradient of the squared L2 norm of the network output. Unlike EWC, MAS computes importance in an unsupervised manner, not requiring task-specific labels.

Riemannian Walk (RWalk)

RegularizationFisher InformationPath Integral+1

Combines the benefits of EWC and path integral-based methods (like SI) by computing parameter importance using both the Fisher information and the contribution of each weight along the optimization path. This unified framework provides more robust importance estimates.

Architecture-based

Progressive Neural Networks

2016

Allocates a new neural network column for each new task and freezes all previous columns. Lateral connections between columns allow forward transfer of features from old tasks to new ones, while the frozen parameters guarantee zero forgetting of previously learned tasks.

ArchitectureZero ForgettingRL+1

PackNet

ArchitecturePruningZero Forgetting+1

Iteratively assigns free parameters to new tasks through network pruning. After training on a task, PackNet prunes unimportant weights and freezes the remaining ones, freeing capacity for future tasks. This packing strategy achieves zero forgetting while efficiently utilizing network capacity.

Dynamically Expandable Networks (DEN)

ArchitectureDynamic ExpansionSelective+1

Dynamically decides whether to reuse existing neurons, add new ones, or split neurons when learning a new task. By selectively expanding the network architecture based on task requirements, DEN balances model capacity with task complexity.

SupSup (Supermasks in Superposition)

ArchitectureMaskingZero Forgetting+1

Learns a separate binary mask (supermask) for each task applied to a randomly initialized, fixed backbone network. Task identity determines which mask to apply, enabling zero-interference between tasks. Multiple tasks can be handled without growing the network's parameter count.

Hard Attention to the Task (HAT)

ArchitectureAttention MaskingTask-specific+1

Learns a hard attention mask for each task that selectively gates hidden units. During training on new tasks, a gradient compensation mechanism prevents updates to units that are important for previous tasks. This provides near-zero forgetting with minimal capacity overhead.

Meta-learning

OML (Online Meta-Learning)

Meta-learningMAMLFew-shot+1

Adapts Model-Agnostic Meta-Learning (MAML) for continual learning by meta-learning representations that are robust to catastrophic forgetting. The learned representations enable fast adaptation to new tasks with minimal interference to previously learned ones.

A Neuromodulated Meta-Learning Algorithm (ANML)

Meta-learningNeuromodulationFew-shot+1

Uses a neuromodulatory network to gate the activations of a prediction network during meta-learning. This biologically-inspired gating mechanism learns to protect important representations while allowing plasticity for new learning, achieving strong continual few-shot performance.

La-MAML (Look-ahead MAML)

2021

Proposes a look-ahead variant of MAML for online continual learning that modulates per-parameter learning rates. By meta-learning the learning rate for each parameter, La-MAML can dynamically control plasticity and stability on a per-parameter basis.

Meta-learningMAMLOnline+1

Meta-Experience Replay (MER)

Meta-learningReplayReptile+1

Combines experience replay with meta-learning by formulating the replay buffer optimization as a meta-learning problem. MER treats each training example as a separate task and uses Reptile-style meta-updates to find parameters that work well across all examples in the buffer.

Hybrid

GDumb (Greedy Sampler, Dumb Learner)

A surprisingly effective baseline that greedily stores samples in a memory buffer using class-balanced reservoir sampling, then trains a model from scratch on only the buffer contents at evaluation time. Despite its simplicity, GDumb matches or outperforms many sophisticated continual learning methods.

HybridBaselineSimple+2

CoPE (Continual Prototype Evolution)

2021

Combines replay with prototype-based learning by maintaining evolving class prototypes in a learned embedding space. Prototypes are updated using a momentum mechanism as new data arrives, enabling efficient nearest-prototype classification in continual learning settings.

HybridPrototypeReplay+1

Expert Gate