Dynamically decides whether to reuse existing neurons, add new ones, or split neurons when learning a new task. By selectively expanding the network architecture based on task requirements, DEN balances model capacity with task complexity.
2016
Allocates a new neural network column for each new task and freezes all previous columns. Lateral connections between columns allow forward transfer of features from old tasks to new ones, while the frozen parameters guarantee zero forgetting of previously learned tasks.
2018
Iteratively assigns free parameters to new tasks through network pruning. After training on a task, PackNet prunes unimportant weights and freezes the remaining ones, freeing capacity for future tasks. This packing strategy achieves zero forgetting while efficiently utilizing network capacity.
2020
Learns a separate binary mask (supermask) for each task applied to a randomly initialized, fixed backbone network. Task identity determines which mask to apply, enabling zero-interference between tasks. Multiple tasks can be handled without growing the network's parameter count.