Gido M. van de Ven

Incrementally learning new information from a non-stationary stream of data, referred to as 'continual learning', is a key feature of natural intelligence, but a challenging problem for deep neural networks. In recent years, numerous deep learning methods for continual learning have been proposed, but comparing their performances is difficult due to the lack of a common framework. To help address this, we describe three fundamental types, or 'scenarios', of continual learning: task-incremental, domain-incremental and class-incremental learning. Each of these scenarios has its own set of challenges. To illustrate this, we provide a comprehensive empirical comparison of currently used continual learning strategies, by performing the Split MNIST and Split CIFAR-100 protocols according to each scenario. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of the effectiveness of different strategies. The proposed categorization aims to structure the continual learning field, by forming a key foundation for clearly defining benchmark problems.

Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe three continual learning scenarios based on whether at test time task identity is provided and--in case it is not--whether it must be inferred. Any sequence of well-defined tasks can be performed according to each scenario. Using the split and permuted MNIST task protocols, for each scenario we carry out an extensive comparison of recently proposed continual learning methods. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of how efficient different methods are. In particular, when task identity must be inferred (i.e., class incremental learning), we find that regularization-based approaches (e.g., elastic weight consolidation) fail and that replaying representations of previous experiences seems required for solving this scenario.

Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those memories. In artificial neural networks, such memory replay can be implemented as 'generative replay', which can successfully - and surprisingly efficiently - prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario. However, scaling up generative replay to complicated problems with many tasks or complex inputs is challenging. We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network's own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks (e.g., class-incremental learning on CIFAR-100) without storing data, and it provides a novel model for replay in the brain.

The ability to reinstate neuronal assemblies representing mnemonic information is thought to require their consolidation through offline reactivation during sleep/rest. To test this, we detected cell assembly patterns formed by repeated neuronal co-activations in the mouse hippocampus during exploration of spatial environments. We found that the reinstatement of assembly patterns representing a novel, but not a familiar, environment correlated with their offline reactivation and was impaired by closed-loop optogenetic disruption of sharp wave-ripple oscillations. Moreover, we discovered that reactivation was only required for the reinstatement of assembly patterns whose expression was gradually strengthened during encoding of a novel place. The context-dependent reinstatement of assembly patterns whose expression did not gain in strength beyond the first few minutes of spatial encoding was not dependent on reactivation. This demonstrates that the hippocampus can hold concurrent representations of space that markedly differ in their encoding dynamics and their dependence on offline reactivation for consolidation.Video AbstracteyJraWQiOiI4ZjUxYWNhY2IzYjhiNjNlNzFlYmIzYWFmYTU5NmZmYyIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiI2NGM2MzA3ZWQ0NjY1ODI1YTEzZWMxNjZmOTM0YTcwNiIsImtpZCI6IjhmNTFhY2FjYjNiOGI2M2U3MWViYjNhYWZhNTk2ZmZjIiwiZXhwIjoxNjc5NTI1NjQ3fQ.qMBzoMWhkGM9Px-KMy6xUpszcRfk4Vk9ovilJTZh22a8igEB3m5SfM2HUquK7wFP1xZC8dtiID8rCIFXLKPQiiGgi7JU3epoW2lPOw4eGoCieobYEkVblYX-D4fTo71FMEAd0KC-4MRUR-dgCG-xOLTA3v0Budd4Z_NqklcH1LBmZcNjfGKmmb26Y-EzwvTud35QdILLeQU2QrL-Ls2qtWXY0UZM3ufKv_n-F9SiCbh_8c37l_lrAynXR18p898UKBb37Z-1LPldHLRMjczLZUAsmE9eWOt-heSTWCNRiB-H8Q9CSd-A4lGAtrT9hNCf4Z6BowWZAoQjHL09gAAbfg(mp4, (30.51 MB) Download video