Massimo Caccia and Pau Rodriguez and Oleksiy Ostapenko and Fabrice Normandin and Min Lin and Lucas Caccia and Issam Laradji and Irina Rish and Alexande Lacoste and David Vazquez and Laurent Charlin
arXiv e-Print archive - 2020 via Local arXiv
Keywords: cs.AI, cs.LG
First published: 2020/03/12 (5 years ago)
Abstract: Learning from non-stationary data remains a great challenge for machine learning. Continual learning addresses this problem in scenarios where the learning agent faces a stream of changing tasks. In these scenarios, the agent is expected to retain its highest performance on previous tasks without revisiting them while adapting well to the new tasks. Two new recent continual-learning scenarios have been proposed. In meta-continual learning, the model is pre-trained to minimize catastrophic forgetting when trained on a sequence of tasks. In continual-meta learning, the goal is faster remembering, i.e., focusing on how quickly the agent recovers performance rather than measuring the agent's performance without any adaptation. Both scenarios have the potential to propel the field forward. Yet in their original formulations, they each have limitations. As a remedy, we propose a more general scenario where an agent must quickly solve (new) out-of-distribution tasks, while also requiring fast remembering. We show that current continual learning, meta learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. Accordingly, we propose a strong baseline: Continual-MAML, an online extension of the popular MAML algorithm. In our empirical experiments, we show that our method is better suited to the new scenario than the methodologies mentioned above, as well as standard continual learning and meta learning approaches.
[link]
disclaimer: I'm the first author of the paper
## TL;DR
We have made a lot of progress on catastrophic forgetting within the standard evaluation protocol,
i.e. sequentially learning a stream of tasks and testing our models' capacity to remember them all.
We think it's time a new approach to Continual Learning (CL), coined OSAKA, which is more aligned with real-life applications of CL. It brings CL closer to Online Learning and Open-World Learning.
main modifications we propose:
- bring CL closer to Online learning i.e. at test time, the model is continually learning and evaluated on its online predictions
- it's fine to forget, as long as you can quickly remember (just like we humans do)
- we allow pretraining, (because you wouldn't deploy an untrained CL system, right?) but at test time, the model will have to quickly learn new out-of-distribution (OoD) tasks (because the world is full of surprises)
- the tasks distribution is actually a hidden Markov chain. This implies:
- new and old tasks can re-occur (just like in real life). Better remember them quickly if you want to get a good total performance!
- tasks have different lengths
- and the tasks boundaries are unknown (task agnostic setting)
### Bonus:
We provide a unifying framework explaining the space of machine learning setting {supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning} in case it was starting to get confusing :p
## Motivation
We imagine an agent, embedded or not, first pre-trained in a controlled environment and later deployed in the real world, where it faces new or unexpected situations. This scenario is relevant for many applications. For instance, in robotics, the agent is pre-trained in a factory and deployed in homes or
in manufactures where it will need to adapt to new domains and maybe solve new tasks. Likewise, a virtual assistant can be pre-trained on static datasets and deployed in a user’s life to fit its personal needs.
Further motivations can be found in time series forecasting, e.g., market prediction,
game playing, autonomous customer service, recommendation systems, autonomous driving, to name a few. In this scenario, we are interested in the cumulative performance of the agent throughout its lifetime. Differently, standard CL reports the agent’s final performance on all tasks at the end of its life. In order
to succeed in this scenario, agents need the ability to learn new tasks as well as quickly remembering old ones.
## Unifying Framework
We propose a unifying framework explaining the space of machine learning setting {supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning} with meta learning terminology.
https://i.imgur.com/U16kHXk.png
(easier to digest with accompanying text)
## OSAKA
The main features of the evaluation framework are
- task agnosticism
- pre-training is allowed, but OoD tasks at test time
- task revisiting
- controllable non-stationarity
- online evaluation
(see paper for the motivations of the features)
## Continual-MAML: an initial baseline
A simple extension of MAML that is better suited than previous methods in the proposed setting.
https://i.imgur.com/C86WUc8.png
Features are:
- Fast Adapatation
- Dynamic Representation
- Task boundary detection
- Computational efficiency
## Experiments
We provide a suite of 3 benchmarks to test algorithms in the new setting.
The first includes the Omniglot, MNIST and FashionMNIST dataset.
The second and third use the Synbols (Lacoste et al. 2018) and TieredImageNet datasets, respectively.
The first set of experiments shows that the baseline outperforms previous approaches, i.e., supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning, in the new setting.
https://i.imgur.com/IQ1WYTp.png
The second and third experiments lead us to similar conclusions
code: https://github.com/ElementAI/osaka
disclaimer: I'm the first author of the paper TL;DRWe have made a lot of progress on catastrophic forgetting within the standard evaluation protocol, i.e. sequentially learning a stream of tasks and testing our models' capacity to remember them all. We think it's time a new approach to Continual Learning (CL), coined OSAKA, which is more aligned with real-life applications of CL. It brings CL closer to Online Learning and Open-World Learning. main modifications we propose:
Bonus:We provide a unifying framework explaining the space of machine learning setting {supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning} in case it was starting to get confusing :p MotivationWe imagine an agent, embedded or not, first pre-trained in a controlled environment and later deployed in the real world, where it faces new or unexpected situations. This scenario is relevant for many applications. For instance, in robotics, the agent is pre-trained in a factory and deployed in homes or in manufactures where it will need to adapt to new domains and maybe solve new tasks. Likewise, a virtual assistant can be pre-trained on static datasets and deployed in a user’s life to fit its personal needs. Further motivations can be found in time series forecasting, e.g., market prediction, game playing, autonomous customer service, recommendation systems, autonomous driving, to name a few. In this scenario, we are interested in the cumulative performance of the agent throughout its lifetime. Differently, standard CL reports the agent’s final performance on all tasks at the end of its life. In order to succeed in this scenario, agents need the ability to learn new tasks as well as quickly remembering old ones. Unifying FrameworkWe propose a unifying framework explaining the space of machine learning setting {supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning} with meta learning terminology.
(easier to digest with accompanying text) OSAKAThe main features of the evaluation framework are
(see paper for the motivations of the features) Continual-MAML: an initial baselineA simple extension of MAML that is better suited than previous methods in the proposed setting.
Features are:
ExperimentsWe provide a suite of 3 benchmarks to test algorithms in the new setting. The first includes the Omniglot, MNIST and FashionMNIST dataset. The second and third use the Synbols (Lacoste et al. 2018) and TieredImageNet datasets, respectively. The first set of experiments shows that the baseline outperforms previous approaches, i.e., supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning, in the new setting.
The second and third experiments lead us to similar conclusions
Your comment:
|
