Jose L. Part

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor skills as well as to long-term memory consolidation and retrieval. Consequently, lifelong learning capabilities are crucial for computational learning systems and autonomous agents interacting in the real world and processing continuous streams of information. However, lifelong learning remains a long-standing challenge for machine learning and neural network models since the continual acquisition of incrementally available information from non-stationary data distributions generally leads to catastrophic forgetting or interference. This limitation represents a major drawback for state-of-the-art deep neural network models that typically learn representations from stationary batches of training data, thus without accounting for situations in which information becomes incrementally available over time. In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. Although significant advances have been made in domain-specific learning with neural networks, extensive research efforts are required for the development of robust lifelong learning on autonomous agents and robots. We discuss well-established and emerging research motivated by lifelong learning factors in biological systems such as structural plasticity, memory replay, curriculum and transfer learning, intrinsic motivation, and multisensory integration.

We describe our Alexa prize system (called ‘Alana’) which consists of an ensemble of bots, combining rule-based and machine learning systems, and using a contextual ranking mechanism to choose system responses. This paper reports on the version of the system developed and evaluated in the semi-finals of the competition (i.e. up to 15 August 2017), but not on subsequent enhancements. The ranker for this system was trained on real user feedback received during the competition, where we address the problem of how to train on the noisy and sparse feedback obtained during the competition. In order to avoid initial problems of inappropriate and boring utterances coming from big datasets such as Reddit and Twitter, we later focussed on ‘clean’ data sources such as news and facts. We report on experiments with different ranking functions and versions of our NewsBot. We find that a multiturn news strategy is beneficial, and that a ranker trained on the ratings feedback from users is also effective. Our system continuously improved using the data gathered over the course over the competition (1 July – 15 August) . Our final user score (averaged user rating over the whole semi-finals period) was 3.12, and we achieved 3.3 for the averaged user rating over the last week of the semi-finals (8-15 August 2017). We were also able to achieve long dialogues (average 10.7 turns) during the competition period. In subsequent weeks, after the end of the semi-final competition, we have achieved our highest scores of 3.52 (daily average, 18th October), 3.45 (weekly average on 23 and 24 October), and average dialogue lengths of 14.6 turns (1 October), and median dialogue length of 2.25 minutes (average for 7 days on 10th October).

The ability of an object classifier to adapt to new data and incorporate new classes on the fly is of paramount importance for robots operating in the real world. This paper presents an approach for incremental online learning of real-world objects to be used by robots operating in real environments. We combined the representational power of Convolutional Neural Networks with the adaptability features of Self-Organizing Incremental Neural Networks. We evaluated our approach on the RGB-D Object Dataset in terms of classification accuracy and incremental learning of new classes. Our results show that whereas our method does not yet compete with the performance of state-of-the-art batch learning algorithms, it offers the important advantage of being able to adapt to new data and incorporate new classes on the fly. Finally, we aim at establishing a baseline on a publicly available dataset for comparing different approaches to realize online incremental learning in the context of robotics.

Enabling a robot to properly interact with users plays a key role in the effective deployment of robotic platforms in domestic environments. Robots must be able to rely on interaction to improve their behaviour and adaptively understand their operational world. Semantic mapping is the task of building a representation of the environment, that can be enhanced through interaction with the user. In this task, a proper and effective acquisition of semantic attributes of targeted entities is essential for the task accomplishment itself. In this paper, we focus on the problem of learning dialogue policies to support semantic attribute acquisition, so that the effort required by humans in providing knowledge to the robot through dialogue is minimized. To this end, we design our Dialogue Manager as a multi-objective Markov Decision Process, solving the optimisation problem through Reinforcement Learning. The Dialogue Manager interfaces with an online incremental visual classifier, based on a Load-Balancing Self-Organizing Incremental Neural Network (LB-SOINN). Experiments in a simulated scenario show the effectiveness of the proposed solution, suggesting that perceptual information can be properly exploited to reduce human tutoring cost. Moreover, a dialogue policy trained on a small amount of data generalises well to larger datasets, and so the proposed online scheme, as well as the real-time nature of the processing, are suited for an extensive deployment in real scenarios. To this end, this paper provides a demonstration of the complete system on a real robot.

Open-domain social dialogue is one of the long-standing goals of Artificial Intelligence. This year, the Amazon Alexa Prize challenge was announced for the first time, where real customers get to rate systems developed by leading universities worldwide. The aim of the challenge is to converse "coherently and engagingly with humans on popular topics for 20 minutes". We describe our Alexa Prize system (called 'Alana') consisting of an ensemble of bots, combining rule-based and machine learning systems, and using a contextual ranking mechanism to choose a system response. The ranker was trained on real user feedback received during the competition, where we address the problem of how to train on the noisy and sparse feedback obtained during the competition.