Image Augmentation Is All You Need: Regularizing Deep Reinforcement\n Learning from Pixels

Abstract

We propose a simple data augmentation technique that can be applied to\nstandard model-free reinforcement learning algorithms, enabling robust learning\ndirectly from pixels without the need for auxiliary losses or pre-training. The\napproach leverages input perturbations commonly used in computer vision tasks\nto regularize the value function. Existing model-free approaches, such as Soft\nActor-Critic (SAC), are not able to train deep networks effectively from image\npixels. However, the addition of our augmentation method dramatically improves\nSAC's performance, enabling it to reach state-of-the-art performance on the\nDeepMind control suite, surpassing model-based (Dreamer, PlaNet, and SLAC)\nmethods and recently proposed contrastive learning (CURL). Our approach can be\ncombined with any model-free reinforcement learning algorithm, requiring only\nminor modifications. An implementation can be found at\nhttps://sites.google.com/view/data-regularized-q.\n