Heat It Up: Using Robo-gym to Warm-up Deep Reinforcement Learning Algorithms

Autor(es)

Bermudez, Gabriel ; do Carmo Alves, Matheus A. ; Pedro, Gabriel D. G. ; Boaventura, Thiago

• Tipo de documento: Artigo Científico
• Fonte: 2024 LATIN AMERICAN ROBOTICS SYMPOSIUM, LARS 2024; v. N/A, p. 6-pg., 2024-01-01.

Resumo

Training deep Reinforcement Learning (deep RL) algorithms in robotics often requires acquiring a large amount of data, which is a challenging and expensive process. Although there is a gap between simulated and real environments, simulated environments enable safe and continuous data collection with minimal human intervention. In this work, we present a consistent and lightweight approach for training deep RL algorithms in a complex robotic context. Our proposal adapts the robo-gym framework to run a hybrid training process, performing a warm-up using only data from simulations (OpenAI Gym) before incorporating the complexity of the robotics models (ROS and Gazebo). We study the classic inverted pendulum swing-up problem using three different state-of-the-art baselines. Overall, our approach can significantly improve the learning process, boosting the training quality up to 26% by performing warm-ups. Our quickest warm-up takes only 2 minutes and can improve the initial learning point by up to 83%, saving 91% of training time to reach the same reward with a traditional approach. (AU)

• Processo FAPESP: 18/15472-9 - Controle de impedância de atuadores hidráulicos para robôs com pernas e braços
• Beneficiário: Thiago Boaventura Cunha
• Modalidade de apoio: Auxílio à Pesquisa - Jovens Pesquisadores