Actuator coordination for legged mobile robots using reinforcement learning: simulation and implementation

Author(s)

Jeeves Lopes dos Santos ^[1] ; Cairo Lúcio Nascimento Júnior ^[2]

Affiliation

^[1] Instituto Tecnológico de Aeronáutica. Divisão de Engenharia Eletrônica. Laboratório de Máquinas Inteligentes - Brasil

^[2] Instituto Tecnológico de Aeronáutica. Divisão de Engenharia Eletrônica. Laboratório de Máquinas Inteligentes - Brasil

• Document type: Journal article
• Source: Sba : Controle & Automação; v. 23, n. 1, p. 78-93, 2012-02-00.

Abstract

This article presents a solution to the problem of how to coordinate the actuators of a legged robot such that its frontal speed is maximized. It is assumed that the position of each leg actuator is described by a periodic function that has to be determined using a reinforcement learning technique called Learning Automata. Analysis of the robot morphology is used to group similar legs and decrease the number of actuator functions that must be determined. MATLAB/Simulink and the SimMechanics Toolbox are used to simulate the robot walking on a flat surface. The simulated robot response is evaluated by the reinforcement learning technique considering: 1) the robot frontal speed, 2) the smoothness of the robot movements, 3) the largest torque required by all actuators, and 4) the energy consumption. After the reinforcement learning algorithm converges to a solution, the actuators functions are applied to the real robot that was built using the Bioloid Comprehensive Kit, an educational robot kit manufactured by Robotis. The response of the real robot is then evaluated and compared with the simulated robot response. This article presents two case studies: a quadrupedal robot and a tripedal robot. In both cases, each leg has three actuators. The solutions obtained by the proposed methodology are presented and shown to be satisfactory. (AU)