Reusing previous task solutions in multiagent reinforcement learning

• Grant number: 18/00344-5
• Support type: Scholarships abroad - Research Internship - Doctorate
• Effective date (Start): April 01, 2018
• Effective date (End): March 31, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Anna Helena Reali Costa
• Grantee: Felipe Leno da Silva
• Supervisor abroad: Peter Stone
• Home Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Local de pesquisa : University of Texas at Austin (UT), United States
• Associated to the scholarship: 15/16310-4 - Transfer learning in reinforcement learning multi-agent systems, BP.DR

Abstract

Reinforcement Learning (RL) is a popular solution to train autonomous agents to solve tasks in environments with unknowndynamics. Despite the recent successes of RL in solving increasingly challenging tasks, it suffers from an excessive requirementof samples of interactions with the environment in order to learn a good solution. When applied to Multiagent Systems (MAS), RLallows to train agents to solve tasks while coordinating with other agents. However, the sample complexity of MAS solutions iseven higher, and hence Multiagent RL is dependent on additional techniques to be applicable to complex tasks. We here rely onTransfer Learning methods to accelerate Multiagent RL through knowledge reuse. In the main Ph.D. project we proposed aframework in which agents reuse knowledge both from previously learned tasks and advice from other agents. The main purposeof this exchange project is to develop methods to reuse previous task solutions in order to later integrate them in theaforementioned framework. Recently, the use of Curriculum Learning in Single-Agent RL achieved interesting benefits for thelearning process, accelerating the learning process of complex tasks. We here intend to contribute Curriculum Learningalgorithms specialized to Multiagent RL. This is a resubmission of process 2017/13729-0, approved by FAPESP but not realized because of the long time required for issuing a US visa.

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA. A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems. JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, v. 64, p. 645-703, 2019. Web of Science Citations: 1.