Transfer Learning in Reinforcement Learning Multi-Agent Systems

• Grant number: 15/16310-4
• Support Opportunities: Scholarships in Brazil - Doctorate
• Effective date (Start): December 01, 2015
• Effective date (End): September 30, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Anna Helena Reali Costa
• Grantee: Felipe Leno da Silva
• Host Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated scholarship(s): 18/00344-5 - Reusing previous task solutions in multiagent reinforcement learning, BE.EP.DR

Abstract

Reinforcement Learning is a powerful to train intelligent agents because the learning is performed completely autonomously. This learning is accomplished through repetitive interactions between agents and the environment through trial and error, until agents have enough information to properly actuate in order to solve a given task. However, an agent can take a long time to determine which actions are more appropriate for each situation. In order to work around this problem, researchers have started to utilize Transfer Learning solutions in which, after learning a task, the acquired knowledge is reused to accelerate the learning of a new similar task. If multiple agents are acting at the same time in an environment, a fault robust, scalable and highly parallel system can be obtained. However, in this case new problems arise, for example, the state space explosion and the difficulty of predicting consequences of joint actions. Researches proposed partial solutions for these problems, in which Transfer Learning has been proved to be beneficial also to multi-agent domains. Yet, the existing Transfer Learning methods must be improved to allow their application in complex domains. This research aims to propose methods, which deal some questions that have been only superficially answered by the state of art methods. Among these question are: How to properly abstract the knowledge acquired in the learning? How to represent this knowledge? How communicate between agents to transmit learned task knowledge? How to deal with partial observability?

Scientific publications

(7)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA. A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems. JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, v. 64, p. 645-703, 2019. (18/00344-5, 16/21047-3, 15/16310-4)

DA SILVA, FELIPE LENO; WARNELL, GARRETT; COSTA, ANNA HELENA REALI; STONE, PETER. Agents teaching agents: a survey on inter-agent transfer learning. AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS, v. 34, n. 1, APR 2020. (15/16310-4, 18/00344-5)

JACOMINI, RICARDO DE SOUZA; MARTINS, JR., DAVID CORREA; DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA. GeNICE: A Novel Framework for Gene Network Inference by Clustering, Exhaustive Search, and Multivariate Analysis. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 24, n. 8, p. 809-830, AUG 2017. (16/21047-3, 11/50761-2, 15/16310-4, 15/01587-0)

DA SILVA, FELIPE LENO; NISHIDA, CYNTIA E. H.; ROIJERS, DIEDERIK M.; COSTA, ANNA H. REALI. Coordination of Electric Vehicle Charging Through Multiagent Reinforcement Learning. IEEE TRANSACTIONS ON SMART GRID, v. 11, n. 3, p. 2347-2356, MAY 2020. (15/16310-4, 18/00344-5)

DA SILVA, FELIPE LENO; GLATT, RUBEN; REALI COSTA, ANNA HELENA. MOO-MDP: An Object-Oriented Representation for Cooperative Multiagent Reinforcement Learning. IEEE TRANSACTIONS ON CYBERNETICS, v. 49, n. 2, p. 567-579, FEB 2019. (16/21047-3, 15/16310-4)

GLATT, RUBEN; DA SILVA, FELIPE LENO; DA COSTA BIANCHI, REINALDO AUGUSTO; REALI COSTA, ANNA HELENA. DECAF: Deep Case-based Policy Inference for knowledge transfer in Reinforcement Learning. EXPERT SYSTEMS WITH APPLICATIONS, v. 156, OCT 15 2020. (16/21047-3, 15/16310-4, 18/00344-5, 16/18792-9)

PERAFAN VILLOTA, JUAN CARLOS; DA SILVA, FELIPE LENO; JACOMINI, RICARDO DE SOUZA; REALI COSTA, ANNA HELENA. Pairwise registration in indoor environments using adaptive combination of 2D and 3D cues. Image and Vision Computing, v. 69, p. 113-124, JAN 2018. (16/21047-3, 15/16310-4)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

SILVA, Felipe Leno da. Methods and algorithms for knowledge reuse in multiagent reinforcement learning.. 2019. Doctoral Thesis - Universidade de São Paulo (USP). Escola Politécnica (EP/BC) São Paulo.