ALIS: Autonomous Learning in Intelligent System

• Grant number: 16/21047-3
• Support Opportunities: Regular Research Grants
• Start date: February 01, 2017
• End date: January 31, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Anna Helena Reali Costa
• Grantee: Anna Helena Reali Costa
• Host Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated researchers: Reinaldo Augusto da Costa Bianchi

Abstract

Intelligent systems are machines that have their own goals, perceive, respond and learn based on their experiences. Reinforcement Learning (RL) is a powerful tool for this purpose because the system autonomously learns a policy, through trial and error in repeated interactions with the environment. This project seeks to increase the dissemination of the RL technology and advance the frontiers of knowledge of the self-learning area. However, many challenges must still be overcome in order to have a broad use of RL in intelligent systems. Challenges include dealing with uncertainties of sensors and actuators, a dynamic world that changes continuously and requires quick decisions, continuous quantities and the high RL computational cost. Therefore, this scientific research project aims toinvestigate, propose, develop, and evaluate models and methods to make efficient and effective RL in intelligent systems that solve complex problems. In particular, it explores: (I) relational and object-oriented models and algorithms, opening opportunities for generalization in the representation and solution of complex problems; (Ii) distribution and division of workload among several apprentices agents; (Iii) appropriate approximation functions to represent both situations observed by the agent and the knowledge acquired; (Iv) knowledge transfer so that the knowledge acquired byanother agent or after the learning of previous tasks can be reused to accelerate the learning of a new similar task. From the point of view of applications, this project aims to implement and evaluate models and algorithms in areas such as games, robotics, computational biology, among others. (AU)

Scientific publications

(24)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

BIANCHI, REINALDO A. C.; SANTOS, PAULO E.; DA SILVA, ISAAC J.; CELIBERTO, JR., LUIZ A.; DE MANTARAS, RAMON LOPEZ. Heuristically Accelerated Reinforcement Learning by Means of Case-Based Reasoning and Transfer Learning. JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS, v. 91, n. 2, SI, p. 301-312, AUG 2018. (16/21047-3, 16/18792-9)

JACOMINI, RICARDO DE SOUZA; MARTINS, JR., DAVID CORREA; DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA. GeNICE: A Novel Framework for Gene Network Inference by Clustering, Exhaustive Search, and Multivariate Analysis. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 24, n. 8, p. 809-830, AUG 2017. (16/21047-3, 11/50761-2, 15/16310-4, 15/01587-0)

DOS SANTOS, THIAGO FREITAS; SANTOS, PAULO E.; FERREIRA, LEONARDO A.; BIANCHI, REINALDO A. C.; CABALAR, PEDRO; IEEE. Solving a spatial puzzle using Answer Set Programming integrated with Markov Decision Process. 2018 7TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2018-01-01. (17/07833-9, 16/18792-9, 16/21047-3)

BIANCHI, REINALDO A. C.; SANTOS, PAULO E.; DA SILVA, ISAAC J.; CELIBERTO, LUIZ A., JR.; DE MANTARAS, RAMON LOPEZ. Heuristically Accelerated Reinforcement Learning by Means of Case-Based Reasoning and Transfer Learning. JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS, v. 91, n. 2, p. 12-pg., 2018-08-01. (16/21047-3, 16/18792-9)

DA SILVA, FELIPE LENO; ASSOC COMP MACHINERY. Integrating Agent Advice and Previous Task Solutions in Multiagent Reinforcement Learning. AAMAS '19: PROCEEDINGS OF THE 18TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, v. N/A, p. 2-pg., 2019-01-01. (15/16310-4, 16/21047-3, 18/00344-5)

GLATT, RUBEN; DA SILVA, FELIPE LENO; DA COSTA BIANCHI, REINALDO AUGUSTO; REALI COSTA, ANNA HELENA. DECAF: Deep Case-based Policy Inference for knowledge transfer in Reinforcement Learning. EXPERT SYSTEMS WITH APPLICATIONS, v. 156, OCT 15 2020. (16/21047-3, 15/16310-4, 18/00344-5, 16/18792-9)

HOMEM, THIAGO P. D.; PERICO, DANILO H.; SANTOS, PAULO E.; COSTA, ANNA H. R.; BIANCHI, REINALDO A. C.; DE MANTARAS, RAMON LOPEZ; TODT, E; TONIDANDEL, F. A hybrid approach to learn, retrieve and reuse qualitative cases. 2017 LATIN AMERICAN ROBOTICS SYMPOSIUM (LARS) AND 2017 BRAZILIAN SYMPOSIUM ON ROBOTICS (SBR), v. N/A, p. 6-pg., 2017-01-01. (16/21047-3, 16/18792-9)

PERAFAN VILLOTA, JUAN CARLOS; DA SILVA, FELIPE LENO; JACOMINI, RICARDO DE SOUZA; REALI COSTA, ANNA HELENA. Pairwise registration in indoor environments using adaptive combination of 2D and 3D cues. Image and Vision Computing, v. 69, p. 113-124, JAN 2018. (16/21047-3, 15/16310-4)

DA SILVA, FELIPE LENO; GLATT, RUBEN; REALI COSTA, ANNA HELENA. MOO-MDP: An Object-Oriented Representation for Cooperative Multiagent Reinforcement Learning. IEEE TRANSACTIONS ON CYBERNETICS, v. 49, n. 2, p. 567-579, FEB 2019. (16/21047-3, 15/16310-4)

HOMEM, THIAGO P. D.; PERICO, DANILO H.; SANTOS, PAULO E.; COSTA, ANNA H. R.; BIANCHI, REINALDO A. C.; IEEE. Improving Reinforcement Learning results with Qualitative Spatial Representation. 2017 6TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2017-01-01. (16/18792-9, 16/21047-3)

ALMEIDA, AISLAN C.; COSTA, ANNA H. R.; BIANCHI, REINALDO A. C.; TODT, E; TONIDANDEL, F. Vision-based Monte-Carlo Localization for Humanoid Soccer Robots. 2017 LATIN AMERICAN ROBOTICS SYMPOSIUM (LARS) AND 2017 BRAZILIAN SYMPOSIUM ON ROBOTICS (SBR), v. N/A, p. 6-pg., 2017-01-01. (16/21047-3)

HAYAMA NISHIDA, CYNTIA EICO; REALI COSTA, ANNA HELENA; BIANCHI, REINALDO A. C.; IEEE. Controlling Gene Regulatory Networks with FQI-SARSA. 2017 6TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2017-01-01. (16/21047-3)

ALMEIDA, AISLAN C.; NETO, SYLVIO R. J.; BIANCHI, REINALDO A. C.; DONASCIMENTO, TP; COLOMBINI, EL; DEBRITO, AV; GARCIA, LTD; SA, STD; GONCALVES, LMG. Comparing Vision-based Monte-Carlo Localization Methods. 15TH LATIN AMERICAN ROBOTICS SYMPOSIUM 6TH BRAZILIAN ROBOTICS SYMPOSIUM 9TH WORKSHOP ON ROBOTICS IN EDUCATION (LARS/SBR/WRE 2018), v. N/A, p. 6-pg., 2018-01-01. (16/21047-3, 16/18792-9)

HAYAMA NISHIDA, CYNTIA EICO; REALI COSTA, ANNA HELENA; DA COSTA BIANCHI, REINALDO AUGUSTO; IEEE. Control of Gene Regulatory Networks Basin of Attractions with Batch Reinforcement Learning. 2018 7TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2018-01-01. (16/21047-3)

DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA; ACM. Object-Oriented Curriculum Generation for Reinforcement Learning. PROCEEDINGS OF THE 17TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS (AAMAS' 18), v. N/A, p. 9-pg., 2018-01-01. (18/00344-5, 16/21047-3, 15/16310-4)

PERICO, DANILO H.; HOMEM, THIAGO P. D.; ALMEIDA, AISLAN C.; SILVA, ISAAC J.; VILAO, JR., CLAUDIO O.; FERREIRA, VINICIUS N.; BIANCHI, REINALDO A. C.. Humanoid Robot Framework for Research on Cognitive Robotics. JOURNAL OF CONTROL AUTOMATION AND ELECTRICAL SYSTEMS, v. 29, n. 4, p. 470-479, AUG 2018. (16/21047-3, 16/18792-9)

DOS SANTOS, THIAGO FREITAS; SANTOS, PAULO E.; FERREIRA, LEONARDO ANJOLETTO; BIANCHI, REINALDO A. C.; CABALAR, PEDRO. euristics, Answer Set Programming and Markov Decision Process for Solving a Set of Spatial Puzzles{*. APPLIED INTELLIGENCE, v. 52, n. 4, JUL 2021. (16/21047-3, 17/07833-9, 16/18792-9)

BONINI, RODRIGO CESAR; DA SILVA, FELIPE LENO; GLATT, RUBEN; SPINA, EDISON; REALI COSTA, ANNA HELENA; IEEE. A Framework to Discover and Reuse Object-Oriented Options in Reinforcement Learning. 2018 7TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2018-01-01. (16/21047-3, 15/16310-4, 18/00344-5)

DA SILVA, FELIPE LENO; REALI COSTA, ANNA HELENA. A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems. JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, v. 64, p. 645-703, 2019. (18/00344-5, 16/21047-3, 15/16310-4)

FERREIRA, LEONARDO A.; BIANCHI, REINALDO A. C.; SANTOS, PAULO E.; LOPEZ DE MANTARAS, RAMON. Answer set programming for non-stationary Markov decision processes. APPLIED INTELLIGENCE, v. 47, n. 4, p. 993-1007, DEC 2017. (11/19280-8, 16/18792-9, 16/21047-3)

FERREIRA, VINICIUS N.; NETO, SYLVIO R. J.; ALMEIDA, AISLAN C.; BIANCHI, REINALDO A. C.; DONASCIMENTO, TP; COLOMBINI, EL; DEBRITO, AV; GARCIA, LTD; SA, STD; GONCALVES, LMG. A Visual Memory System for Humanoid Robots. 15TH LATIN AMERICAN ROBOTICS SYMPOSIUM 6TH BRAZILIAN ROBOTICS SYMPOSIUM 9TH WORKSHOP ON ROBOTICS IN EDUCATION (LARS/SBR/WRE 2018), v. N/A, p. 6-pg., 2018-01-01. (16/21047-3, 16/18792-9)

DA SILVA, FELIPE LENO; TAYLOR, MATTHEW E.; REALI COSTA, ANNA HELENA; LANG, J. Autonomously Reusing Knowledge in Multiagent Reinforcement Learning. PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, v. N/A, p. 7-pg., 2018-01-01. (16/21047-3, 15/16310-4, 18/00344-5)

DA SILVA, FELIPE LENO; GLATT, RUBEN; REALI COSTA, ANNA HELENA; ASSOC COMP MACHINERY. Simultaneously Learning and Advising in Multiagent Reinforcement Learning. 2022 25TH INTERNATIONAL CONFERENCE ON INFORMATION FUSION (FUSION 2022), v. N/A, p. 9-pg., 2017-01-01. (16/21047-3, 15/16310-4)

AMENDOLA, JOSE; TANNURI, EDUARDO A.; COZMAN, FABIO G.; REALI COSTA, ANNA H.; ASME. PORT CHANNEL NAVIGATION SUBJECTED TO ENVIRONMENTAL CONDITIONS USING REINFORCEMENT LEARNING. PROCEEDINGS OF THE ASME 38TH INTERNATIONAL CONFERENCE ON OCEAN, OFFSHORE AND ARCTIC ENGINEERING, 2019, VOL 7A, v. N/A, p. 10-pg., 2019-01-01. (16/21047-3, 16/18841-0)