Advanced search
Start date
(Reference retrieved automatically from Web of Science through information on FAPESP grant and its corresponding number as mentioned in the publication by the authors.)

Stochastic Abstract Policies: Generalizing Knowledge to Improve Reinforcement Learning

Full text
Koga, Marcelo L. [1] ; Freire, Valdinei [2] ; Costa, Anna H. R. [1]
Total Authors: 3
[1] Univ Sao Paulo, Escola Politecn, BR-05508970 Sao Paulo - Brazil
[2] Univ Sao Paulo, Escola Artes Ciencias & Humanidades, BR-05508970 Sao Paulo - Brazil
Total Affiliations: 2
Document type: Journal article
Source: IEEE TRANSACTIONS ON CYBERNETICS; v. 45, n. 1, p. 77-88, JAN 2015.
Web of Science Citations: 15

Reinforcement learning (RL) enables an agent to learn behavior by acquiring experience through trial-and-error interactions with a dynamic environment. However, knowledge is usually built from scratch and learning to behave may take a long time. Here, we improve the learning performance by leveraging prior knowledge; that is, the learner shows proper behavior from the beginning of a target task, using the knowledge from a set of known, previously solved, source tasks. In this paper, we argue that building stochastic abstract policies that generalize over past experiences is an effective way to provide such improvement and this generalization outperforms the current practice of using a library of policies. We achieve that contributing with a new algorithm, AbsProb-PI-multiple and a framework for transferring knowledge represented as a stochastic abstract policy in new RL tasks. Stochastic abstract policies offer an effective way to encode knowledge because the abstraction they provide not only generalizes solutions but also facilitates extracting the similarities among tasks. We perform experiments in a robotic navigation environment and analyze the agent's behavior throughout the learning process and also assess the transfer ratio for different amounts of source tasks. We compare our method with the transfer of a library of policies, and experiments show that the use of a generalized policy produces better results by more effectively guiding the agent when learning a target task. (AU)

FAPESP's process: 12/02190-9 - Knowledge Transfer among Tasks in Reinforcement Learning
Grantee:Marcelo Li Koga
Support Opportunities: Scholarships in Brazil - Master
FAPESP's process: 11/19280-8 - CogBot: integrating perceptual information and semantic knowledge in cognitive robotics
Grantee:Anna Helena Reali Costa
Support Opportunities: Regular Research Grants