Markov decision processes specified by probabilistic logic programming: representation and solution

• Grant number: 16/22900-1
• Support Opportunities: Scholarships in Brazil - Doctorate (Direct)
• Start date: March 01, 2017
• End date: February 29, 2020
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Leliane Nunes de Barros
• Grantee: Thiago Pereira Bueno
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated research grant: 15/01587-0 - Storage, modeling and analysis of dynamical systems for e-Science applications, AP.ESCIENCE.TEM
• Associated scholarship(s): 17/21763-3 - Efficient solutions for hybrid Markov decision processes specified through probabilistic logic programming: deep neural nets vs symbolic reasoning, BE.EP.DD

Abstract

In the field of probabilistic planning, in order to efficiently solve sequential decision-making problems modeled by Markov Decision Processes (MDPs), it is crucial to handle stochastic knowledge defined over complex relations that may involve features of transitivity, symmetry, recursion, determinism and context-specific independence. However, traditional representational formalisms used by the automated planning community, e.g. PPDDL and RDDL, do not allow a compact and explicit description of these aspects of representation. On the other hand, recently, a new paradigm of probabilistic modeling has gained traction in the area of Knowledge Representation and Reasoning under uncertainty. By adding resources to manipulate probability distributions into general-purpose programming languages, the so-called probabilistic programming languages have offered a new framework of abstraction that naturally allows the specification of complex probabilistic models involving symmetry relations, transitivity and recursion that can induce cyclic dependencies between variables --- which is not allowed by the semantics of traditional planning problem specification languages. So, in this research project we aim to investigate the advantages of probabilistic programming languages for representation and solution of sequential decision-making and probabilistic planning problems commonly studied in the area of artificial intelligence. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

LOVATTO, ANGELO G.; BUENO, THIAGO P.; MAUA, DENIS D.; DE BARROS, LELIANE N.. Decision-Aware Model Learning for Actor-Critic Methods: When Theory Does Not Meet Practice. NEURIPS WORKSHOPS, 2020, v. 137, p. 11-pg., 2020-01-01. (16/22900-1)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

BUENO, Thiago Pereira. Planning in stochastic computation graphs: solving stochastic nonlinear problems with backpropagation. 2021. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Matemática e Estatística (IME/SBI) São Paulo.