Continuous Markov decision process with imprecise probabilities: algorithms and applications

• Grant number: 12/20520-6
• Support Opportunities: Scholarships abroad - Research
• Start date: January 12, 2013
• End date: February 11, 2013
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Karina Valdivia Delgado
• Grantee: Karina Valdivia Delgado
• Host Investigator: Scott Sanner
• Host Institution: Escola de Artes, Ciências e Humanidades (EACH). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Institution abroad: National ICT Australia (NICTA), Australia

Abstract

The main subject of this project is related to the area of planning under uncertainty in Artificial Intelligence (AI).Planning problems involving actions with probabilistic effects can be modeled by Markov Decision Processes (MDPs), efficient solutions capable of solving this type of problems with a reasonable number of states were proposed. However, known extensions of an MDP are better suited to represent practical problems of greatest interest for real applications, including: (1) an MDP described by state variables, called factored MDP that allows to represent the state space in a more compact form and (2) an MDP whose probabilities are not completely known, called MDP-IP, which allows to describe problems more realistically. In this project, which will be held in collaboration with Scott Sanner from the main research center of Australia NICTA, we are interested in extending previous results defining new algorithms for MDP-IPs with continuous variables. Thus, it will be possible to develop practical applications, such as urban traffic planning and automation of industrial assembly lines. (AU)