Decision making and trajectory planning for intelligent vehicles using partially observable Markov decision processes and inverse reinforcement learning

• Grant number: 18/19732-5
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: January 01, 2019
• End date: January 10, 2022
• Field of knowledge: Engineering - Electrical Engineering
• Principal Investigator: Denis Fernando Wolf
• Grantee: Júnior Anderson Rodrigues da Silva
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment, AP.TEM

Abstract

Autonomous mobile robots that share the same environment with people require motion planning approaches that resemble, in some aspects, the human behavior. Such behavior facilitates other agents to predict the robot's movement’s vice-verse, resulting in a smoother joint navigation. This is presumably the scenario that autonomous vehicles will face: they will share the traffic roads with other vehicles (autonomous or not), integrating cooperatively to them. This doctorate project proposes the development and implementation of a decision making framework for autonomous vehicles in an urban scenario. The actions of other vehicles will be inferred through Inverse Reinforcement Learning, whereby the behavior of agents is learned through the imitation of an expert. Decision making will take place through Partially Observable Markov Decision Processes, since the actions of other vehicles can not be observed directly. Finally, a trajectory will be planned and executed considering the expert driving style. The test platform CaRINA II, a probe vehicle under development by the Laboratory of Mobile Robotic at USP São Carlos, will be used in results validation. The test environment will be the Campus 2 at USP - São Carlos, since it has attributes of an urban scenario. This project is part of the Thematic Project supported by FAPESP, process n. 2014/50851-0 "National Institute of Science and Technology for Cooperative Autonomous Systems Applied to Security and Environment". (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

RODRIGUES DA SILVA, JUNIOR ANDERSON; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO; IEEE. Decision Making for Autonomous Vehicles at Signalized Intersection under Uncertain Traffic Signal Phase and Timing Information. 2021 20TH INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS (ICAR), v. N/A, p. 6-pg., 2021-01-01. (18/19732-5, 14/50851-0)

RODRIGUES DA SILVA, JUNIOR ANDERSON; GOMES, IAGO PACHECO; WOLF, DENIS FERNANDO; GRASSI, VALDIR. parse Road Network Model for Autonomous Navigation Using Clothoid. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, v. 23, n. 2, p. 885-898, FEB 2022. (18/19732-5, 14/50851-0)

SILVA, JUNIOR A. R.; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO; IEEE. Continuous Deep Maximum Entropy Inverse Reinforcement Learning using online POMDP. 2019 19TH INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS (ICAR), v. N/A, p. 6-pg., 2019-01-01. (18/19732-5, 14/50851-0)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

SILVA, Júnior Anderson Rodrigues da. Autonomous driving: learning to make decisions in uncertain environments. 2023. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB) São Carlos.