Advanced search
Start date

Spatial instance segmentation in monocular images through convolutional neural networks

Grant number: 19/03366-2
Support Opportunities:Scholarships in Brazil - Doctorate
Effective date (Start): June 01, 2019
Effective date (End): January 31, 2022
Field of knowledge:Engineering - Electrical Engineering
Principal Investigator:Denis Fernando Wolf
Grantee:Angelica Tiemi Mizuno Nakamura
Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
Associated research grant:14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment, AP.TEM


The driver assistance systems for partial or full automation of the vehicle are systems that help in decision making and have an important role in traffic safety and efficiency. Such systems require robust perception system that should be able to handle occlusions and complex urban scenarios in order to allow the vehicle understanding about the environment where it is navigating. Therefore, this doctoral project proposes a convolutional neural network based method for spatial instance segmentation, such that the instance segmentation and depth information are estimated simultaneously from a monocular image. The extraction of features relevant to tasks learning will be performed by a convolutional neural network, which will be designed as an encoder-decoder architecture for pixel-level classification and multi-task learning. This architecture will consist of a unique encoder and split decoders. Aiming better performance of the network, two research objects will be studied, the set of loss functions that provides the best convergence during training and the architecture design that favors parameters sharing between tasks' decoders during the learning process. For validation, the performance of the proposed method will be compared in existing benchmarks. Furthermore, real experiments will be executed within the CaRINA II platform, an ongoing autonomous vehicle project held by the Mobile Robotics Laboratory (LRM) with Intelligent Systems Laboratory (LASI) of the University of São Paulo at São Carlos. This project is part of the thematic project supported by FAPESP, process n. 2014/50851-0 "National Institute of Science and Technology for cooperative autonomous systems applied to security and environment". (AU)

News published in Agência FAPESP Newsletter about the scholarship:
Articles published in other media outlets (0 total):
More itemsLess items

Scientific publications
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
MIZUNO NAKAMURA, ANGELICA TIEMI; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO. Leveraging convergence behavior to balance conflicting tasks in multi-task learning. Neurocomputing, v. 511, p. 11-pg., . (14/50851-0, 19/03366-2)
HORITA, LUIZ R. T.; NAKAMURA, ANGELICA T. M.; WOLF, DENIS F.; GRASSI JUNIOR, VALDIR; IEEE. Improving multi-goal and target-driven reinforcement learning with supervised auxiliary task. 2021 20TH INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS (ICAR), v. N/A, p. 6-pg., . (19/03366-2, 14/50851-0)
NAKAMURA, ANGELICA TIEMI MIZUNO; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO. An effective combination of loss gradients for multi-task learning applied on instance segmentation and depth estimation. ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, v. 100, . (19/03366-2)
Academic Publications
(References retrieved automatically from State of São Paulo Research Institutions)
NAKAMURA, Angelica Tiemi Mizuno. Leveraging convergence behavior to balance conflicting tasks in multi-task learning. 2022. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB) São Carlos.

Please report errors in scientific publications list using this form.