Spatial instance segmentation in monocular images through convolutional neural networks

• Grant number: 19/03366-2
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: June 01, 2019
• End date: January 31, 2022
• Field of knowledge: Engineering - Electrical Engineering
• Principal Investigator: Denis Fernando Wolf
• Grantee: Angelica Tiemi Mizuno Nakamura
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment, AP.TEM

Abstract

The driver assistance systems for partial or full automation of the vehicle are systems that help in decision making and have an important role in traffic safety and efficiency. Such systems require robust perception system that should be able to handle occlusions and complex urban scenarios in order to allow the vehicle understanding about the environment where it is navigating. Therefore, this doctoral project proposes a convolutional neural network based method for spatial instance segmentation, such that the instance segmentation and depth information are estimated simultaneously from a monocular image. The extraction of features relevant to tasks learning will be performed by a convolutional neural network, which will be designed as an encoder-decoder architecture for pixel-level classification and multi-task learning. This architecture will consist of a unique encoder and split decoders. Aiming better performance of the network, two research objects will be studied, the set of loss functions that provides the best convergence during training and the architecture design that favors parameters sharing between tasks' decoders during the learning process. For validation, the performance of the proposed method will be compared in existing benchmarks. Furthermore, real experiments will be executed within the CaRINA II platform, an ongoing autonomous vehicle project held by the Mobile Robotics Laboratory (LRM) with Intelligent Systems Laboratory (LASI) of the University of São Paulo at São Carlos. This project is part of the thematic project supported by FAPESP, process n. 2014/50851-0 "National Institute of Science and Technology for cooperative autonomous systems applied to security and environment". (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

NAKAMURA, ANGELICA TIEMI MIZUNO; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO. An effective combination of loss gradients for multi-task learning applied on instance segmentation and depth estimation. ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, v. 100, APR 2021. (19/03366-2)

MIZUNO NAKAMURA, ANGELICA TIEMI; GRASSI JR, VALDIR; WOLF, DENIS FERNANDO. Leveraging convergence behavior to balance conflicting tasks in multi-task learning. Neurocomputing, v. 511, p. 11-pg., 2022-09-13. (14/50851-0, 19/03366-2)

HORITA, LUIZ R. T.; NAKAMURA, ANGELICA T. M.; WOLF, DENIS F.; GRASSI JUNIOR, VALDIR; IEEE. Improving multi-goal and target-driven reinforcement learning with supervised auxiliary task. 2021 20TH INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS (ICAR), v. N/A, p. 6-pg., 2021-01-01. (19/03366-2, 14/50851-0)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

NAKAMURA, Angelica Tiemi Mizuno. Leveraging convergence behavior to balance conflicting tasks in multi-task learning. 2022. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB) São Carlos.