Convolutional neural networks in environment-aware sensor fusion

• Grant number: 18/02122-0
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: March 31, 2018
• End date: July 30, 2018
• Field of knowledge: Interdisciplinary Subjects
• Principal Investigator: José Eduardo Cogo Castanho
• Grantee: Caio Fischer Silva
• Supervisor: Paulo Vinicius Koerich Borges
• Host Institution: Faculdade de Engenharia (FE). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Institution abroad: Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia
• Associated to the scholarship: 17/09228-5 - Applying convolutional neural networks in mobile robots navigation, BP.IC

Abstract

A reliable sensor fusion is crucial to the operation of a safe and efficient autonomous navigation system. Given that all sensors have their advantages and drawbacks, a single sensor is often not sufficient to reliably represents the environment and the use of multiple sensors has become a common practice. Most systems rely on a static sensor fusion strategy, that tends to perform poorly in heterogeneous environments. The use of machine learning algorithms may be useful to automatic change the sensor fusion strategy according to the environment. This project aims to evaluate the end-to-end approach in the training of an artificial neural network to dynamically adapts the sensor fusion strategy to the environment. In the end-to-end paradigm, the image from an onboard camera is directly associated with the sensors confidence, without any handcrafted feature extractor. The proposed system will be tested in the experimental setup developed by the Robotics Perception Team in the Autonomous System Laboratory at the Commonwealth Scientific and Industrial Research Organisation (CSIRO). This proposal describes the procedures to collect and label the training data, the experiments to be conducted and the performance measure used to quantify the performance of the sensor fusion systems in the object detection task. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

SILVA, CAIO FISCHER; BORGES, PAULO V. K.; CASTANHO, JOSE E. C.; GUSIKHIN, O; MADANI, K; ZAYTOON, J. Environment-aware Sensor Fusion using Deep Learning. ICINCO: PROCEEDINGS OF THE 16TH INTERNATIONAL CONFERENCE ON INFORMATICS IN CONTROL, AUTOMATION AND ROBOTICS, VOL 2, v. N/A, p. 9-pg., 2019-01-01. (18/02122-0)