LiDAR-guided and Vision fusion: the use of WayFAST neural network for enhancing the bird's-eye-view representation

• Grant number: 24/00519-0
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: April 21, 2024
• End date: August 16, 2024
• Field of knowledge: Engineering - Mechanical Engineering
• Principal Investigator: Marcelo Becker
• Grantee: Gianluca Capezzuto Sardinha
• Supervisor: Girish Chowdhary
• Host Institution: Escola de Engenharia de São Carlos (EESC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Institution abroad: University of Illinois at Urbana-Champaign, United States
• Associated to the scholarship: 22/08433-2 - Application of ablative methods in neural networks for robotic navigation in rugged fields, BP.IC

Abstract

Robots are essential in performing demanding tasks in outdoor settings like agriculture, construction, and remote exploration. However, their ability to navigate challenging terrains, known as traversability, poses a significant challenge. Traditional methods often produce false positives, particularly in scenarios involving tall grass, muddy areas, and snowy surfaces. Addressing these challenges necessitates effective information fusion. This research project aims to improve the bird's-eye-view representation of the WayFAST neural network through the fusion of LiDAR-guided and vision data. WayFAST, a self-supervised traversability prediction system, is designed for the autonomous navigation of wheeled robots in unstructured outdoor environments. The project emphasizes the comprehensive fusion of data from camera and LiDAR sensors, utilizing a bird's-eye perspective. The implementation of the proposed pipeline for LiDAR-camera fusion will generate a traversability map. This map is a vital tool for the robot's control system, enabling safe navigation by identifying suitable local routes and avoiding potential hazards in its environment.