Sensor Fusion of Automotive Sensors for Context-based Detection of Critical Regions on Drivable Area

• Grant number: 24/23074-4
• Support Opportunities: Regular Research Grants
• Start date: June 01, 2025
• End date: May 31, 2027
• Field of knowledge: Engineering - Electrical Engineering - Electrical, Magnetic and Electronic Measurements, Instrumentation
• Principal Investigator: Bruno Augusto Angélico
• Grantee: Bruno Augusto Angélico
• Principal researcher abroad: Thomas Brandmeier
• Institution abroad: Technical University of Ingolstadt., Germany
• Host Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated researchers: Fernando Santos Osório ; Joao Francisco Justo Filho ; Marcio Lobo Netto
• Associated research grant: 23/04628-6 - Safe control of automotive systems, AP.R

Abstract

While the automotive sector has played a crucial role in society, it has also raised major concerns in terms of fatalities and injuries. Thousands die annually in accidents, with around 90% attributed to human error. To address this concern, autonomous navigation emerged as a key solution, driven by advancements in sensors (such as Camera, LiDAR, and Radar), computational power, and artificial intelligence. Although fully autonomous fleets remain a distant goal, these technologies can enhance safety systems by preemptively activating safety functions, such as airbags, steering, and braking, improving the standard active and passive safety functions, and minimizing injuries and fatalities. To achieve this, precise sensing of the road environment is mandatory, yet real-world conditions, like diverse traffic scenarios, unexpected obstacles, and adverse weather conditions, can pose significant challenges. Such challenges can affect the performance and robustness of the perception algorithms, especially in state-of-the-art object-oriented approaches, where changes in object representation due to sensor degradation, change of perspective, under-representation in the training data, or even a not included class, could lead to critical false negatives. In order to address these challenges, this proposal aims to research the development of artificial intelligence algorithms based on the early fusion of different sensors, which despite of high potential to explore cross-correlation between different information sources, are still not deeply explored in the literature. To perform context-oriented detection or segmentation, learning to recognize regions in the vehicle's environment that may represent a potential collision risk. Therefore, to achieve this goal, through the joint research between USP and the THI CARISSMA Institute, both theoretical and practical aspects will be deepened, through the exchange of students, organization of workshops, and research collaboration aiming to evolve the definition and configuration of test vehicles, evaluation of automotive sensors under a controlled environment and weather conditions, generation of a multi-modal dataset of key scenarios, research and development of artificial intelligence algorithms (sensors calibration, machine learning and sensor fusion). (AU)