Investigation of Visual Question Answering Approaches for Natural Disaster Management

• Grant number: 23/12736-3
• Support Opportunities: Scholarships in Brazil - Master
• Start date: May 01, 2024
• End date: February 28, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Daniel Carlos Guimarães Pedronette
• Grantee: Marina Chagas Bulach Gapski
• Host Institution: Instituto de Geociências e Ciências Exatas (IGCE). Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Rio Claro , SP, Brazil
• Associated research grant: 18/15597-6 - Aplication and investigation of unsupervised learning methods in retrieval and classification tasks, AP.JP2

Abstract

Climate change is making natural disasters more frequent. Floods, hurri-canes, and earthquakes are becoming more intense and frequent in several coun-tries. In this scenario, it is essential to have rescue teams ready to save potentialvictims of accidents, building collapses, and landslides. For these teams, it isvital to be ready for what might be found in destroyed areas. Robots like dronesare extremely useful because they are able to send images of areas for recogni-tion without risking human lives. We intend to create a model, based on weaksupervised machine learning models, that can answer some simple and directquestions about destroyed areas. For that purpose, a dataset composed of im-ages of flooded or destroyed areas will be used. The model will be trained torecognize which areas are destroyed and which are not, using a knowledge graph,and then a simple language model will be constructed to answer some directquestions for supporting disaster management tasks.