Advanced search
Start date
(Reference retrieved automatically from Web of Science through information on FAPESP grant and its corresponding number as mentioned in the publication by the authors.)

Neural relational inference for disaster multimedia retrieval

Full text
Fadel, Samuel G. [1] ; Torres, Ricardo da S. [2]
Total Authors: 2
[1] Univ Estadual Campinas, Inst Comp, Ave Albert Einstein 1251, BR-13083852 Campinas - Brazil
[2] NTNU Norwegian Univ Sci & Technol, Fac Informat Technol & Elect Engn, Dept ICT & Nat Sci, Alesund - Norway
Total Affiliations: 2
Document type: Journal article
Source: MULTIMEDIA TOOLS AND APPLICATIONS; v. 79, n. 35-36 JUL 2020.
Web of Science Citations: 0

Events around the world are increasingly documented on social media, especially by the people experiencing them, as these platforms become more popular over time. As a consequence, social media turns into a valuable source of data for understanding those events. Due to their destructive potential, natural disasters are among events of particular interest to response operations and environmental monitoring agencies. However, this amount of information also makes it challenging to identify relevant content pertaining to those events. In this paper, we use a relational neural network model for identifying this type of content. The model is particularly suitable for unstructured text, that is, text with no particular arrangement of words, such as tags, which is commonplace in social media data. In addition, our method can be combined with a CNN for handling multimodal data where text and visual data are available. We perform experiments in three different scenarios, where different modalities are evaluated: visual, textual, and both. Our method achieves competitive performance in both modalities by themselves, while significantly outperforms the baseline on the multimodal scenario. We also demonstrate the behavior of the proposed method in different applications by performing additional experiments in the CUB-200-2011 multimodal dataset. (AU)

FAPESP's process: 14/50715-9 - Characterizing and predicting biomass production in sugarcane and eucalyptus plantations in Brazil
Grantee:Rubens Augusto Camargo Lamparelli
Support type: Research Grants - Research Partnership for Technological Innovation - PITE
FAPESP's process: 16/50250-1 - The secret of playing football: Brazil versus the Netherlands
Grantee:Sergio Augusto Cunha
Support type: Research Projects - Thematic Grants
FAPESP's process: 17/24005-2 - Temporal relational reasoning with neural networks
Grantee:Samuel Gomes Fadel
Support type: Scholarships in Brazil - Doctorate
FAPESP's process: 17/20945-0 - Multi-user equipment approved in great 16/50250-1: local positioning system
Grantee:Sergio Augusto Cunha
Support type: Multi-user Equipment Program
FAPESP's process: 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction?
Grantee:Alexandre Xavier Falcão
Support type: Research Projects - Thematic Grants
FAPESP's process: 13/50169-1 - Towards an understanding of tipping points within tropical South American biomes
Grantee:Ricardo da Silva Torres
Support type: Research Grants - Research Partnership for Technological Innovation - PITE
FAPESP's process: 13/50155-0 - Combining new technologies to monitor phenology from leaves to ecosystems
Grantee:Leonor Patricia Cerdeira Morellato
Support type: Research Program on Global Climate Change - University-Industry Cooperative Research (PITE)
FAPESP's process: 15/24494-8 - Communications and processing of big data in cloud and fog computing
Grantee:Nelson Luis Saldanha da Fonseca
Support type: Research Projects - Thematic Grants