Scholarship 22/12204-9 - Processamento de imagens, Processamento de linguagem natural - BV FAPESP
Advanced search
Start date
Betweenand

Development of methods for image captioning: a framework based on computer vision and natural language processing

Grant number: 22/12204-9
Support Opportunities:Scholarships in Brazil - Post-Doctoral
Start date until: December 01, 2022
End date until: March 31, 2023
Field of knowledge:Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
Principal Investigator:Roberto Marcondes Cesar Junior
Grantee:Rafael Jeferson Pezzuto Damaceno
Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
Associated research grant:15/22308-2 - Intermediate representations in Computational Science for knowledge discovery, AP.TEM

Abstract

Scene description is a process that aims to associate one or more textual sentences to an image. Concerning urban infrastructure, this is a technique that allows the description of urban scenarios, such as the characterization of sidewalks in terms of size, the identification of objects and the positional relationship they have with each other, among others. In recent years important advances have been achieved in this task through the use of deep neural network techniques. These advances are facilitated by the greater availability of GPUs and large data sets. Furthermore, advances achieved by neural networks have been mainly obtained in individual modalities such as vision, language or sound. In many cases, real-world problems have components that are embedded in more than one modality - as can be the scenarios of Urban Informatics. In this sense, this project aims at developing computational methods to improve the processes of description of urban scenes. As object of study, we intend to explore remote-sensing and street-level images. The project will involve the use of computer vision techniques associated with natural language processing. The main idea is to use the textual language framework to improve the generated descriptions.

News published in Agência FAPESP Newsletter about the scholarship:
More itemsLess items
Articles published in other media outlets ( ):
More itemsLess items
VEICULO: TITULO (DATA)
VEICULO: TITULO (DATA)

Please report errors in scientific publications list using this form.