A Bag-of-Graphs Approach for Cross-Modal Representations

• Grant number: 16/18429-1
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: March 01, 2017
• End date: April 30, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Ricardo da Silva Torres
• Grantee: Rafael de Oliveira Werneck
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated scholarship(s): 17/16453-5 - A Bag-of-Graphs Approach for Cross-Modal Representations, BE.EP.DR

Abstract

In recent years, there has been a rapid growth of multimedia content, especially on the World Wide Web. A multimedia object, when associated with a subject, it is encoded under different perspectives, for example, using text, images, or videos, usually referred to as their multi-modal components. By exploiting the different information provided by these different modalities, we could have a better understanding of the content described by the multimedia object. The challenge of this problem is to find a way to define and correlate these modalities, and learn from this correlation in order to provide more efficient and effective services (such as retrieval and classification).In this thesis, we propose to create a vectorial representation of a multimedia object through a bag-of-words approach, which represents the information present in multiple modalities. For this, we will model the relationship between the different modalities of a multimedia object using graphs. These graphs are the input of a bag-of-words framework model that generates the vectorial representation of the multimedia object. We performed experiments with our approach for remote sensing image classification problem using a multi-feature scenario. We applied our approach to two datasets, in which we studied its parameter settings, and made statistical analysis in its results. We showed that our approach achieved effective results on remote sensing images.

Scientific publications

(7)

(The scientific publications listed on this page originate from the Web of Science or SciELO databases. Their authors have cited FAPESP grant or fellowship project numbers awarded to Principal Investigators or Fellowship Recipients, whether or not they are among the authors. This information is collected automatically and retrieved directly from those bibliometric databases.)

NOGUEIRA, KEILLER; FADEL, SAMUEL G.; DOURADO, ICARO C.; WERNECK, RAFAEL DE O.; MUNOZ, V, JAVIER A.; PENATTI, OTAVIO A. B.; CALUMBY, RODRIGO T.; LI, LIN TZY; DOS SANTOS, JEFERSSON A.; TORRES, RICARDO DA S.. Exploiting ConvNet Diversity for Flooding Identification. IEEE Geoscience and Remote Sensing Letters, v. 15, n. 9, p. 1446-1450, SEP 2018. (14/50715-9, 16/18429-1, 13/50155-0, 15/24494-8, 14/12236-1, 13/50169-1)

WERNECK, RAFAEL DE OLIVEIRA; RAVEAUX, ROMAIN; TABBONE, SALVATORE; TORRES, RICARDO DA SILVA. Learning cost function for graph classification with open-set methods. PATTERN RECOGNITION LETTERS, v. 128, p. 8-15, DEC 1 2019. (14/50715-9, 16/18429-1, 17/20945-0, 13/50155-0, 17/16453-5, 15/24494-8, 14/12236-1, 13/50169-1)

WERNECK, RAFAEL DE OLIVEIRA; DE ALMEIDA, WALDIR RODRIGUES; STEIN, BERNARDO VECCHIA; PAZINATO, DANIEL VATANABE; MENDES JUNIOR, PEDRO RIBEIRO; BIZETTO PENATTI, OTAVIO AUGUSTO; ROCHA, ANDERSON; TORRES, RICARDO DA SILVA. Kuaa: A unified framework for design, deployment, execution, and recommendation of machine learning experiments. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, v. 78, n. 1, p. 59-76, JAN 2018. (16/18429-1, 15/19222-9, 13/50155-0, 14/12236-1, 13/50169-1)

WERNECK, RAFAEL DE OLIVEIRA; DE ALMEIDA, WALDIR RODRIGUES; STEIN, BERNARDO VECCHIA; PAZINATO, DANIEL VATANABE; MENDES JUNIOR, PEDRO RIBEIRO; BIZETTO PENATTI, OTAVIO AUGUSTO; ROCHA, ANDERSON; TORRES, RICARDO DA SILVA. Kuaa: A unified framework for design, deployment, execution, and recommendation of machine learning experiments. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, v. 78, p. 18-pg., 2018-01-01. (13/50169-1, 13/50155-0, 14/12236-1, 16/18429-1, 15/19222-9)

WERNECK, RAFAEL DE O.; DOURADO, ICARO C.; FADEL, SAMUEL G.; TABBONE, SALVATORE; TORRES, RICARDO DA S.; IEEE. GRAPH-BASED EARLY-FUSION FOR FLOOD DETECTION. 2018 25TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), v. N/A, p. 5-pg., 2018-01-01. (13/50169-1, 13/50155-0, 14/12236-1, 16/18429-1, 14/50715-9, 17/16453-5, 17/24005-2)

WERNECK, RAFAEL DE O.; RAVEAUX, ROMAIN; TABBONE, SALVATORE; TORRES, RICARDO DA S.; BAI, X; HANCOCK, ER; HO, TK; WILSON, RC; BIGGIO, B; ROBLESKELLY, A. Learning Cost Functions for Graph Matching. STRUCTURAL, SYNTACTIC, AND STATISTICAL PATTERN RECOGNITION, S+SSPR 2018, v. 11004, p. 10-pg., 2018-01-01. (14/50715-9, 17/20945-0, 14/12236-1, 16/18429-1, 16/50250-1, 13/50169-1, 13/50155-0, 15/24494-8)

SILVA, FERNANDA B.; WERNECK, RAFAEL DE O.; GOLDENSTEIN, SIOME; TABBONE, SALVATORE; TORRES, RICARDO DA S.. Graph-based bag-of-words for classification. PATTERN RECOGNITION, v. 74, p. 266-285, FEB 2018. (16/18429-1, 12/50468-6, 13/11378-4, 13/50155-0, 14/12236-1, 12/16172-2, 13/50169-1)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

WERNECK, Rafael de Oliveira. Aprendizado de representações e correspondências baseadas em grafos para tarefas de classificação. 2019. Doctoral Thesis - Universidade Estadual de Campinas (UNICAMP). Instituto de Computação Campinas, SP.