A bag-of-graphs approach for cross-modal representations

• Grant number: 17/16453-5
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate
• Start date: January 01, 2018
• End date: June 30, 2018
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Ricardo da Silva Torres
• Grantee: Rafael de Oliveira Werneck
• Supervisor: Tabbone Salvatore Antoine
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Institution abroad: Université de Lorraine (UL), France
• Associated to the scholarship: 16/18429-1 - A bag-of-graphs approach for cross-modal representations, BP.DR

Abstract

In recent years, there has been a rapid growth of multimedia content, especially on the World Wide Web. A multimedia object, when associated with a subject, is encoded under different perspectives, for example, using text, images, or videos, usually referred to as their multi-modal components. By exploiting the different views provided by these different modalities, we could have a better understanding of the content of the multimedia object. The challenge is to find a way to define and correlate these modalities, and learn from this correlation in order to provide more efficient and effective services (such as retrieval and classification).In this research project, we propose to create a vectorial representation of a multimedia object through a bag-of-words approach, which represents the information present in multiple modalities. For this, we will model the relationship between the different modalities of a multimedia object using graphs. These graphs are the input of a bag-of-words framework model that generates the vectorial representation of the multimedia object. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

WERNECK, RAFAEL DE O.; DOURADO, ICARO C.; FADEL, SAMUEL G.; TABBONE, SALVATORE; TORRES, RICARDO DA S.; IEEE. GRAPH-BASED EARLY-FUSION FOR FLOOD DETECTION. 2018 25TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), v. N/A, p. 5-pg., 2018-01-01. (13/50169-1, 13/50155-0, 14/12236-1, 16/18429-1, 14/50715-9, 17/16453-5, 17/24005-2)

WERNECK, RAFAEL DE OLIVEIRA; RAVEAUX, ROMAIN; TABBONE, SALVATORE; TORRES, RICARDO DA SILVA. Learning cost function for graph classification with open-set methods. PATTERN RECOGNITION LETTERS, v. 128, p. 8-15, DEC 1 2019. (14/50715-9, 16/18429-1, 17/20945-0, 13/50155-0, 17/16453-5, 15/24494-8, 14/12236-1, 13/50169-1)