Multimodal retrieval with relevance feedback based on genetic programming

Author(s)

Calumby, Rodrigo Tripodi ^{[1, 2]} ; Torres, Ricardo da Silva ^[2] ; Goncalves, Marcos Andre ^[3]

Affiliation

^[1] Univ Feira de Santana, Dept Exact Sci, Feira De Santana - Brazil

^[2] Univ Estadual Campinas, Inst Comp, RECOD Lab, Campinas, SP - Brazil

^[3] Univ Fed Minas Gerais, Dept Comp Sci, Belo Horizonte, MG - Brazil

• Document type: Journal article
• Source: MULTIMEDIA TOOLS AND APPLICATIONS; v. 69, n. 3, p. 991-1019, APR 2014.
• Web of Science Citations: 10

Abstract

This paper presents a framework for multimodal retrieval with relevance feedback based on genetic programming. In this supervised learning-to-rank framework, genetic programming is used for the discovery of effective combination functions of (multimodal) similarity measures using the information obtained throughout the user relevance feedback iterations. With these new functions, several similarity measures, including those extracted from different modalities (e.g., text, and content), are combined into one single measure that properly encodes the user preferences. This framework was instantiated for multimodal image retrieval using visual and textual features and was validated using two image collections, one from the Washington University and another from the ImageCLEF Photographic Retrieval Task. For this image retrieval instance several multimodal relevance feedback techniques were implemented and evaluated. The proposed approach has produced statistically significant better results for multimodal retrieval over single modality approaches and superior effectiveness when compared to the best submissions of the ImageCLEF Photographic Retrieval Task 2008. (AU)

• FAPESP's process: 07/52015-0 - Approximation methods for visual computing
• Grantee: Jorge Stolfi
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 09/18438-7 - Large-scale classification and retrieval for complex data
• Grantee: Ricardo da Silva Torres
• Support Opportunities: Regular Research Grants