Multimodal retrieval with relevance feedback based on genetic programming

Autor(es)

Calumby, Rodrigo Tripodi ^{[1, 2]} ; Torres, Ricardo da Silva ^[2] ; Goncalves, Marcos Andre ^[3]

Afiliação do(s) autor(es)

^[1] Univ Feira de Santana, Dept Exact Sci, Feira De Santana - Brazil

^[2] Univ Estadual Campinas, Inst Comp, RECOD Lab, Campinas, SP - Brazil

^[3] Univ Fed Minas Gerais, Dept Comp Sci, Belo Horizonte, MG - Brazil

• Tipo de documento: Artigo Científico
• Fonte: MULTIMEDIA TOOLS AND APPLICATIONS; v. 69, n. 3, p. 991-1019, APR 2014.
• Citações Web of Science: 10

Resumo

This paper presents a framework for multimodal retrieval with relevance feedback based on genetic programming. In this supervised learning-to-rank framework, genetic programming is used for the discovery of effective combination functions of (multimodal) similarity measures using the information obtained throughout the user relevance feedback iterations. With these new functions, several similarity measures, including those extracted from different modalities (e.g., text, and content), are combined into one single measure that properly encodes the user preferences. This framework was instantiated for multimodal image retrieval using visual and textual features and was validated using two image collections, one from the Washington University and another from the ImageCLEF Photographic Retrieval Task. For this image retrieval instance several multimodal relevance feedback techniques were implemented and evaluated. The proposed approach has produced statistically significant better results for multimodal retrieval over single modality approaches and superior effectiveness when compared to the best submissions of the ImageCLEF Photographic Retrieval Task 2008. (AU)

• Processo FAPESP: 07/52015-0 - Métodos de aproximação para computação visual
• Beneficiário: Jorge Stolfi
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 09/18438-7 - Classificação e busca em grande escala para dados complexos
• Beneficiário: Ricardo da Silva Torres
• Modalidade de apoio: Auxílio à Pesquisa - Regular