Advanced search
Start date
(Reference retrieved automatically from Web of Science through information on FAPESP grant and its corresponding number as mentioned in the publication by the authors.)

Relevance prediction in similarity-search systems using extreme value theory

Full text
Oliveira, Alberto [1] ; Oakley, Eric [1] ; Torres, Ricardo da Silva [1] ; Rocha, Anderson [1]
Total Authors: 4
[1] Univ Campinas UNICAMP, Inst Comp, Campinas, SP - Brazil
Total Affiliations: 1
Document type: Journal article
Web of Science Citations: 0

Among the challenges present in the design of retrieval systems, how to accurately assess their performance is perhaps one of the most important. Many applications such as rank aggregation or relevance feedback can be significantly improved with online effectiveness estimation of queries. Thus, developing methodologies that can estimate performance with minimal supervision and at query time is of utmost importance for improving the results of existing retrieval systems. In this work, we explore score-based, post-retrieval approaches for relevance prediction of search systems. We first introduce two statistical methods based on the Extreme Value Theory to estimate which of the top - k objects retrieved for a query are relevant. Our prediction approach uses this estimation as a method to infer the overall performance of a query. The two relevance prediction methods were evaluated in image datasets covering several modalities and scoring approaches. We conducted experiments comparing the ground-truth relevances of several ranks with predictions generated by our proposed approach, measuring their effectiveness by way of normalized accuracy and Matthews Correlation Coefficient. Furthermore, we also evaluate the precision deducted from our approaches with the system's expected performance. Those experiments show that the proposed approaches succeed in most relevance prediction scenarios of the top-ranked objects of a query, obtaining high accuracy. (C) 2019 Elsevier Inc. All rights reserved. (AU)

FAPESP's process: 14/50715-9 - Characterizing and predicting biomass production in sugarcane and eucalyptus plantations in Brazil
Grantee:Rubens Augusto Camargo Lamparelli
Support type: Research Grants - Research Partnership for Technological Innovation - PITE
FAPESP's process: 16/50250-1 - The secret of playing football: Brazil versus the Netherlands
Grantee:Sergio Augusto Cunha
Support type: Research Projects - Thematic Grants
FAPESP's process: 17/20945-0 - Multi-user equipment approved in great 16/50250-1: local positioning system
Grantee:Sergio Augusto Cunha
Support type: Multi-user Equipment Program
FAPESP's process: 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction?
Grantee:Alexandre Xavier Falcão
Support type: Research Projects - Thematic Grants
FAPESP's process: 17/12646-3 - Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events
Grantee:Anderson de Rezende Rocha
Support type: Research Projects - Thematic Grants
FAPESP's process: 13/50155-0 - Combining new technologies to monitor phenology from leaves to ecosystems
Grantee:Leonor Patricia Cerdeira Morellato
Support type: Research Program on Global Climate Change - University-Industry Cooperative Research (PITE)
FAPESP's process: 15/24494-8 - Communications and processing of big data in cloud and fog computing
Grantee:Nelson Luis Saldanha da Fonseca
Support type: Research Projects - Thematic Grants