| Full text | |
| Author(s): |
Total Authors: 4
|
| Affiliation: | [1] Univ Campinas UNICAMP, Inst Comp, Campinas, SP - Brazil
Total Affiliations: 1
|
| Document type: | Journal article |
| Source: | JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION; v. 60, p. 236-249, APR 2019. |
| Web of Science Citations: | 0 |
| Abstract | |
Among the challenges present in the design of retrieval systems, how to accurately assess their performance is perhaps one of the most important. Many applications such as rank aggregation or relevance feedback can be significantly improved with online effectiveness estimation of queries. Thus, developing methodologies that can estimate performance with minimal supervision and at query time is of utmost importance for improving the results of existing retrieval systems. In this work, we explore score-based, post-retrieval approaches for relevance prediction of search systems. We first introduce two statistical methods based on the Extreme Value Theory to estimate which of the top - k objects retrieved for a query are relevant. Our prediction approach uses this estimation as a method to infer the overall performance of a query. The two relevance prediction methods were evaluated in image datasets covering several modalities and scoring approaches. We conducted experiments comparing the ground-truth relevances of several ranks with predictions generated by our proposed approach, measuring their effectiveness by way of normalized accuracy and Matthews Correlation Coefficient. Furthermore, we also evaluate the precision deducted from our approaches with the system's expected performance. Those experiments show that the proposed approaches succeed in most relevance prediction scenarios of the top-ranked objects of a query, obtaining high accuracy. (C) 2019 Elsevier Inc. All rights reserved. (AU) | |
| FAPESP's process: | 14/50715-9 - Characterizing and predicting biomass production in sugarcane and eucalyptus plantations in Brazil |
| Grantee: | Rubens Augusto Camargo Lamparelli |
| Support Opportunities: | Research Grants - Research Partnership for Technological Innovation - PITE |
| FAPESP's process: | 16/50250-1 - The secret of playing football: Brazil versus the Netherlands |
| Grantee: | Sergio Augusto Cunha |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 17/20945-0 - Multi-user equipment approved in great 16/50250-1: local positioning system |
| Grantee: | Sergio Augusto Cunha |
| Support Opportunities: | Multi-user Equipment Program |
| FAPESP's process: | 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction? |
| Grantee: | Alexandre Xavier Falcão |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 17/12646-3 - Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events |
| Grantee: | Anderson de Rezende Rocha |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 13/50155-0 - Combining new technologies to monitor phenology from leaves to ecosystems |
| Grantee: | Leonor Patricia Cerdeira Morellato |
| Support Opportunities: | Research Program on Global Climate Change - University-Industry Cooperative Research (PITE) |
| FAPESP's process: | 15/24494-8 - Communications and processing of big data in cloud and fog computing |
| Grantee: | Nelson Luis Saldanha da Fonseca |
| Support Opportunities: | Research Projects - Thematic Grants |