Self-Supervised Image Re-Ranking based on Hypergraphs and Graph Convolutional Networks

Author(s)

Lopes, Leonardo Tadeu ; Valem, Lucas Pascotti ; Pedronette, Daniel Carlos Guimardes

• Document type: Journal article
• Source: 2024 37TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES, SIBGRAPI 2024; v. N/A, p. 6-pg., 2024-01-01.

Abstract

Image retrieval approaches typically involve two fundamental stages: visual content representation and similarity measurement. Traditional methods rely on pairwise dissimilarity metrics, such as Euclidean distance, which overlook the global structure of datasets. Aiming to address this limitation, various unsupervised post-processing approaches have been developed to redefine similarity measures. Diffusion processes and rank-based methods compute a more effective similarity by considering the relationships among images and the overall dataset structure. However, neither approach is capable of defining novel image representations. This paper aims to overcome this limitation by proposing a novel self-supervised image re-ranking method. The proposed method exploits a hypergraph model, clustering strategies, and Graph Convolutional Networks (GCNs). Initially, an unsupervised rank-based manifold learning method computes global similarities to define small and reliable clusters, which are used as soft labels for training a semi-supervised GCN model. This GCN undergoes a two-stage training process: an initial classification-focused stage followed by a retrieval-focused stage. The final GCN embeddings are employed for retrieval tasks using the cosine similarity. An experimental evaluation conducted on four public datasets with three different visual features indicates that the proposed approach outperforms traditional and recent rank-based methods. (AU)

• FAPESP's process: 18/15597-6 - Aplication and investigation of unsupervised learning methods in retrieval and classification tasks
• Grantee: Daniel Carlos Guimarães Pedronette
• Support Opportunities: Research Grants - Young Investigators Grants - Phase 2