Generative networks and feature learning for cross domain visual search

• Grant number: 17/22366-8
• Support Opportunities: Scholarships in Brazil - Doctorate (Direct)
• Start date: March 01, 2018
• End date: February 28, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Moacir Antonelli Ponti
• Grantee: Leo Sampaio Ferraz Ribeiro
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated scholarship(s): 19/02808-1 - Finding correspondences between cross-domain representations using unsupervised feature matching, BE.EP.DD

Abstract

Feature learning methods have reached state of the art on many areas of study. Despite excellent results achieved on benchmark datasets, there is still little understanding of its inner workings, and applications to be explored, particularly when considering architectures that go beyond traditional convolutional neural networks. In this project, we propose the use of feature learning for applications that encompass domain mapping, specifically applications that regard image retrieval with queries from different domains and applications with a limited amount of labelled data. These challenges can be tackled using deep learning, with the development of novel architectures based on multi-stream networks, convolutional autoencoders and generative models. The expected results include the development of models that can successfully map between different domains and that are also capable of generalizing to unseen data. (AU)

Scientific publications

(6)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

PONTI, MOACIR A.; DOS SANTOS, FERNANDO P.; RIBEIRO, LEO S. F.; CAVALLARI, GABRIEL B.; IEEE COMP SOC. Training Deep Networks from Zero to Hero: avoiding pitfalls and going beyond. 2021 34TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI 2021), v. N/A, p. 8-pg., 2021-01-01. (19/07316-0, 17/22366-8, 19/02033-0)

DOS SANTOS, FERNANDO P.; RIBEIRO, LEONARDO S. F.; PONTI, MOACIR A.. Generalization of feature embeddings transferred from different video anomaly detection domains. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, v. 60, p. 407-416, APR 2019. (13/07375-0, 18/22482-0, 17/22366-8)

RIBEIRO, LEO SAMPAIO FERRAZ; BUI, TU; COLLOMOSSE, JOHN; PONTI, MOACIR. Sketchformer: Transformer-based Representation for Sketched Structure. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), v. N/A, p. 10-pg., 2020-01-01. (18/22482-0, 19/02808-1, 17/22366-8)

FERRAZ RIBEIRO, LEO SAMPAIO; BUI, TU; COLLOMOSSE, JOHN; PONTI, MOACIR; IEEE COMP SOC. Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), v. N/A, p. 10-pg., 2021-01-01. (19/02808-1, 17/22366-8, 19/07316-0)

SAMPAIO FERRAZ RIBEIRO, LEO; BUI, TU; COLLOMOSSE, JOHN; PONTI, MOACIR. Scene designer: compositional sketch-based image retrieval with contrastive learning and an auxiliary synthesis task. MULTIMEDIA TOOLS AND APPLICATIONS, v. N/A, p. 23-pg., 2022-12-20. (19/02808-1, 17/22366-8, 19/07316-0)

CAVALLARI, GABRIEL B.; RIBEIRO, LEONARDO S. F.; PONTI, MOACIR A.; IEEE. Unsupervised representation learning using convolutional and stacked auto-encoders: a domain and cross-domain feature space analysis. PROCEEDINGS 2018 31ST SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 7-pg., 2018-01-01. (17/22366-8, 16/16111-4, 13/07375-0)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

RIBEIRO, Leo Sampaio Ferraz. Cross Domain Visual Search with Feature Learning using Multi-stream Transformer-based Architectures. 2023. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB) São Carlos.