Representation learning of sketches and images for recognition, search and cross-domain synthesis

• Grant number: 21/06462-2
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: December 01, 2021
• End date: October 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Moacir Antonelli Ponti
• Grantee: Luísa Balleroni Shimabucoro
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated scholarship(s): 22/09913-8 - Evaluation of few-shot learning models using performance estimates and ranking, BE.EP.IC

Abstract

Representation Learning often used to perform search, classification and cross-domain generation tasks have shown great progress in the last few decades with the help of deep learning models, particularly with regards to Recurrent Neural Network (RNN) architectures applied to the domain of sketches. Nonetheless, these models neglect spatial-temporal dependencies, which results in limited representations due to contextualization issues. Transformer-based architectures, however, not only present an improvement opportunity with respect to the previously referred aspects but also have a reduced computational cost, which allows for additional advances inside this research scope. Therefore, this project aims to conduct studies and experiments within the field of representation learning applied to sketches so as to analyze its behavior and be able to explore new methods which can lead to the refinement of the classification, image search, and sketch synthesis activities. (AU)