Feature learning applied to sketch-based image retrieval and low-altitude remote sensing

• Grant number: 16/16111-4
• Support Opportunities: Regular Research Grants
• Start date: February 01, 2017
• End date: January 31, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Moacir Antonelli Ponti
• Grantee: Moacir Antonelli Ponti
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated researchers: John Collomosse

Abstract

Deep learning methods have reached state of the art performance in several areas. Although research in this field have achieved excellent results in benchmark datasets, there is lack of understanding about how the methods work, and applications yet to be investigated, in particular when going beyond standard convolutional neural networks architectures. In this project we propose the use of feature learning applied to the analysis of low-altitude remote sensing for precision agriculture and the sketch-based image retrieval. Each task has its own challenges, but in common there is limited labelled data to be trained with. Those can be solved using deep learning framework by exploring new architectures based on auto-encoders, Siamese networks and also generative models. We propose to evaluate the models not only using benchmark datasets, but also assess the quality of the representations by using visualisation and projection techniques as a way to analyse the output feature spaces. The expected results include the development of models that, trained with limited availability of labels (or even unsupervised), are still able to generalise for unseen data and categories. In addition to the contributions in the computer vision field, we expect to advance the state-of-the-art on the applications. (AU)

Scientific publications

(9)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

BUI, TU; RIBEIRO, LEONARDO; PONTI, MOACIR; COLLOMOSSE, JOHN. Sketching out the details: Sketch-based image retrieval using convolutional neural networks with multi-stage regression. COMPUTERS & GRAPHICS-UK, v. 71, p. 77-87, APR 2018. (16/16111-4, 13/07375-0, 17/10068-2)

PONTI, MOACIR A.; IEEE. Relevance image sampling from collection using importance selection on randomized optimum-path trees. 2017 6TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2017-01-01. (16/16111-4)

BUI, T.; RIBEIRO, L.; PONTI, M.; COLLOMOSSE, J.. Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network. COMPUTER VISION AND IMAGE UNDERSTANDING, v. 164, p. 11-pg., 2017-11-01. (15/26050-0, 16/16111-4)

BUI, TU; RIBEIRO, LEONARDO; PONTI, MOACIR; COLLOMOSSE, JOHN; JAWAHAR, CV; LI, H; MORI, G; SCHINDLER, K. Deep Manifold Alignment for Mid-Grain Sketch Based Image Retrieval. COMPUTER VISION - ACCV 2018, PT III, v. 11363, p. 16-pg., 2019-01-01. (17/10068-2, 16/16111-4, 13/07375-0)

CAVALLARI, GABRIEL B.; RIBEIRO, LEONARDO S. F.; PONTI, MOACIR A.; IEEE. Unsupervised representation learning using convolutional and stacked auto-encoders: a domain and cross-domain feature space analysis. PROCEEDINGS 2018 31ST SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 7-pg., 2018-01-01. (17/22366-8, 16/16111-4, 13/07375-0)

BUI, T.; RIBEIRO, L.; PONTI, M.; COLLOMOSSE, J.. Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network. COMPUTER VISION AND IMAGE UNDERSTANDING, v. 164, n. SI, p. 27-37, NOV 2017. (15/26050-0, 16/16111-4)

NAZARE, TIAGO S.; PARANHOS DA COSTA, GABRIEL B.; DE MELLO, RODRIGO F.; PONTI, MOACIR A.; IEEE. Color quantization in transfer learning and noisy scenarios: an empirical analysis using convolutional networks. PROCEEDINGS 2018 31ST SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 7-pg., 2018-01-01. (15/05310-3, 17/16548-6, 13/07375-0, 15/04883-0, 16/16111-4)

PONTI, MOACIR A.; DA COSTA, GABRIEL B. PARANHOS; SANTOS, FERNANDO P.; SILVEIRA, KAUE U.. Supervised and unsupervised relevance sampling in handcrafted and deep learning features obtained from image collections. APPLIED SOFT COMPUTING, v. 80, p. 414-424, JUL 2019. (16/16111-4, 13/07375-0, 15/05310-3)

DOS SANTOS, FERNANDO PEREIRA; PONTI, MOACIR A.; IEEE. Robust feature spaces from pre-trained deep network layers for skin lesion classification. PROCEEDINGS 2018 31ST SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2018-01-01. (16/16111-4, 13/07375-0)