Advanced search
Start date

Feature learning applied to sketch-based image retrieval and low-altitude remote sensing

Grant number: 16/16111-4
Support type:Regular Research Grants
Duration: February 01, 2017 - January 31, 2019
Field of knowledge:Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
Principal Investigator:Moacir Antonelli Ponti
Grantee:Moacir Antonelli Ponti
Home Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
Assoc. researchers: John Collomosse


Deep learning methods have reached state of the art performance in several areas. Although research in this field have achieved excellent results in benchmark datasets, there is lack of understanding about how the methods work, and applications yet to be investigated, in particular when going beyond standard convolutional neural networks architectures. In this project we propose the use of feature learning applied to the analysis of low-altitude remote sensing for precision agriculture and the sketch-based image retrieval. Each task has its own challenges, but in common there is limited labelled data to be trained with. Those can be solved using deep learning framework by exploring new architectures based on auto-encoders, Siamese networks and also generative models. We propose to evaluate the models not only using benchmark datasets, but also assess the quality of the representations by using visualisation and projection techniques as a way to analyse the output feature spaces. The expected results include the development of models that, trained with limited availability of labels (or even unsupervised), are still able to generalise for unseen data and categories. In addition to the contributions in the computer vision field, we expect to advance the state-of-the-art on the applications. (AU)

Scientific publications
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
PONTI, MOACIR A.; DA COSTA, GABRIEL B. PARANHOS; SANTOS, FERNANDO P.; SILVEIRA, KAUE U. Supervised and unsupervised relevance sampling in handcrafted and deep learning features obtained from image collections. APPLIED SOFT COMPUTING, v. 80, p. 414-424, JUL 2019. Web of Science Citations: 0.
BUI, TU; RIBEIRO, LEONARDO; PONTI, MOACIR; COLLOMOSSE, JOHN. Sketching out the details: Sketch-based image retrieval using convolutional neural networks with multi-stage regression. COMPUTERS & GRAPHICS-UK, v. 71, p. 77-87, APR 2018. Web of Science Citations: 3.
BUI, T.; RIBEIRO, L.; PONTI, M.; COLLOMOSSE, J. Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network. COMPUTER VISION AND IMAGE UNDERSTANDING, v. 164, n. SI, p. 27-37, NOV 2017. Web of Science Citations: 10.

Please report errors in scientific publications list by writing to: