Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network

Autor(es)

Bui, T. ; Ribeiro, L. ; Ponti, M. ; Collomosse, J.

• Tipo de documento: Artigo Científico
• Fonte: COMPUTER VISION AND IMAGE UNDERSTANDING; v. 164, p. 11-pg., 2017-11-01.

Resumo

\We present an efficient representation for sketch based image retrieval (SBIR) derived from a triplet loss convolutional neural network (CNN). We treat SBIR as a cross-domain modelling problem, in which a depiction invariant embedding of sketch and photo data is learned by regression over a siamese CNN architecture with half-shared weights and modified triplet loss function. Uniquely, we demonstrate the ability of our learned image descriptor to generalise beyond the categories of object present in our training data, forming a basis for general cross-category SBIR. We explore appropriate strategies for training, and for deriving a compact image descriptor from the learned representation suitable for indexing data on resource constrained e.g. mobile devices. We show the learned descriptors to outperform state of the art SBIR on the defacto standard Flickrl5k dataset using a significantly more compact (56 bits per image, i. e. approximate to 105KB total) search index than previous methods. Datasets and models are available from the CVSSP datasets server at www.cvssp.org. (C) 2017 Elsevier Inc. All rights reserved. (AU)

• Processo FAPESP: 15/26050-0 - Ranking de imagens multi-instâncias para recuperação de imagens baseada em rascunhos
• Beneficiário: Leo Sampaio Ferraz Ribeiro
• Modalidade de apoio: Bolsas no Exterior - Estágio de Pesquisa - Iniciação Científica
• Processo FAPESP: 16/16111-4 - Aprendizado de características na recuperação de imagens baseada em rascunhos e no sensoriamento remoto de baixa altitude
• Beneficiário: Moacir Antonelli Ponti
• Modalidade de apoio: Auxílio à Pesquisa - Regular