Deep Manifold Alignment for Mid-Grain Sketch Based Image Retrieval

Autor(es)

Bui, Tu ; Ribeiro, Leonardo ; Ponti, Moacir ; Collomosse, John ; Jawahar, CV ; Li, H ; Mori, G ; Schindler, K

• Tipo de documento: Artigo Científico
• Fonte: COMPUTER VISION - ACCV 2018, PT III; v. 11363, p. 16-pg., 2019-01-01.

Resumo

We present an algorithm for visually searching image collections using free-hand sketched queries. Prior sketch based image retrieval (SBIR) algorithms adopt either a category-level or fine-grain (instance-level) definition of cross-domain similarity-returning images that match the sketched object class (category-level SBIR), or a specific instance of that object (fine-grain SBIR). In this paper we take the middle-ground; proposing an SBIR algorithm that returns images sharing both the object category and key visual characteristics of the sketched query without assuming photo-approximate sketches from the user. We describe a deeply learned cross-domain embedding in which 'mid-grain' sketch-image similarity may be measured, reporting on the efficacy of unsupervised and semi-supervised manifold alignment techniques to encourage better intra-category (mid-grain) discrimination within that embedding. We propose a new mid-grain sketch-image dataset (MidGrain65c) and demonstrate not only mid-grain discrimination, but also improved category-level discrimination using our approach. (AU)

• Processo FAPESP: 17/10068-2 - Métodos de redução de dimensionalidade em representações geradas por redes convolucionais triplet
• Beneficiário: Leo Sampaio Ferraz Ribeiro
• Modalidade de apoio: Bolsas no Brasil - Iniciação Científica
• Processo FAPESP: 16/16111-4 - Aprendizado de características na recuperação de imagens baseada em rascunhos e no sensoriamento remoto de baixa altitude
• Beneficiário: Moacir Antonelli Ponti
• Modalidade de apoio: Auxílio à Pesquisa - Regular
• Processo FAPESP: 13/07375-0 - CeMEAI - Centro de Ciências Matemáticas Aplicadas à Indústria
• Beneficiário: Francisco Louzada Neto
• Modalidade de apoio: Auxílio à Pesquisa - Centros de Pesquisa, Inovação e Difusão - CEPIDs