Alignment of Local and Global Features from Multiple Layers of Convolutional Neural Network for Image Classification

Author(s)

dos Santos, Fernando Pereira ; Ponti, Moacir Antonelli ; IEEE

• Document type: Journal article
• Source: 2019 32ND SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI); v. N/A, p. 8-pg., 2019-01-01.

Abstract

Convolutional networks have been extensively applied to obtain features spaces for classification tasks. Although those achieve high accuracy in many scenarios, typically only the top layers of the network are explored. Hence, a relevant question arises from this fact: are initial layers useful in terms of discriminative ability? In this paper, we leverage the complementary description offered by such first layers. Our method consists of features extraction in multiple layers, followed by feature selection, fusion of feature maps from the different layers, and space alignment. Through an extensive experimentation with different datasets and studying different training strategies, our results show that local information, coming from the first layers, may significantly improve the classification performance when merged with a global descriptor extracted from a top layer of the network. We report different methods for reducing the dimensionality of the local descriptors, and guidelines on how to align them so that to perform fusion. Our study encourages future studies on combining feature maps from multiple layers, which may be relevant in particular for transfer learning scenarios. (AU)

• FAPESP's process: 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry
• Grantee: Francisco Louzada Neto
• Support Opportunities: Research Grants - Research, Innovation and Dissemination Centers - RIDC
• FAPESP's process: 18/22482-0 - Learning features from visual content under limited supervision using multiple domains
• Grantee: Moacir Antonelli Ponti
• Support Opportunities: Regular Research Grants