Integration of manually extracted image features into convolutional neural networks

• Grant number: 24/23406-7
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: February 01, 2025
• End date: January 31, 2027
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Nina Sumiko Tomita Hirata
• Grantee: Guilherme Dias Nunes de Abreu
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated research grant: 22/15304-4 - Learning context rich representations for computer vision, AP.TEM

Abstract

Convolutional neural networks (CNNs) are widely used in computer vision for different important tasks in image analysis. In scenarios with limited training data, some studies in the literature show that integrating features extracted from images by manually designed algorithms into CNNs can help improve performance. In this project, we are interested in better understanding this fact. In particular, we intend to investigate whether feature maps, when added as additional channels to the CNN input image, work as a positive inductive bias in the sense of facilitating the training process or leading to superior performance. Different adaptations of CNN architectures to process these additional channels will be investigated. We also intend to investigate whether there is any clear relationship between the amount of training data and eventual performance improvements when the networks are trained with additional channels. In terms of application domain, the studied methods will be evaluated in the context of computer vision tasks related to the analysis of eye fundus images. (AU)