A Human-in-the-loop Approach to Build Convolutional Neural Networks

• Grant number: 23/09210-0
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: November 01, 2023
• End date: February 28, 2027
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Alexandre Xavier Falcão
• Grantee: Matheus Abrantes Cerqueira
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil

Abstract

Convolutional Neural Networks (CNNs) highly improved image-pattern recognition tasks primarily due to their feature learning function (encoder). However, the absence of designing methodologies under expert control often results in obscure (``opaque-box'') functionality models that are more complex than necessary. This project aims to develop a human-in-the-loop approach to build the encoder of a CNN layer by layer under expert control over its depth and functionality. The proposal extends a previous method developed during the MSc work of the beneficiary, which estimates all filters of an encoder with a given architecture from markers drawn by the expert in discriminative regions of a few selected images. The method does not require large annotated datasets since it does not rely on backpropagation. The current project will investigate solutions for each process step: representative image selection, discriminative (attention) regions for drawing markers, methods to estimate filters from marker pixels, and visual feedback approaches to guide filter selection by the expert layer by layer. By that, the expert should understand and control the model's complexity for a given problem. For validation, the project will address the classification of gastrointestinal parasites from microscopy images -- a real-world application in which our research group has complete control over the image acquisition, data annotation, and data management processes. For international experience, part of the project (a BEPE project) will be developed either at the University of Rennes, under the supervision of Ewa Kijak, or at Utrecht University, under the supervision of Alexandru Telea - both have collaborated with our research group.

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

CERQUEIRA, MATHEUS A.; SPRENGER, FLAVIA; TEIXEIRA, BERNARDO C. A.; GUIMARAES, SILVIO JAMIL E.; FALCAO, ALEXANDRE X.. Interactive Ground-Truth-Free Image Selection for FLIM Segmentation Encoders. 2024 37TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES, SIBGRAPI 2024, v. N/A, p. 6-pg., 2024-01-01. (23/09210-0, 23/14427-8, 13/07375-0)