On the Study of Semantics in Deep Learning Models

• Grant number: 15/25739-4
• Support Opportunities: Scholarships in Brazil - Master
• Start date: March 01, 2016
• End date: July 01, 2018
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: João Paulo Papa
• Grantee: Gustavo Henrique de Rosa
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Associated scholarship(s): 16/21243-7 - Learning Dropout Parameters for Convolutional Neural Networks, BE.EP.MS

Abstract

Deep learning architectures have been extensively studied in the last years, mainly due to their discriminative power and effectiveness in many crucial problems in computer vision, such as face and people identification, as well as object recognition, just to name a few. However, one problem related to these models concerns with their number of parameters, which can easily reach thousands of hundreds. Another drawback is related to the need for large datasets for training purposes, as well as their high probability of overtraining, mainly because of their complex architecture. Although some recent works have proposed different solutions to alleviate this problem, such approaches still require parameters, which need to be fine-tuned, and also are skilled-dependent. This proposal aims at "learning how these techniques do learn", i.e., to learn their good points and shortcomings. Techniques such as Restricted Boltzmann Machines and Deep Belief Nets will be studied in order to obtain a better understanding of their working mechanism through meta-heuristic-based optimization. This proposal also comprises an internship at Middlesex University, United Kingdom.

Scientific publications

(16)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

PAPA, JOAO PAULO; ROSA, GUSTAVO HENRIQUE; PAPA, LUCIENE PATRICI. A binary-constrained Geometric Semantic Genetic Programming for feature selection purposes. PATTERN RECOGNITION LETTERS, v. 100, p. 59-66, DEC 1 2017. (13/07375-0, 14/16250-9, 16/19403-6, 10/15566-1, 14/12236-1, 15/25739-4)

PASSOS, LEANDRO A.; DE SOUZA, JR., LUIS A.; MENDEL, ROBERT; EBIGBO, ALANNA; PROBST, ANDREAS; MESSMANN, HELMUT; PALM, CHRISTOPH; PAPA, JOAO PAULO. Barrett's esophagus analysis using infinity Restricted Boltzmann Machines. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, v. 59, p. 475-485, FEB 2019. (14/16250-9, 14/12236-1, 13/07375-0, 15/25739-4, 16/21243-7)

AMORIM, WILLIAN PARAGUASSU; ROSA, GUSTAVO HENRIQUE; THOMAZELLA, ROGERIO; COGO CASTANHO, JOSE EDUARDO; LOFRANO DOTTO, FABIO ROMANO; RODRIGUES JUNIOR, OSWALDO PONS; MARANA, APARECIDO NILCEU; PAPA, JOAO PAULO. Semi-supervised learning with connectivity-driven convolutional neural networks. PATTERN RECOGNITION LETTERS, v. 128, p. 16-22, DEC 1 2019. (14/12236-1, 13/07375-0, 15/25739-4, 16/19403-6)

PISANI, R.; COSTA, K.; ROSA, G.; PEREIRA, D.; PAPA, J.; TAVARES, J. M. R. S.; IEEE. RIVER SEDIMENT YIELD CLASSIFICATION USING REMOTE SENSING IMAGERY. 2016 9TH IAPR WORKSHOP ON PATTERN RECOGNITION IN REMOTE SENSING (PRRS), v. N/A, p. 6-pg., 2016-01-01. (14/16250-9, 15/00801-9, 15/25739-4)

PEREIRA, CLAYTON R.; WEBER, SILKE A. T.; HOOK, CHRISTIAN; ROSA, GUSTAVO H.; PAPA, JOAO; IEEE. Deep Learning-aided Parkinson's Disease Diagnosis from Handwritten Dynamics. 2016 29TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 7-pg., 2016-01-01. (15/25739-4, 14/16250-9, 10/15566-1)

THOMAZELLA, R.; CASTANHO, J. E.; DOTTO, F. R. L.; RODRIGUES JUNIOR, O. P.; ROSA, G. H.; MARANA, A. N.; PAPA, J. P.; IEEE. ENVIRONMENTAL MONITORING USING DRONE IMAGES AND CONVOLUTIONAL NEURAL NETWORKS. IGARSS 2018 - 2018 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, v. N/A, p. 4-pg., 2018-01-01. (13/07375-0, 15/25739-4, 14/12236-1, 16/19403-6)

DE ROSA, GUSTAVO H.; PAPA, JOAO P.; YANG, XIN-S. Handling dropout probability estimation in convolution neural networks using meta-heuristics. SOFT COMPUTING, v. 22, n. 18, SI, p. 6147-6156, SEP 2018. (14/12236-1, 14/16250-9, 15/25739-4)

AMORIM, WILLIAN PARAGUASSU; TETILA, EVERTON CASTELAO; PISTORI, HEMERSON; PAPA, JOAO PAULO. Semi-supervised learning with convolutional neural networks for UAV images automatic recognition. COMPUTERS AND ELECTRONICS IN AGRICULTURE, v. 164, SEP 2019. (14/12236-1, 13/07375-0, 15/25739-4, 16/19403-6)

PAPA, JOAO PAULO; ROSA, GUSTAVO H.; PEREIRA, DANILLO R.; YANG, XIN-SHE. Quaternion-based Deep Belief Networks fine-tuning. APPLIED SOFT COMPUTING, v. 60, p. 328-335, NOV 2017. (14/16250-9, 14/12236-1, 15/25739-4)

PEREIRA, DANILLO ROBERTO; REBOUCAS FILHO, PEDRO P.; DE ROSA, GUSTAVO HENRIQUE; PAPA, JOAO PAULO; DE ALBUQUERQUE, VICTOR HUGO C.; IEEE. Stroke Lesion Detection Using Convolutional Neural Networks. 2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), v. N/A, p. 6-pg., 2018-01-01. (14/16250-9, 14/12236-1, 15/25739-4, 16/21243-7)

DE ROSA, GUSTAVO H.; PAPA, JOAO P.; YANG, XIN-S. Handling dropout probability estimation in convolution neural networks using meta-heuristics. SOFT COMPUTING, v. 22, n. 18, p. 10-pg., 2018-09-01. (15/25739-4, 14/16250-9, 14/12236-1)

PEREIRA, CLAYTON R.; PEREIRA, DANILO R.; ROSA, GUSTAVO H.; ALBUQUERQUE, VICTOR H. C.; WEBER, SILKE A. T.; HOOK, CHRISTIAN; PAPA, JOAO P.. Handwritten dynamics dynamics assessment through convolutional neural networks: An application to Parkinson's disease identification. ARTIFICIAL INTELLIGENCE IN MEDICINE, v. 87, p. 67-77, MAY 2018. (13/07375-0, 14/16250-9, 16/19403-6, 10/15566-1, 14/12236-1, 15/25739-4)

ROSA, G. H.; PAPA, J. P.; PAPA, L. P.; OCHOA, G. Feature Selection Using Geometric Semantic Genetic Programming. PROCEEDINGS OF THE 2017 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE COMPANION (GECCO'17 COMPANION), v. N/A, p. 2-pg., 2017-01-01. (14/12236-1, 15/25739-4)

PASSOS, LEANDRO A.; PEREIRA, CLAYTON R.; REZENDE, EDMAR R. S.; CARVALHO, TIAGO J.; WEBER, SILKE A. T.; HOOK, CHRISTIAN; PAPA, JOAO P.; IEEE. Parkinson Disease Identification using Residual Networks and Optimum-Path Forest. 2018 IEEE 12TH INTERNATIONAL SYMPOSIUM ON APPLIED COMPUTATIONAL INTELLIGENCE AND INFORMATICS (SACI), v. N/A, p. 5-pg., 2018-01-01. (13/07375-0, 16/21243-7, 14/12236-1, 15/25739-4, 14/16250-9)

PEREIRA, CLAYTON R.; PASSOS, LEANDRO A.; LOPES, RICARDO R.; WEBER, SILKE A. T.; HOOK, CHRISTIAN; PAPA, JOAO PAULO; FELSBERG, M; HEYDEN, A; KRUGER, N. Parkinson's Disease Identification Using Restricted Boltzmann Machines. COMPUTER ANALYSIS OF IMAGES AND PATTERNS, v. 10425, p. 11-pg., 2017-01-01. (14/16250-9, 15/25739-4, 14/12236-1, 16/21243-7)

ROSA, GUSTAVO H.; AFONSO, LUIS C. S.; BALDASSIN, ALEXANDRO; PAPA, JOAO P.; YANG, XIN-SHE; FELSBERG, M; HEYDEN, A; KRUGER, N. Quaternionic Flower Pollination Algorithm. COMPUTER ANALYSIS OF IMAGES AND PATTERNS, v. 10425, p. 12-pg., 2017-01-01. (14/16250-9, 15/25739-4, 14/12236-1, 16/21243-7)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

ROSA, Gustavo Henrique de. Meta-Heuristic Optimization-based Regularization for Deep Learning Architectures. 2018. Master's Dissertation - Universidade Estadual Paulista (Unesp). Instituto de Biociências Letras e Ciências Exatas. São José do Rio Preto São José do Rio Preto.