Advanced search
Start date
Betweenand

Hyperparameter optimization in deep learning arquitectures

Grant number: 13/20387-7
Support type:Scholarships abroad - Research
Effective date (Start): March 01, 2014
Effective date (End): February 28, 2015
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Principal Investigator:João Paulo Papa
Grantee:João Paulo Papa
Host: David Cox
Home Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
Local de pesquisa : Harvard University, Cambridge, United States  
Associated research grant:09/16206-1 - New trends on optimum-path forest-based pattern recognition, AP.JP

Abstract

%Deep learning architectures have been extensively studied in the last years, since their philosophy consists into using a complex hierarchical structure for information learning and representation, being such representation analogous to the human neural processing. Such architectures are composed by several steps, which aim to use an image (in case of computer vision applications) in a filtering process using a filter bank for further information extraction and sampling. Then, this image is modified and forwarded to a new operation layer, in the same way as conducted in the previous one. At the final of the process, it is obtained a high dimensional description of this image, being such representation employed in a traditional classification process. The main problem concerns with the number of parameters of the hole process (filter number and sizes, sampling rate and classifiers' parameters, for instance), which are so called hyperparameters and they are fundamental for the success of the information extraction, representation and classification. Therefore, we can model the task of finding such parameters as an optimization problem. Since a few number of works have addressed evolutionary optimization techniques in this context, this post-doctorate research project aims to employ and evaluate such optimization techniques for hyperparameter optimization in deep learning architectures. Additionally, this work as the goal to evaluate the Optimum-Path Forest classifier, which was developed by the proposer of this project, in the context of data classification in deep learning, since OPF has never been applied for such purpose so far. (AU)

Scientific publications (15)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
AMORIM, WILLIAN P.; FALCAO, ALEXANDRE X.; PAPA, JOAO P. Multi-label semi-supervised classification through optimum-path forest. INFORMATION SCIENCES, v. 465, p. 86-104, OCT 2018. Web of Science Citations: 3.
PEREIRA, DANILLO R.; PISANI, RODRIGO J.; DE SOUZA, ANDRE N.; PAPA, JOAO P. An Ensemble-Based Stacked Sequential Learning Algorithm for Remote Sensing Imagery Classification. IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, v. 10, n. 4, p. 1525-1541, APR 2017. Web of Science Citations: 0.
PAPA, JOAO PAULO; NACHIF FERNANDES, SILAS EVANDRO; FALCAO, ALEXANDRE XAVIER. Optimum-Path Forest based on k-connectivity: Theory and applications. PATTERN RECOGNITION LETTERS, v. 87, n. SI, p. 117-126, FEB 1 2017. Web of Science Citations: 16.
AMORIM, WILLIAN P.; FALCAO, ALEXANDRE X.; PAPA, JOAO P.; CARVALHO, MARCELO H. Improving semi-supervised learning through optimum connectivity. PATTERN RECOGNITION, v. 60, p. 72-85, DEC 2016. Web of Science Citations: 13.
PASSOS JUNIOR, LEANDRO APARECIDO; OBA RAMOS, CAIO CESAR; RODRIGUES, DOUGLAS; PEREIRA, DANILLO ROBERTO; DE SOUZA, ANDRE NUNES; PONTARA DA COSTA, KELTON AUGUSTO; PAPA, JOAO PAULO. Unsupervised non-technical losses identification through optimum-path forest. Electric Power Systems Research, v. 140, p. 413-423, NOV 2016. Web of Science Citations: 13.
PEREIRA, CLAYTON R.; PEREIRA, DANILO R.; SILVA, FRANCISCO A.; MASIEIRO, JOAO P.; WEBER, SILKE A. T.; HOOK, CHRISTIAN; PAPA, JOAO P. A new computer vision-based approach to aid the diagnosis of Parkinson's disease. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, v. 136, p. 79+, NOV 2016. Web of Science Citations: 16.
PAPA, JOAO PAULO; SCHEIRER, WALTER; COX, DAVID DANIEL. Fine-tuning Deep Belief Networks using Harmony Search. APPLIED SOFT COMPUTING, v. 46, p. 875-885, SEP 2016. Web of Science Citations: 36.
PIRES, RAFAEL G.; PEREIRA, DANILLO R.; PEREIRA, LUIS A. M.; MANSANO, ALEX F.; PAPA, JOO P. Projections onto convex sets parameter estimation through harmony search and its application for image restoration. NATURAL COMPUTING, v. 15, n. 3, SI, p. 493-502, SEP 2016. Web of Science Citations: 3.
OSAKU, DANIEL; PEREIRA, DANILLO R.; LEVADA, ALEXANDRE L. M.; PAPA, JOAO P. Fine-Tuning Contextual-Based Optimum-Path Forest for Land-Cover Classification. IEEE Geoscience and Remote Sensing Letters, v. 13, n. 5, p. 735-739, MAY 2016. Web of Science Citations: 1.
PEREIRA, DANILLO R.; PAZOTI, MARIO A.; PEREIRA, LUIS A. M.; RODRIGUES, DOUGLAS; RAMOS, CAIO O.; SOUZA, ANDRE N.; PAPA, JOAO P. Social-Spider Optimization-based Support Vector Machines applied for energy theft detection. COMPUTERS & ELECTRICAL ENGINEERING, v. 49, p. 25-38, JAN 2016. Web of Science Citations: 17.
OSAKU, D.; NAKAMURA, R. Y. M.; PEREIRA, L. A. M.; PISANI, R. J.; LEVADA, A. L. M.; CAPPABIANCO, F. A. M.; FALCO, A. X.; PAPA, JOAO P. Improving land cover classification through contextual-based optimum-path forest. INFORMATION SCIENCES, v. 324, p. 60-87, DEC 10 2015. Web of Science Citations: 13.
PAPA, JOAO P.; ROSA, GUSTAVO H.; MARANA, APARECIDO N.; SCHEIRER, WALTER; COX, DAVID D. Model selection for Discriminative Restricted Boltzmann Machines through meta-heuristic techniques. JOURNAL OF COMPUTATIONAL SCIENCE, v. 9, n. SI, p. 14-18, JUL 2015. Web of Science Citations: 24.
SAITO, PRISCILA T. M.; NAKAMURA, RODRIGO Y. M.; AMORIM, WILLIAN P.; PAPA, JOAO P.; DE REZENDE, PEDRO J.; FALCAO, ALEXANDRE X. Choosing the Most Effective Pattern Classification Model under Learning-Time Constraint. PLoS One, v. 10, n. 6 JUN 26 2015. Web of Science Citations: 3.
PEREIRA, DANILLO R.; DELPIANO, JOSE; PAPA, JOAO P. On the optical flow model selection through metaheuristics. EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING, MAY 9 2015. Web of Science Citations: 4.
COSTA, KELTON A. P.; PEREIRA, LUIS A. M.; NAKAMURA, RODRIGO Y. M.; PEREIRA, CLAYTON R.; PAPA, JOAO P.; FALCAO, ALEXANDRE XAVIER. A nature-inspired approach to speed up optimum-path forest clustering and its application to intrusion detection in computer networks. INFORMATION SCIENCES, v. 294, p. 95-108, FEB 10 2015. Web of Science Citations: 30.

Please report errors in scientific publications list by writing to: cdi@fapesp.br.