| Full text | |
| Author(s): |
Goncalves do Santos, Claudio Filipi
;
Colombo, Danilo
;
Roder, Mateus
;
Papa, Joao Paulo
;
IEEE COMP SOC
Total Authors: 5
|
| Document type: | Journal article |
| Source: | 2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR); v. N/A, p. 6-pg., 2021-01-01. |
| Abstract | |
Different techniques have emerged in the deep learning scenario, such as Convolutional Neural Networks, Deep Belief Networks, and Long Short-Term Memory Networks, to cite a few. In lockstep, regularization methods, which aim to prevent overfitting by penalizing the weight connections, or turning off some units, have been widely studied either. In this paper, we present a novel approach called MaxDropout, a regularizer for deep neural network models that works in a supervised fashion by removing (shutting off) the prominent neurons (i.e., most active) in each hidden layer. The model forces fewer activated units to learn more representative information, thus providing sparsity. Regarding the experiments, we show that it is possible to improve existing neural networks and provide better results in neural networks when Dropout is replaced by MaxDropout. The proposed method was evaluated in image classification, achieving comparable results to existing regularizers, such as Cutout and RandomErasing, also improving the accuracy of neural networks that uses Dropout by replacing the existing layer by MaxDropout. (AU) | |
| FAPESP's process: | 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction? |
| Grantee: | Alexandre Xavier Falcão |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 19/07825-1 - Deep Boltzmann machines for event recognition in videos |
| Grantee: | Mateus Roder |
| Support Opportunities: | Scholarships in Brazil - Master |
| FAPESP's process: | 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry |
| Grantee: | Francisco Louzada Neto |
| Support Opportunities: | Research Grants - Research, Innovation and Dissemination Centers - RIDC |
| FAPESP's process: | 17/25908-6 - Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert |
| Grantee: | João Paulo Papa |
| Support Opportunities: | Research Grants - Research Partnership for Technological Innovation - PITE |