Convolutional neural network simplification with progressive retraining

Autor(es)

Osaku, D. ^[1] ; Gomes, J. F. ^[1] ; Falcao, A. X. ^[1]

Afiliação do(s) autor(es)

^[1] Univ Estadual Campinas, Campinas - Brazil

• Tipo de documento: Artigo Científico
• Fonte: PATTERN RECOGNITION LETTERS; v. 150, p. 235-241, OCT 2021.
• Citações Web of Science: 0

Resumo

Kernel pruning methods have been proposed to speed up (simplify) convolutional neural network (CNN) models. However, the effectiveness of a simplified model is often below the original one. This letter presents new methods based on objective and subjective relevance criteria for kernel elimination in a layer-by-layer fashion. During the process, a CNN model is retrained only when the current layer is en-tirely simplified by adjusting the weights from the next layer to the first one and preserving weights of subsequent layers not involved in the process. We call this strategy progressive retraining, differently from kernel pruning methods that usually retrain the entire model after eliminating one or a few ker-nels. Our subjective relevance criterion exploits humans' ability to recognize visual patterns and improve the designer's understanding of the simplification process. We show that our methods can increase ef-fectiveness with considerable model simplification, outperforming two popular approaches and another one from the state-of-the-art on four challenging image datasets. An indirect comparison with 14 recent methods on a famous image dataset also places our approach using the objective criterion among the most competitive ones. (c) 2021 Elsevier B.V. All rights reserved. (AU)

• Processo FAPESP: 14/12236-1 - AnImaLS: Anotação de Imagem em Larga Escala: o que máquinas e especialistas podem aprender interagindo?
• Beneficiário: Alexandre Xavier Falcão
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 17/12974-0 - Aprendizado de máquina guiado por visualização de dados multidimensionais
• Beneficiário: Daniel Osaku
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado