Image classification in frequency domain with 2SReLU: A second harmonics superposition activation function

Author(s)

Watanabe, Thomio ^[1] ; Wolf, Denis F. ^[1]

Affiliation

^[1] Univ Sao Paulo, Inst Math & Comp Sci, Sao Paulo - Brazil

• Document type: Journal article
• Source: APPLIED SOFT COMPUTING; v. 112, NOV 2021.
• Web of Science Citations: 0

Abstract

Deep Convolutional Neural Networks are able to identify complex patterns and perform tasks with super-human capabilities. However, besides the exceptional results, they are not completely understood and it is still impractical to hand-engineer similar solutions. In this work, an image classification Convolutional Neural Network and its building blocks are described from a frequency domain perspective. Some network layers have established counterparts in the frequency domain like the convolutional and pooling layers. We propose the 2SReLU layer, a novel non-linear activation function that preserves high frequency components in deep networks. A convolution-free network is presented, and it is demonstrated that in the frequency domain it is possible to achieve competitive results without using the computationally costly convolution operation. A source code implementation in PyTorch is provided at: https://gitlab.com/thomio/2srelu. (C) 2021 Elsevier B.V. All rights reserved. (AU)

• FAPESP's process: 15/26293-0 - Autonomous vehicles multi obstacle tracking with sensor fusion.
• Grantee: Thomio Watanabe
• Support Opportunities: Scholarships in Brazil - Doctorate