Class-specific early exit design methodology for convolutional neural networks

Author(s)

Bonato, Vanderlei ^[1] ; Bouganis, Christos-Savvas ^[2]

Affiliation

^[1] Univ Sao Paulo, Inst Math & Comp Sci, Sao Carlos, SP - Brazil

^[2] Imperial Coll London, Dept Elect & Elect Engn, London - England

• Document type: Journal article
• Source: APPLIED SOFT COMPUTING; v. 107, AUG 2021.
• Web of Science Citations: 0

Abstract

Convolutional Neural Network-based (CNN) inference is a demanding computational task where a long sequence of operations is applied to an input as dictated by the network topology. Optimisations by data quantisation, data reuse, network pruning, and dedicated hardware architectures have a strong impact on reducing both energy consumption and hardware resource requirements, and on improving inference latency. Implementing new applications from established models available from both academic and industrial worlds is common nowadays. Further optimisations by preserving model architecture have been proposed via early exiting approaches, where additional exit points are included in order to evaluate classifications of samples that produce feature maps with sufficient evidence to be classified before reaching the final model exit. This paper proposes a methodology for designing early-exit networks from a given baseline model aiming to improve the average latency for a targeted subset class constrained by the original accuracy for all classes. Results demonstrate average time saving in the order of 2.09x to 8.79x for dataset CIFAR10 and 15.00x to 20.71x for CIFAR100 for baseline models ResNet-21, ResNet-110, Inceptionv3-159, and DenseNet-121. (C) 2021 Elsevier B.V. All rights reserved. (AU)

• FAPESP's process: 19/05286-6 - Exploration of an FPGA-orientated hardware infrastructure for ultra-low latency DNN deployment
• Grantee: Vanderlei Bonato
• Support Opportunities: Scholarships abroad - Research