Neural architecture search with interpretable meta-features and fast predictors

Author(s)

Pereira, Gean T. ; Santos, Iury B. A. ; Garcia, Luis P. F. ; Urruty, Thierry ; Visani, Muriel ; de Carvalho, Andre C. P. L. F.

• Document type: Journal article
• Source: INFORMATION SCIENCES; v. 649, p. 17-pg., 2023-09-13.

Abstract

Neural Architecture Search (NAS) is well-known for automatizing neural architecture design and finding better architectures. Although NAS methods have shown substantial progress over the years, most still suffer from data inefficiency, high model complexity, and lack of interpretability. This paper proposes a solution for these problems by introducing a Prediction-based and interpretable Meta-Learning method called MbML-NAS, capable of generalizing to different search spaces and datasets using less data than several state-of-the-art NAS methods. The proposal uses interpretable meta-features extracted from neural architectures and regression models as meta-predictors to infer Convolutional Networks performances. Experiments compared MbMLNAS with a graph-based Neural Predictor, state-of-the-art NAS methods, a lower bound, and an upper bound Oracle baseline. Furthermore, an interpretability analysis of meta-features and meta-predictors is presented. As a result, using at least 172 examples representing 0.04% and 1.1% of the popular NAS-Bench-101 and NAS-Bench-201 search spaces, MbML-NAS find architectures with better or comparable performances than most baselines, including the Oracle. Moreover, the results showed the potential for using simple meta-features to generalize across NAS search spaces and datasets, encoding neural architectures so that even linear models can accurately predict their performances. Additionally, novel meta-datasets suitable for NAS are proposed to facilitate research on NAS. (AU)

• FAPESP's process: 19/19994-2 - Meta-learning applied to search deep neural network architectures
• Grantee: Gean Trindade Pereira
• Support Opportunities: Scholarships in Brazil - Doctorate
• FAPESP's process: 20/09835-1 - IARA - Artificial Intelligence in the Remaking of Urban Environments
• Grantee: André Carlos Ponce de Leon Ferreira de Carvalho
• Support Opportunities: Research Grants - Research Centers in Engineering Program