A Grammar-based multi-objective neuroevolutionary algorithm to generate fully convolutional networks with novel topologies

Author(s)

Miranda, Thiago Z. ; Sardinha, Diorge B. ; Neri, Ferrante ; Basgalupp, Marcio P. ; Cerri, Ricardo

• Document type: Journal article
• Source: APPLIED SOFT COMPUTING; v. 149, p. 12-pg., 2023-10-30.

Abstract

The design of complex and deep neural networks is often performed by identifying and combining building blocks and progressively selecting the most promising combination. Neuroevolution automates this process by employing evolutionary algorithms to guide the search. Within this field, grammar-based evolutionary algorithms have been demonstrated to be powerful tools to describe and thus encode complex neural architectures effectively. Following this trend, the present work proposes a novel grammar-based multi-objective neuroevolutionary for generating Fully Convolutional Networks. The proposed method, named Multi-Objective gRammatical Evolution for FUlly convolutional Networks (MOREFUN), includes a new efficient way to encode skip connections, facilitating the description of complex search spaces and the injection of domain knowledge in the search procedure, generation of fully convolutional networks, upsampling of lower-resolution inputs in multi-input layers, usage of multi-objective fitness, and inclusion of data augmentation and optimiser settings to the grammar. Our best networks outperformed previous grammar evolution algorithms, achieving 90.5% accuracy on CIFAR-10 without using transfer learning, ensembles, or test-time data augmentation. Our best models had 13.39 +/- 5.25 trainable parameters and the evolutionary process required 90 min per generation. (AU)

• FAPESP's process: 22/14098-1 - Neuro-GEMA: a grammar-based evolutionary method to automatically design flexible convolutional neural networks
• Grantee: Márcio Porto Basgalupp
• Support Opportunities: Regular Research Grants
• FAPESP's process: 22/03590-2 - GECCO 2022 - The Genetic and Evolutionary Computation Conference
• Grantee: Márcio Porto Basgalupp
• Support Opportunities: Research Grants - Meeting - Abroad
• FAPESP's process: 20/09835-1 - IARA - Artificial Intelligence in the Remaking of Urban Environments
• Grantee: André Carlos Ponce de Leon Ferreira de Carvalho
• Support Opportunities: Research Grants - Research Centers in Engineering Program
• FAPESP's process: 22/07458-1 - Automatic selection and recommendation of machine learning algorithms
• Grantee: Márcio Porto Basgalupp
• Support Opportunities: Regular Research Grants
• FAPESP's process: 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry
• Grantee: Francisco Louzada Neto
• Support Opportunities: Research Grants - Research, Innovation and Dissemination Centers - RIDC