Swapping training optimizers and tiny partial datasets to improve performance of lighter neural networks for edge devices

Autor(es)

Nascimento, Alexandre M. ; de Melo, Vinicius V. ; Basgalupp, Marcio P.

• Tipo de documento: Artigo Científico
• Fonte: 38TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, SAC 2023; v. N/A, p. 8-pg., 2023-01-01.

Resumo

Most strategies to port ANN on edge devices train large ANN with large datasets on resourceful processors and shrink them (by pruning or using quantization) to fit them on less powerful processors. Those shrunk ANN still usually demand a considerable processor and has lower accuracy. That restricts applications on widely adopted and cheaper 8-bit/16-bit microcontrollers. Also, large datasets make ANN retraining on edge devices unfeasible, shifting it to a cloud environment that makes user experience susceptible to connectivity latencies, restricting some real-time applications. Here, strategies are proposed to train and improve small ANN accuracy by swapping training optimizers and tiny datasets. Around 330,000 ANNs were trained with tiny fractions (<1%) of the MNIST dataset and validated with its complete testing set. Genetic algorithm and tournament heuristics were used to search for the best optimizers combination. Results demonstrated that optimizers combination could improve ANN average accuracy.Moreover, it can achieve similar accuracy with smaller datasets.Among eight optimizers, the top ANNs used most frequently a combination of Adam, Adadelta, and Adamax. This study has potential implications by finding lighter and better ANNs compatible with less powerful edge devices and indicating a research agenda on mixing optimizers for ANN training. (AU)

• Processo FAPESP: 20/09835-1 - IARA - Inteligência Artificial Recriando Ambientes
• Beneficiário: André Carlos Ponce de Leon Ferreira de Carvalho
• Modalidade de apoio: Auxílio à Pesquisa - Programa Centros de Pesquisa em Engenharia
• Processo FAPESP: 22/07458-1 - Construção e seleção automática de algoritmos de aprendizado de máquina
• Beneficiário: Márcio Porto Basgalupp
• Modalidade de apoio: Auxílio à Pesquisa - Regular