DeepPruning: Efficient Neural Networks by Exploring Pruning Techniques.

• Grant number: 23/11163-0
• Support Opportunities: Regular Research Grants
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Artur Jordão Lima Correia
• Grantee: Artur Jordão Lima Correia
• Host Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated researchers: Anna Helena Reali Costa ; Cíntia Borges Margi
• Associated scholarship(s): 24/17684-4 - Data Augmentation through Distilled Samples and Detection of Critical Learning Periods, BP.IC; 23/16871-2 - Efficient and Robust Neural Networks: Analyzing the trade-off between computational cost and adversarial robustness of pruned neural networks, BP.TT

Abstract

Machine learning has driven unprecedented advances toward the automation of different cognitive tasks. Modern machine learning models have embraced deep neural networks (deep learning) as the central paradigm for modeling patterns from data. Despite the positive results, often, neural networks suffer from high computational overload, which imposes several technological, financial and organizational challenges for their development and study. For example, modern neural networks require massive computational infrastructure, which entails significant financial investment and demands high electrical energy consumption. According to previous works, the development of neural networks and complex machine learning models in infrastructures with unsustainable energy sources contributes to carbon emission (CO2) and raises important environmental issues. Due to these concerns, several studies have been conducted to improve the efficiency and environmental impact (Green AI) of neural networks. In this sense, the objectives of this project comprehend the elaboration of solutions to transform computationally prohibitive neural networks into more efficient models and analysis of the obtained models in different hardware configurations as well as scenarios where decision-making plays a critical role. In order to achieve these objectives, this project proposes to develop solutions based on pruning techniques, which identify and remove components (neurons and/or layers) from a neural network, preserving compromises between sustainability, efficiency and predictive performance. Among the expected contributions, we highlight the applicability of neural networks to scenarios devoid of massive computing infrastructure and with limited computational resources, such as universities and small research centers. Additionally, the project highlights where we are assessing and incorporating sustainability into AI. (AU)