Unravelling the building blocks of deep learning

• Grant number: 19/26617-0
• Support Opportunities: Scholarships abroad - Research Internship - Post-doctor
• Effective date (Start): January 11, 2021
• Effective date (End): September 29, 2021
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: André Carlos Ponce de Leon Ferreira de Carvalho
• Grantee: Tiago Botari
• Supervisor: Klaus-Robert Muller
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Research place: Technical University of Berlin (TU), Germany
• Associated to the scholarship: 17/06161-7 - Interpretability of deep networks, BP.PD

Abstract

Deep learning (DL), a class of machine learning algorithms, has demonstrated a high capacity to induce models in different knowledge domains. DL models are today considered state of the art in many applications, such as image and voice recognition, simulation of game players, among others. While DL models have presented high predictive accuracy in many real-world tasks, little is understood regarding the internal working mechanism and decision-making process performed by the models. Transparency became one of the main issues in DL models since precision is not the only target pursued by humans when solving problems. For this end, it is fundamental today to advance in the understanding of DL models. The objective of this project is to investigate, create and validate interpretations for DL models. For that, we will identify the intern structures in the DL models that are responsible to represent specific data abstractions and use this knowledge to increase transparency. Using data generated from controlled models, we will induce DL models with different levels of complexity to investigate of a specific feature. These controlled models will be advent from statistical physics, symmetry group transformations, among others. The induced DL model will be analysed using methodologies stemming from graph theory, information theory, statistical physic, among others. We will take advantage as well of new methodologies from DL research community. The results produced during this project can lead to advances in the understanding, learning process, increase transparency and interpretability of DL models. Moreover, the development of new strategies for transfer learning and definition of neural network architecture. (AU)