Knowledge Distillation for Time Series Models

• Grant number: 24/14856-9
• Support Opportunities: Scholarships in Brazil - Master
• Start date: October 01, 2024
• End date: September 30, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Diego Furtado Silva
• Grantee: Adilson Junior Alves Medronha
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 22/03176-1 - Machine learning for time series obtained in mHealth applications, AP.PNGP.PI
• Associated scholarship(s): 25/18189-0 - Lightweight Time Series Classification Models via Pruning, BE.EP.MS

Abstract

Deep neural networks require high computational costs, especially in mHealth applications with limited hardware. The project proposes to explore knowledge distillation in time series classification tasks. This technique involves transferring the learning from a larger network (teacher) to a smaller one (student), which is trained to reproduce similar outputs more efficiently. This approach still needs to be explored in time series classification tasks.