Statistical machine learning: toward better uncertainty quantification

• Grant number: 23/07068-1
• Support Opportunities: Regular Research Grants
• Start date: October 01, 2023
• End date: September 30, 2025
• Field of knowledge: Physical Sciences and Mathematics - Probability and Statistics - Statistics
• Principal Investigator: Rafael Izbicki
• Grantee: Rafael Izbicki
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil

Abstract

Machine Learning (ML) and Statistics have emerged as powerful disciplines in the fieldof data analysis, each offering unique perspectives and methodologies for extracting valuable insights from complex datasets. The goal of this work is to investigate how statistics can effectively evaluate theuncertainty of ML methods.The proposal consists of three interconnected aims that address different aspects of uncertainty quantification. Aim 1 focuses on developing scalable prediction intervals with asymptotic conditional coverage based on regression methods. We aim to overcome the limitations of existing methods that either lack coverage guarantees or fail to scale well to higher dimensional feature spaces. Building upon the work of Aim 1, Aim 2 aims to recalibrate full predictive distributions (PDs) to achieve individual or conditional calibration. By assessing and targeting conditional coverage across the entire input feature space, we aim to improve the reliability of PDs and provide instance-wise uncertainties. Finally,Aim 3 expands the scope of uncertainty quantification by focusing on measuring the epistemic uncertainty associated with estimated conditional densities. By developing innovative techniques to quantify uncertainty in conditional density estimation, we enable more robust parameter estimates, predictions, and decision-making processes across various disciplines. (AU)