Statistical machine learning: toward better uncertainty quantification

• Grant number: 23/07068-1
• Support Opportunities: Regular Research Grants
• Start date: October 01, 2023
• End date: September 30, 2025
• Field of knowledge: Physical Sciences and Mathematics - Probability and Statistics - Statistics
• Principal Investigator: Rafael Izbicki
• Grantee: Rafael Izbicki
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil

Abstract

Machine Learning (ML) and Statistics have emerged as powerful disciplines in the fieldof data analysis, each offering unique perspectives and methodologies for extracting valuable insights from complex datasets. The goal of this work is to investigate how statistics can effectively evaluate theuncertainty of ML methods.The proposal consists of three interconnected aims that address different aspects of uncertainty quantification. Aim 1 focuses on developing scalable prediction intervals with asymptotic conditional coverage based on regression methods. We aim to overcome the limitations of existing methods that either lack coverage guarantees or fail to scale well to higher dimensional feature spaces. Building upon the work of Aim 1, Aim 2 aims to recalibrate full predictive distributions (PDs) to achieve individual or conditional calibration. By assessing and targeting conditional coverage across the entire input feature space, we aim to improve the reliability of PDs and provide instance-wise uncertainties. Finally,Aim 3 expands the scope of uncertainty quantification by focusing on measuring the epistemic uncertainty associated with estimated conditional densities. By developing innovative techniques to quantify uncertainty in conditional density estimation, we enable more robust parameter estimates, predictions, and decision-making processes across various disciplines. (AU)

Scientific publications

(5)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

NAKAZONO, L.; VALENCA, R. R.; SOARES, G.; IZBICKI, R.; IVEZIC, Z.; R LIMA, E., V; HIRATA, N. S. T.; SODRE JR, L.; OVERZIER, R.; ALMEIDA-FERNANDES, F.; et al. The Quasar Catalogue for S-PLUS DR4 (QuCatS) and the estimation of photometric redshifts. Monthly Notices of the Royal Astronomical Society, v. 531, n. 1, p. 13-pg., 2024-05-14. (19/26492-3, 23/07068-1, 21/12744-0, 21/08983-0, 18/20977-2, 19/11321-9, 11/51680-6, 21/09468-1, 23/05003-0, 19/01312-2, 15/22308-2, 22/15304-4)

VALLE, DENIS; LEITE, RODRIGO; IZBICKI, RAFAEL; SILVA, CARLOS; HANEDA, LEO. Local uncertainty maps for land-use/land-cover classification without remote sensing and modeling work using a class-conditional conformal approach. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, v. 135, p. 13-pg., 2024-12-10. (19/11321-9, 23/07068-1)

CABEZAS, LUBEN M. C.; OTTO, MATEUS P.; IZBICKI, RAFAEL; STERN, RAFAEL B.. Regression trees for fast and adaptive prediction intervals. INFORMATION SCIENCES, v. 686, p. 31-pg., 2024-08-27. (23/07068-1, 13/07699-0, 19/11321-9, 22/08579-7, 21/02178-8)

LASSANCE, RODRIGO F. L.; IZBICKI, RAFAEL; STERN, RAFAEL B.. Adding imprecision to hypotheses: A Bayesian framework for testing practical significance in nonparametric settings. INTERNATIONAL JOURNAL OF APPROXIMATE REASONING, v. 178, p. 25-pg., 2024-11-29. (13/07699-0, 19/11321-9, 23/07068-1)

ANDRADE SILVA, JOAO FLAVIO; IZBICKI, RAFAEL; BASTOS, LEONARDO S.; SOARES, GUILHERME P.. Monitoring Viral Infections in Severe Acute Respiratory Syndrome Patients in Brazil. DEVELOPMENTS IN STATISTICAL MODELLING, IWSM 2024, v. N/A, p. 6-pg., 2024-01-01. (23/07068-1)