Statistical Machine Learning - Toward Better Uncertainty Quantification

• Grant number: 23/07068-1
• Support Opportunities: Regular Research Grants
• Start date: October 01, 2023
• End date: September 30, 2025
• Field of knowledge: Physical Sciences and Mathematics - Probability and Statistics - Statistics
• Principal Investigator: Rafael Izbicki
• Grantee: Rafael Izbicki
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil
• City of the host institution: São Carlos

Abstract

Machine Learning (ML) and Statistics have emerged as powerful disciplines in the fieldof data analysis, each offering unique perspectives and methodologies for extracting valuable insights from complex datasets. The goal of this work is to investigate how statistics can effectively evaluate theuncertainty of ML methods.The proposal consists of three interconnected aims that address different aspects of uncertainty quantification. Aim 1 focuses on developing scalable prediction intervals with asymptotic conditional coverage based on regression methods. We aim to overcome the limitations of existing methods that either lack coverage guarantees or fail to scale well to higher dimensional feature spaces. Building upon the work of Aim 1, Aim 2 aims to recalibrate full predictive distributions (PDs) to achieve individual or conditional calibration. By assessing and targeting conditional coverage across the entire input feature space, we aim to improve the reliability of PDs and provide instance-wise uncertainties. Finally,Aim 3 expands the scope of uncertainty quantification by focusing on measuring the epistemic uncertainty associated with estimated conditional densities. By developing innovative techniques to quantify uncertainty in conditional density estimation, we enable more robust parameter estimates, predictions, and decision-making processes across various disciplines. (AU)

Scientific publications

(8)

(The scientific publications listed on this page originate from the Web of Science or SciELO databases. Their authors have cited FAPESP grant or fellowship project numbers awarded to Principal Investigators or Fellowship Recipients, whether or not they are among the authors. This information is collected automatically and retrieved directly from those bibliometric databases.)

LASSANCE, RODRIGO F. L.; IZBICKI, RAFAEL; STERN, RAFAEL B.. Adding imprecision to hypotheses: A Bayesian framework for testing practical significance in nonparametric settings. INTERNATIONAL JOURNAL OF APPROXIMATE REASONING, v. 178, p. 25-pg., 2024-11-29. (13/07699-0, 19/11321-9, 23/07068-1)

CABEZAS, LUBEN M. C.; SANTOS, VAGNER S.; RAMOS, THIAGO R.; IZBICKI, RAFAEL. Epistemic Uncertainty in Conformal Scores: A Unified Approach. NEURIPS WORKSHOPS, 2020, v. 286, p. 28-pg., 2025-01-01. (23/07068-1, 22/08579-7, 19/11321-9, 23/05587-1)

ANDRADE SILVA, JOAO FLAVIO; IZBICKI, RAFAEL; BASTOS, LEONARDO S.; SOARES, GUILHERME P.. Monitoring Viral Infections in Severe Acute Respiratory Syndrome Patients in Brazil. DEVELOPMENTS IN STATISTICAL MODELLING, IWSM 2024, v. N/A, p. 6-pg., 2024-01-01. (23/07068-1)

XIAO, YANG; SOARES, GUILHERME; BASTOS, LEONARDO; IZBICKI, RAFAEL; MORAGA, PAULA. Dengue nowcasting in Brazil by combining official surveillance data and Google Trends information. PLoS Neglected Tropical Diseases, v. 19, n. 8, p. 21-pg., 2025-08-01. (23/07068-1)

FROHLICH, ALEK; RAMOS, THIAGO; CABELLO DOS SANTOS, GUSTAVO MOTTA; CARLOTTI BUZATTO, ISABELA PANZERI; IZBICKI, RAFAEL; TIEZZI, DANIEL GUIMARAES. PersonalizedUS: Interpretable Breast Cancer Risk Assessment with Local Coverage Uncertainty Quantification. THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 20, v. N/A, p. 9-pg., 2025-01-01. (19/11321-9, 23/07068-1)

NAKAZONO, L.; VALENCA, R. R.; SOARES, G.; IZBICKI, R.; IVEZIC, Z.; R LIMA, E., V; HIRATA, N. S. T.; SODRE JR, L.; OVERZIER, R.; ALMEIDA-FERNANDES, F.; et al. The Quasar Catalogue for S-PLUS DR4 (QuCatS) and the estimation of photometric redshifts. Monthly Notices of the Royal Astronomical Society, v. 531, n. 1, p. 13-pg., 2024-05-14. (19/26492-3, 23/07068-1, 21/12744-0, 21/08983-0, 18/20977-2, 19/11321-9, 11/51680-6, 21/09468-1, 23/05003-0, 19/01312-2, 15/22308-2, 22/15304-4)

VALLE, DENIS; LEITE, RODRIGO; IZBICKI, RAFAEL; SILVA, CARLOS; HANEDA, LEO. Local uncertainty maps for land-use/land-cover classification without remote sensing and modeling work using a class-conditional conformal approach. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, v. 135, p. 13-pg., 2024-12-10. (19/11321-9, 23/07068-1)

CABEZAS, LUBEN M. C.; OTTO, MATEUS P.; IZBICKI, RAFAEL; STERN, RAFAEL B.. Regression trees for fast and adaptive prediction intervals. INFORMATION SCIENCES, v. 686, p. 31-pg., 2024-08-27. (23/07068-1, 13/07699-0, 19/11321-9, 22/08579-7, 21/02178-8)