A DIF-Driven Threshold Tuning Method for Improving Group Fairness

Autor(es)

Minatel, Diego ; Parmezan, Antonio R. S. ; dos Santos, Nicolas Roque ; Curi, Mariana ; Lopes, Alneu de Andrade

• Tipo de documento: Artigo Científico
• Fonte: 40TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING; v. N/A, p. 9-pg., 2025-01-01.

Resumo

To promote social good, current decision support systems based on machine learning must not propagate society's various types of discrimination. Consequently, a desirable behavior for classifiers used in decision-making is that their results do not favor or disadvantage any specific sociodemographic group. One way to achieve this behavior is through post-processing methods, which apply threshold tuning to select the decision boundary that enhances the impartiality of the trained model's decisions. Various strategies have been proposed to determine the optimal threshold, but finding the trade-off between fairness and predictive performance remains challenging. Recently, the application of Differential Item Functioning (DIF) concepts has proven effective for such a purpose in model selection, which is a similar application. This finding makes using DIF in threshold tuning appealing and an unexplored contribution to the literature on fairness in machine learning. This paper addresses this gap and proposes DIF-PP, a novel post-processing method based on DIF. We experimentally evaluated our method against three baselines using 15 datasets, six classification algorithms with 16 settings for each one, four group fairness metrics, one predictive performance measure, one multi-criteria measure, and one statistical significance test. Our experimental results indicate that DIF-PP provides the best trade-off between group fairness metrics and predictive performance, making it the optimal choice for threshold tuning of binary classifiers applied to decision-making tasks involving people. (AU)

• Processo FAPESP: 20/09835-1 - IARA - Inteligência Artificial Recriando Ambientes
• Beneficiário: André Carlos Ponce de Leon Ferreira de Carvalho
• Modalidade de apoio: Auxílio à Pesquisa - Programa Centros de Pesquisa Aplicada
• Processo FAPESP: 22/09091-8 - Criminalidade, insegurança e legitimidade: uma abordagem transdisciplinar
• Beneficiário: Luis Gustavo Nonato
• Modalidade de apoio: Auxílio à Pesquisa - Programa eScience e Data Science - Temático
• Processo FAPESP: 22/02176-8 - Extração de características com aprendizado profundo em cenários de dados e processamento limitados
• Beneficiário: Antonio Rafael Sabino Parmezan
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado