Mitigating subjectivity and bias in AI development indices: A robust approach to redefining country rankings

Autor(es)

Campello, Betania Silva Carneiro ; Pelegrina, Guilherme Dean ; Pelissari, Renata ; Suyama, Ricardo ; Duarte, Leonardo Tomazeli

• Tipo de documento: Artigo Científico
• Fonte: EXPERT SYSTEMS WITH APPLICATIONS; v. 255, p. 13-pg., 2024-07-24.

Resumo

Artificial Intelligence (AI) indices have emerged to assess countries' progress in AI development, aiding decision-making on investments and policy choices. Typically, these indices are derived from a linear weighted sum of various criteria, employing deterministic weights. However, this approach fails to capture interactions among criteria, and the use of deterministic weights is susceptible to debate. To mitigate these issues, we conduct a methodological analysis based on Choquet integral (CI) and Stochastic Multicriteria Acceptability Analysis 2 (SMAA-2). We assess correlations between different AI dimensions and employ CI to model them. Additionally, we apply SMAA-2 to conduct a sensitivity analysis using both weighted sum and CI in order to evaluate the stability of the indices with regard to the weights. Finally, we introduce a ranking methodology based on SMAA-2, which considers several sets of weights to derive the ranking of countries. In the computational analysis, we evaluate our approach using the dataset employed in The Global AI Index, as proposed by the British news website Tortoise. The results reveal that our approach effectively mitigates bias. Furthermore, we scrutinize changes in the ranking resulting from weight adjustments and demonstrate that our proposed rankings closely align with those derived from variations in weights, indicating robustness. (AU)

• Processo FAPESP: 23/04159-6 - Desafios em decisões com múltiplos aspectos: integrando técnicas de aprendizado de máquina e da pesquisa operacional
• Beneficiário: Betania Silva Carneiro Campello
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado
• Processo FAPESP: 20/10572-5 - Novas abordagens para lidar com imparcialidade e transparência em problemas de aprendizado de máquina
• Beneficiário: Guilherme Dean Pelegrina
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado
• Processo FAPESP: 20/09838-0 - BI0S - Brazilian Institute of Data Science
• Beneficiário: João Marcos Travassos Romano
• Modalidade de apoio: Auxílio à Pesquisa - Programa Centros de Pesquisa em Engenharia