Assessing Adversarial Effects of Noise in Missing Data Imputation

Autor(es)

Mangussi, Arthur Dantas ; Pereira, Ricardo Cardoso ; Abreu, Pedro Henriques ; Lorena, Ana Carolina

• Tipo de documento: Artigo Científico
• Fonte: INTELLIGENT SYSTEMS, BRACIS 2024, PT I; v. 15412, p. 15-pg., 2025-01-01.

Resumo

In real-world scenarios, a wide variety of datasets contain inconsistencies. One example of such inconsistency is missing data (MD), which refers to the absence of information in one or more variables. Missing imputation strategies emerged as a possible solution for addressing this problem, which can replace the missing values based on mean, median, or Machine Learning (ML) techniques. The performance of such strategies depends on multiple factors. One factor that influences the missing value imputation (MVI) methods is the presence of noisy instances, described as anything that obscures the relationship between the features of an instance and its class, having an adversarial effect. However, the interaction between MD and noisy instances has received little attention in the literature. This work fills this gap by investigating missing and noisy data interplay. Our experimental setup begins with generating missingness under the Missing Not at Random (MNAR) mechanism in a multivariate scenario and performing imputation using seven state-of-the-art MVI methods. Our methodology involves applying a noise filter before performing the imputation task and evaluating the quality of the imputation directly. Additionally, we measure the classification performance with the new estimates. This approach is applied to both synthetic data and 11 real-world datasets. The effects of noise filtering before imputation are evaluated. The results show that noise preprocessing before the imputation task improves the imputation quality and the classification performance for imputed datasets. (AU)

• Processo FAPESP: 21/06870-3 - Além da seleção de algoritmos: meta-aprendizado para análise e entendimento de dados e algoritmos
• Beneficiário: Ana Carolina Lorena
• Modalidade de apoio: Auxílio à Pesquisa - Jovens Pesquisadores - Fase 2
• Processo FAPESP: 23/13688-2 - An Autoencoder model for dealing with missing and noise data
• Beneficiário: Arthur Dantas Mangussi
• Modalidade de apoio: Bolsas no Exterior - Estágio de Pesquisa - Mestrado