Clinicians' perception of oral potentially malignant disorders: a pitfall for image annotation in supervised learning

Autor(es):Mostrar menos -

Araujo, Anna Luiza Damaceno ; de Souza, Eduardo Santos Carlos ; Faustino, Isabel Schausltz Pereira ; Saldivia-Siracusa, Cristina ; Brito-Sarracino, Tamires ; Adudurate Lopes, Marcio ; Vargas, Pablo Agustin ; Pearson, Alexander T. ; Kowalski, Luiz Paulo ; de Carvalho, Andre Carlos Ponce de Leon Ferreira ; Santos-Silva, Alan Roger

• Tipo de documento: Artigo Científico
• Fonte: ORAL SURGERY ORAL MEDICINE ORAL PATHOLOGY ORAL RADIOLOGY; v. 136, n. 3, p. 7-pg., 2023-09-01.

Resumo

Objective. The present study aims to quantify clinicians' perceptions of oral potentially malignant disorders (OPMDs) when evaluating, classifying, and manually annotating clinical images, as well as to understand the source of inter-observer variability when assessing these lesions. The hypothesis was that different interpretations could affect the quality of the annotations used to train a Supervised Learning model.Study design. Forty-six clinical images from 37 patients were reviewed, classified, and manually annotated at the pixel level by 3 labelers. We compared the inter-examiner assessment based on clinical criteria through the K statistics (Fleiss's kappa). The segmentations were also compared using the mean pixel-wise intersection over union (IoU). Results. The inter-observer agreement for homogeneous/non-homogeneous criteria was substantial (K = 63, 95% CI: 0.47-0.80). For the subclassification of non-homogeneous lesions, the inter-observer agreement was moderate (K = 43, 95% CI: 0.34-0.53) (P < .001). The mean IoU of 0.53 (0.22) was considered low.Conclusion. The subjective clinical assessment (based on human visual observation, variable criteria that have suffered adjustments over the years, different educational backgrounds, and personal experience) may explain the source of inter-observer discordance for the classification and annotation of OPMD. Therefore, there is a strong probability of transferring the subjectivity of human analysis to artificial intelligence models. The use of large data sets and segmentation based on the union of all labelers' annotations holds the potential to overcome this limitation. (Oral Surg Oral Med Oral Pathol Oral Radiol 2023;136:315-321) (AU)

• Processo FAPESP: 22/07276-0 - Preditores clinicopatológicos e digitais de recorrência e malignização da leucoplasia oral e da leucoplasia verrucosa proliferativa: um estudo clínico adjunto ao uso da inteligência artificial
• Beneficiário: Caique Mariano Pedroso
• Modalidade de apoio: Bolsas no Brasil - Doutorado
• Processo FAPESP: 21/14585-7 - Inteligência artificial aplicada ao diagnóstico clínico e histopatológico do Câncer de Cabeça e Pescoço
• Beneficiário: Anna Luiza Damaceno Araujo
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado
• Processo FAPESP: 09/53839-2 - Criação do Laboratório de Patologia Digital através do uso do escaneador de lâminas histológicas (Aperio® Scanscope CS)
• Beneficiário: Oslei Paes de Almeida
• Modalidade de apoio: Auxílio à Pesquisa - Programa Equipamentos Multiusuários