Data, depth, and design: Learning reliable models for skin lesion analysis

Autor(es)

Valle, Eduardo ^[1] ; Fornaciali, Michel ^[1] ; Menegola, Afonso ^[1] ; Tavares, Julia ^[1] ; Bittencourt, Flavia Vasques ^[2] ; Li, Lin Tzy ^{[3, 4]} ; Avila, Sandra ^[3]

Afiliação do(s) autor(es)

^[1] Univ Campinas UNICAMP, Sch Elect & Comp Engn, Av Albert Einstein 400, BR-13083852 Campinas, SP - Brazil

^[2] Fed Univ Minas Gerais UFMG, Sch Med, Alamedalvaro Celso 55, BR-30150260 Belo Horizonte, MG - Brazil

^[3] Univ Campinas UNICAMP, Inst Comp, Av Albert Einstein 1251, BR-13083852 Campinas, SP - Brazil

^[4] Samsung R&D Inst Brazil SRBR, Campinas, SP - Brazil

• Tipo de documento: Artigo Científico
• Fonte: Neurocomputing; v. 383, p. 303-313, MAR 28 2020.
• Citações Web of Science: 0

Resumo

Deep learning fostered a leap ahead in automated skin lesion analysis in the last two years. Those models, however, are expensive to train and difficult to parameterize. Objective: We investigate methodological issues for designing and evaluating deep learning models for skin lesion analysis. We explore ten choices faced by researchers: use of transfer learning, model architecture, train dataset, image resolution, type of data augmentation, input normalization, use of segmentation, duration of training, additional use of Support Vector Machines, and test data augmentation. Methods: We perform two full factorial experiments, for five different test datasets, resulting in 2560 exhaustive trials in our main experiment, and 1280 trials in our assessment of transfer learning. We analyze both with multi-way analyses of variance (ANOVA). We use the exhaustive trials to simulate sequential decisions and ensembles, with and without the use of privileged information from the test set. Results main experiment: Amount of train data has disproportionate influence, explaining almost half the variation in performance. Of the other factors, test data augmentation and input resolution are the most influential. Deeper models, when combined, with extra data, also help. - transfer experiment: Transfer learning is critical, its absence brings huge performance penalties. - simulations: Ensembles of models are the best option to provide reliable results with limited resources, without using privileged information and sacrificing methodological rigor. Conclusions and Significance: Advancing research on automated skin lesion analysis requires curating larger public datasets. Indirect use of privileged information from the test set to design the models is a subtle, but frequent methodological mistake that leads to overoptimistic results. Ensembles of models are a cost-effective alternative to the expensive full-factorial and to the unstable sequential designs. (C) 2019 Elsevier B.V. All rights reserved. (AU)

• Processo FAPESP: 19/05018-1 - Triagem Automática de Doenças para o Mundo Real
• Beneficiário: Eduardo Alves Do Valle Junior
• Modalidade de apoio: Bolsas no Exterior - Pesquisa
• Processo FAPESP: 17/16246-0 - Análise de mídias sensíveis usando arquiteturas de aprendizado profundo
• Beneficiário: Sandra Eliza Fontes de Avila
• Modalidade de apoio: Auxílio à Pesquisa - Regular