| Texto completo | |
| Autor(es): |
Braz, Leodecio
;
Teixeira, Vinicius
;
Pedrini, Helio
;
Dias, Zanoni
;
Setubal, JC
;
Silva, WM
Número total de Autores: 6
|
| Tipo de documento: | Artigo Científico |
| Fonte: | ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2020; v. 12558, p. 12-pg., 2020-01-01. |
| Resumo | |
Deep learning techniques have been increasingly applied to provide more accurate results in the classification of medical images and in the classification and generation of report texts. The main objective of this paper is to investigate the influence of fusing several features of heterogeneous modalities to improve musculoskeletal abnormality detection in comparison with the individual results of image and text classification. In this work, we propose a novel image-text classification framework, named ImTeNet, to learn relevant features from image and text information for binary classification of musculoskeletal radiography. Initially, we use a caption generator model to artificially create textual data for a dataset lacking text information. Then, we apply the ImTeNet, a multi-modal information model that consists of two distinct networks, DenseNet-169 and BERT, to perform image and text classification tasks respectively, and a fusion module that receives a concatenation of feature vectors extracted from both. To evaluate our proposed approach, we used the Musculoskeletal Radiographs (MURA) dataset and compare the results obtained with image and text classification scheme individually. (AU) | |
| Processo FAPESP: | 19/20875-8 - Classificação de imagens de radiografias de tórax utilizando redes neurais profundas |
| Beneficiário: | Vinicius Teixeira de Melo |
| Modalidade de apoio: | Bolsas no Brasil - Mestrado |
| Processo FAPESP: | 17/12646-3 - Déjà vu: coerência temporal, espacial e de caracterização de dados heterogêneos para análise e interpretação de integridade |
| Beneficiário: | Anderson de Rezende Rocha |
| Modalidade de apoio: | Auxílio à Pesquisa - Temático |
| Processo FAPESP: | 15/11937-9 - Investigação de problemas difíceis do ponto de vista algorítmico e estrutural |
| Beneficiário: | Flávio Keidi Miyazawa |
| Modalidade de apoio: | Auxílio à Pesquisa - Temático |
| Processo FAPESP: | 17/16246-0 - Análise de mídias sensíveis usando arquiteturas de aprendizado profundo |
| Beneficiário: | Sandra Eliza Fontes de Avila |
| Modalidade de apoio: | Auxílio à Pesquisa - Regular |