Deep bidirectional transformers for discovering latent knowledge in medical papers on Acute Myeloid Leukemia

• Grant number: 22/07236-9
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: September 01, 2022
• End date: December 10, 2022
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Tiago Agostinho de Almeida
• Grantee: Matheus Vargas Volpon Berto
• Supervisor: Carolina Evaristo Scarton
• Host Institution: Centro de Ciências em Gestão e Tecnologia (CCGT). Universidade Federal de São Carlos (UFSCAR). Campus de Sorocaba. Sorocaba , SP, Brazil
• Institution abroad: University of Sheffield, England
• Associated to the scholarship: 21/13054-8 - Discovering latent knowledge in medical papers on acute myeloid Leukemia, BP.IC

Abstract

The volume of information produced and accessed through the Internet is large and growing. Consequently, the amount of information available in scientific papers follows the same trend, making manual analysis of all existing content unfeasible. Several strategies and architectures for artificial neural networks have emerged to represent texts using dense vectors, called word vectors. These techniques are continually evolving and are capable of processing increasingly more significant sets of texts with less computational resources. They allow for training huge text representation models for specific knowledge domains, as is the case of PubMedBERT. Using a large corpus of a single area, the model can better capture the relationships between words. When creating representation models from prefaces of scientific papers in materials science, Tshitoyan et al. (2019) observed that the knowledge of certain relationships between elements was latent. Furthermore, they demonstrated that some relationships found in the texts were discovered only years ahead. In this context, this research project proposes to train word vectors from scientific papers about a specific disease to capture and analyze whether it is possible to obtain latent knowledge that can accelerate the discovery of new diagnoses, prognoses, and treatments. (AU)