Small Language Models for Question Answering in Medical Databases

• Grant number: 25/06561-1
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: September 01, 2025
• End date: August 31, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: João Paulo Papa
• Grantee: Ana Lara Alves Garcia
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil

Abstract

In recent years, artificial intelligence (AI) has advanced significantly, driven by the development of hardware with greater processing capacity and the consequent evolution of Natural Language Processing (NLP) models. In the medical context, although Large Language Models (LLMs) demonstrate considerable potential, their implementation faces challenges such as judgment biases, limitations in accuracy, coherence, transparency, interpretation, ethical issues, and the need for large computational resources. As a promising alternative, Small Language Models (SLMs) offer similar performance with a considerably smaller number of parameters. This project proposes a comparative analysis between different LLMs and SLMs in the Question Answering (QA) task in medical databases in Brazilian Portuguese, evaluating effectiveness and efficiency through benchmarks developed from Revalida tests and information from Brazilian medical societies. Techniques such as Retrieval-Augmented Generation (RAG), Zero-Shot Learning, Few-Shot Learning and Fine-Tuning will be explored with approaches such as LoRA and QLoRA. The central hypothesis is that there is no single ideal model for all medical applications, considering the cost-benefit relationship between effectiveness and efficiency. It is expected that the results of this study will contribute to the adaptation and implementation of these technologies in clinical practice in Brazil, enabling significant advances in AI-based healthcare.