Legal Document-Based, Domain-Driven Q&A System: LLMs in Perspective

Autor(es)

do Espirito Santo, Felipe Oliveira ; Peres, Sarajane Marques ; Gramachot, Givanildo de Sousa ; Franco Brandaot, Anarosa Alves ; Cozmant, Fabio Gagliardi

• Tipo de documento: Artigo Científico
• Fonte: 2024 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN 2024; v. N/A, p. 9-pg., 2024-01-01.

Resumo

Question Answering systems based on large language models are widely employed today, benefiting from continuous enhancements and improved performance. The legal domain has become a particularly active focus for Question Answering systems, given its complexity and social importance. This paper offers a discussion on how larger and smaller language models can be used to build a legal document-based Question Answering system. We present a novel model, named Cocoruta, generated by fine-tuning with a corpus of legal documents. In addition, we examine five LLMs as they answer questions related to the legal aspects of a specific domain - the Blue Amazon, a region of particular interest involving environmental issues. The results suggest that while LLMs are not yet of sufficient quality for use as core in legal context Question Answering systems, fine-tuning on specialized corpora imparts a beneficial bias to their legal discourse. Despite having fewer parameters, the Cocoruta model competes well with larger LLMs in this aspect. (AU)

• Processo FAPESP: 19/07665-4 - Centro de Inteligência Artificial
• Beneficiário: Fabio Gagliardi Cozman
• Modalidade de apoio: Auxílio à Pesquisa - Programa eScience e Data Science - Centros de Pesquisa em Engenharia