Legal Document-Based, Domain-Driven Q&A System: LLMs in Perspective

Author(s)

do Espirito Santo, Felipe Oliveira ; Peres, Sarajane Marques ; Gramachot, Givanildo de Sousa ; Franco Brandaot, Anarosa Alves ; Cozmant, Fabio Gagliardi

• Document type: Journal article
• Source: 2024 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN 2024; v. N/A, p. 9-pg., 2024-01-01.

Abstract

Question Answering systems based on large language models are widely employed today, benefiting from continuous enhancements and improved performance. The legal domain has become a particularly active focus for Question Answering systems, given its complexity and social importance. This paper offers a discussion on how larger and smaller language models can be used to build a legal document-based Question Answering system. We present a novel model, named Cocoruta, generated by fine-tuning with a corpus of legal documents. In addition, we examine five LLMs as they answer questions related to the legal aspects of a specific domain - the Blue Amazon, a region of particular interest involving environmental issues. The results suggest that while LLMs are not yet of sufficient quality for use as core in legal context Question Answering systems, fine-tuning on specialized corpora imparts a beneficial bias to their legal discourse. Despite having fewer parameters, the Cocoruta model competes well with larger LLMs in this aspect. (AU)

• FAPESP's process: 19/07665-4 - Center for Artificial Intelligence
• Grantee: Fabio Gagliardi Cozman
• Support Opportunities: Research Grants - Research Program in eScience and Data Science - Research Centers in Engineering Program