Development of Large Language Models for Applications in the Legal Domain
Graph-based total recall information retrieval on text document corpora
Induction of Topic-Based Bayesian Networks from Text for the Prediction of Sugar C...
Full text | |
Author(s): Show less - |
Alcoforado, Alexandre
;
Ferraz, Thomas Palmeira
;
Gerber, Rodrigo
;
Bustos, Enzo
;
Oliveira, Andre Seidel
;
Veloso, Bruno Miguel
;
Siqueira, Fabio Levy
;
Reali Costa, Anna Helena
;
Pinheiro, V
;
Gamallo, P
;
Amaro, R
;
Scarton, C
;
Batista, F
;
Silva, D
;
Magro, C
;
Pinto, H
Total Authors: 16
|
Document type: | Journal article |
Source: | COMPUTATIONAL PROCESSING OF THE PORTUGUESE LANGUAGE, PROPOR 2022; v. 13208, p. 12-pg., 2022-01-01. |
Abstract | |
Traditional text classification approaches often require a good amount of labeled data, which is difficult to obtain, especially in restricted domains or less widespread languages. This lack of labeled data has led to the rise of low-resource methods, that assume low data availability in natural language processing. Among them, zero-shot learning stands out, which consists of learning a classifier without any previously labeled data. The best results reported with this approach use language models such as Transformers, but fall into two problems: high execution time and inability to handle long texts as input. This paper proposes a new model, ZeroBERTo, which leverages an unsupervised clustering step to obtain a compressed data representation before the classification task. We show that ZeroBERTo has better performance for long inputs and shorter execution time, outperforming XLM-R by about 12% in the F1 score in the FolhaUOL dataset. (AU) | |
FAPESP's process: | 19/07665-4 - Center for Artificial Intelligence |
Grantee: | Fabio Gagliardi Cozman |
Support Opportunities: | Research Grants - Research Program in eScience and Data Science - Research Centers in Engineering Program |