| Texto completo | |
| Autor(es): |
Pirozelli, Paulo
;
Jose, Marcos M.
;
Filho, Paulo de Tarso P.
;
Brandao, Anarosa A. F.
;
Cozman, Fabio G.
Número total de Autores: 5
|
| Tipo de documento: | Artigo Científico |
| Fonte: | NEURAL-SYMBOLIC LEARNING AND REASONING, PT I, NESY 2024; v. 14979, p. 18-pg., 2024-01-01. |
| Resumo | |
Transformer models have shown impressive abilities in natural language tasks such as text generation and question answering. Still, it is not clear whether these models can successfully conduct a rule-guided task such as logical reasoning. In this paper, we investigate the extent to which encoder-only transformer language models (LMs) can reason according to logical rules. We ask whether these LMs can deduce theorems in propositional calculus and first-order logic, if their relative success in these problems reflects general logical capabilities, and which layers contribute the most to the task. First, we show for several encoder-only LMs that they can be trained, to a reasonable degree, to determine logical validity on various datasets. Next, by cross-probing fine-tuned models on these datasets, we show that LMs have difficulty in transferring their putative logical reasoning ability, which suggests that they may have learned dataset-specific features instead of a general capability. Finally, we conduct a layerwise probing experiment, which shows that the hypothesis classification task is mostly solved through higher layers. (AU) | |
| Processo FAPESP: | 19/07665-4 - Centro de Inteligência Artificial |
| Beneficiário: | Fabio Gagliardi Cozman |
| Modalidade de apoio: | Auxílio à Pesquisa - Programa eScience e Data Science - Centros de Pesquisa Aplicada |