| Full text | |
| Author(s): |
Yang, Jing
;
Rocha, Anderson
Total Authors: 2
|
| Document type: | Journal article |
| Source: | 2024 IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY, WIFS 2024; v. N/A, p. 6-pg., 2024-01-01. |
| Abstract | |
Fact verification is a crucial process in journalism for combating disinformation. Computational methods to aid journalists in the task often require adapting a model to specific domains and generating explanations. However, most automated fact-checking methods rely on three-class datasets, which do not accurately reflect real-world misinformation. Moreover, fact-checking explanations are often generated based on text summarization of evidence, failing to address the relationship between the claim and the evidence. To address these issues, we extend the self-rationalization method-typically used in natural language inference (NLI) tasks-to fact verification. Self-rationalization refers to a model's ability to generate explanations or justifications for its responses, which is essential for reliable fact-checking. We propose a label-adaptive learning approach: first, we fine-tune a model to learn veracity prediction with annotated labels (step-1 model). Then, we fine-tune the step-1 model again to learn self-rationalization, using the same data and additional annotated explanations. This approach allows the model to adapt to a new domain more effectively than fine-tuning end-to-end self-rationalization directly. Our results show that our label-adaptive approach improves veracity prediction by more than ten percentage points (Macro F1) on both the PubHealth and AVeriTec datasets, outperforming the GPT-4 model. Furthermore, to address the high cost of explanation annotation, we generated 64 synthetic explanations from three large language models: GPT-4-turbo, GPT-3.5-turbo, and Llama-3-8B and few-shot fine-tune our step-1 model. The few-shot synthetic explanation fine-tuned model performed comparably to the fully fine-tuned self-rationalization model, demonstrating the potential of low-budget learning with synthetic data. Our label-adaptive self-rationalization approach presents a promising direction for future research on real-world explainable fact-checking with different labeling schemes(1). (AU) | |
| FAPESP's process: | 23/12865-8 - Horus: artificial intelligence techniques to detect and forestall synthetic realities |
| Grantee: | Anderson de Rezende Rocha |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 19/04053-8 - Event reconstruction from heterogeneous visual data |
| Grantee: | Jing Yang |
| Support Opportunities: | Scholarships in Brazil - Doctorate |