Explanation of Embeddings Produced by Language Models

• Grant number: 25/10383-1
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: July 01, 2025
• End date: June 30, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Luis Gustavo Nonato
• Grantee: Lucas Greff Meneses
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 22/09091-8 - Criminality, insecurity, and legitimacy: a transdisciplinary approach, AP.ESCIENCE.TEM

Abstract

Large-scale language models (LLMs) have become central tools for artificial intelligence due to their ability to produce cohesive texts, answer questions accurately, and perform complex tasks with near-human performance. These achievements stem from high-dimensional embeddings that encode semantic, syntactic, and contextual information, but whose interpretation remains difficult.To address this issue, this project will investigate how the internal transformations of Transformer blocks can be locally approximated by Jacobian matrices, a technique already tested by the proposer and his advisor for the purpose of explaining dimensionality reduction techniques. By applying this approach to embeddings of different layers, we aim to generate quantitative explanations that highlight the most relevant dimensions and reveal internal relationships, offering interpretable tools for future analyses and ethical use of LLMs. (AU)