Data Governance for Data Science Analysis and Processing Environments

• Grant number: 25/06891-1
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: June 01, 2025
• End date: May 31, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Paulo Roberto Miranda Meirelles
• Grantee: Pedro Cruz Martins
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated research grant: 23/18026-8 - Center for Data Science in Public Statistics, AP.CCD

Abstract

In recent years, the adoption of data-driven strategies to support decision-making and product development has become a central practice in innovation-oriented organizations. This data is typically characterized by high volume, generation speed, and heterogeneity, requiring specialized infrastructure, mature operational practices, and robust governance mechanisms to ensure scalability, operational efficiency, and sustainable use of information. However, centralized architectures, such as data lakes and data warehouses, present structural limitations in accommodating the increasing complexity of data domains and the diversity of sources involved-especially in distributed scenarios. In this context, the proposed initiative envisions the development of a decentralized data platform based on the principles of Data Mesh, aiming to provide greater domain autonomy, system interoperability, and federated governance. The undergraduate research intern will collaborate in the development and modernization of the extraction, transformation, and loading (ETL) pipelines used in the project. Their participation will have a formative character, focusing on the critical analysis of the solutions adopted by Fundação SEADE and the study of new technologies aligned with the project's objectives. This experience will offer the intern an environment conducive to developing advanced technical and analytical skills, while engaging with real-world problems of high relevance to society. (AU)