Research on available tools for testing machine learning-based systems

• Grant number: 25/10837-2
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: August 01, 2025
• End date: July 31, 2026
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Lilian Berton
• Grantee: Samuel Rodela Steidle
• Host Institution: Instituto de Ciência e Tecnologia (ICT). Universidade Federal de São Paulo (UNIFESP). Campus São José dos Campos. São José dos Campos , SP, Brazil
• Company: Secretaria de Desenvolvimento Econômico (São Paulo - Estado). Instituto de Pesquisas Tecnológicas S/A (IPT)
• Associated research grant: 20/09850-0 - Applied Artificial Intelligence Research Center: accelerating the evolution of industries toward standard 5.0, AP.PCPE

Abstract

Most current ML-based systems are still tested in a somewhat unsystematic manner. This can lead to errors in the results of systems that use this technology. For interconnected systems, this is even more concerning, as errors in one system can propagate to others within the supply chain. Despite the limited number of techniques and corresponding metrics in the testing of ML-based systems, there are academic and industry initiatives aimed at developing tools to support their testing. This project will focus on surveying and experimenting with available testing tools. An important aspect of this project will be the experimentation of these tools from the perspectives of robustness and scalability, as they will need to be applied to real-world industry systems. (AU)