Mutation-based software testing with high efficiency and low technical debt: automated process and free support environment prototype

• Grant number: 19/23160-0
• Support Opportunities: Regular Research Grants
• Start date: February 01, 2022
• End date: January 31, 2025
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Auri Marcelo Rizzo Vincenzi
• Grantee: Auri Marcelo Rizzo Vincenzi
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil
• Associated researchers: Ana Cristina Ramada Paiva ; Anderson Belgamo ; Delano Medeiros Beder ; Fabiano Cutigi Ferrari ; Jose Carlos Maldonado ; Márcio Eduardo Delamaro ; Rodrigo Elias Bianchi ; Rodrigo Funabashi Jorge
• Associated scholarship(s): 23/08409-7 - Study, implementation, and validation of test environment, BP.TT

Abstract

Considering the relevance of automating testing activities for the software product industry and the relevance of mutation-based testing from scientific evidence, this project aims to define a mutation-based testing process that can be performed in a 100% automated manner through a supportive test environment, specified, and validated with industrial and open-source applications. Methods: Investigate and automate three classic mutation test problems: 1) mutant generation; 2) execution of mutants, and 3) analysis of live and equivalent mutants in the test process. In the generation, it is intended to use static analysis and control and data flow information to select the points that the mutations shall be performed, in addition to selecting specific types of operators to be used depending on the characteristics of the product under test. In the execution of mutants, the selection of good test cases with a high probability of killing non-equivalent mutants is very important. To do this, it is known that different automatic test data generators must be combined and, possibly, new generation algorithms must be developed to kill mutants generated by specific mutation operators. In the analysis of living and equivalent mutants, we need to define and evaluate automated strategies for the determination of equivalent mutants, using heuristics, Bayesian learning, and the frequency of execution of the mutants by the test cases. Expected results: a free testing process and a support test environment, as well as experimentation data that allows the generation of benchmarks for testing C, Java, and Python, as well as for the development of new research related to the mutation test. Process and Environment will be applied and evaluated in industrial partners which are formally supporting the present project with interest in its results to be used on production. Technical debt in the context of this proposal will be conducted to characterize the risks associated with the production and release of software products based on mutation testing criterion. Conclusions: It is intended to enable the application of mutation testing as an important mechanism for ensuring the quality of software products in a fully automated way, favoring the technological transfer and application of the mutation test to the industry, with the consequent evolution of the software production capacity of the Brazilian software industry. (AU)

Scientific publications

(6)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

KUROISHI, PEDRO HENRIQUE; PAIVA, ANA CRISTINA RAMADA; MALDONADO, JOSE CARLOS; VINCENZI, AURI MARCELO RIZZO. Testing infrastructures to support mobile application testing: A systematic mapping study. INFORMATION AND SOFTWARE TECHNOLOGY, v. 177, p. 19-pg., 2024-09-13. (23/00001-9, 19/23160-0)

GUERINO, LUCCA R.; VINCENZI, AURI M. R.. An Experimental Study Evaluating Cost, Adequacy, and Effectiveness of Pynguin's Test Sets. PROCEEDINGS OF THE 8TH BRAZILIAN SYMPOSIUM ON SYSTEMATIC AND AUTOMATED SOFT-WARE TESTING, SAST 2023, v. N/A, p. 10-pg., 2023-01-01. (19/23160-0)

GUILHERME, VITOR H.; VINCENZI, AURI M. R.. An initial investigation of ChatGPT unit test generation capability. PROCEEDINGS OF THE 8TH BRAZILIAN SYMPOSIUM ON SYSTEMATIC AND AUTOMATED SOFT-WARE TESTING, SAST 2023, v. N/A, p. 10-pg., 2023-01-01. (19/23160-0)

COSTA MEIRELES, MARIA ALCIMAR; ROCHA, SABRINA; MALDONADO, JOSE CARLOS; CONTE, TAYANA. An experience report on the use of Active Learning in Empirical Software Engineering Education: Understanding the pros and cons from the student's perspective. 2024 ACM/IEEE 44TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: SOFTWARE ENGINEERING EDUCATION AND TRAINING, ICSE-SEET 2024, v. N/A, p. 11-pg., 2024-01-01. (20/05191-2, 19/23160-0)

KUROISHI, PEDRO HENRIQUE; MALDONADO, JOSE CARLOS; VINCENZI, AURI MARCELO RIZZO. Towards the definition of a research agenda on mobile application testing based on a tertiary study. INFORMATION AND SOFTWARE TECHNOLOGY, v. 167, p. 22-pg., 2023-12-06. (23/00001-9, 19/23160-0)

FERNANDES, LEO; RIBEIRO, MARCIO; GHEYI, ROHIT; DELAMARO, MARCIO; GUIMARAES, MARCIO; SANTOS, ANDRE. Put Your Hands In The Air! Reducing Manual Effort in Mutation Testing. 36TH BRAZILIAN SYMPOSIUM ON SOFTWARE ENGINEERING, SBES 2022, v. N/A, p. 10-pg., 2022-01-01. (20/09560-2, 19/23160-0)