Put Your Hands In The Air! Reducing Manual Effort in Mutation Testing

Author(s)

Fernandes, Leo ; Ribeiro, Marcio ; Gheyi, Rohit ; Delamaro, Marcio ; Guimaraes, Marcio ; Santos, Andre

• Document type: Journal article
• Source: 36TH BRAZILIAN SYMPOSIUM ON SOFTWARE ENGINEERING, SBES 2022; v. N/A, p. 10-pg., 2022-01-01.

Abstract

Mutation testing has attracted a lot of interest because of its reputation as a powerful adequacy criterion for test suites and for its ability to guide the test case generation. However, the presence of equivalent mutants hinders its usage in industry. The Equivalent Mutant Problem has already been proven undecidable, but manually detecting equivalent mutants is an error-prone and time-consuming task. Thus, partial solutions can help to reduce this cost. To minimize this problem, we introduce an approach to suggest equivalent mutants. Our approach is based on automated behavioral testing, which consists of test cases based on the behavior of the original program. We perform static analysis to automatically generate tests for the entities impacted by the mutation. For each mutant analyzed, our approach can suggest the mutant as equivalent or non-equivalent. In the case of non-equivalent mutants, our approach provides the test case capable of killing it. For the equivalent mutants suggested, we also provide a ranking of mutants with a strong or weak chance of the mutant being indeed equivalent. To evaluate the approach, we execute it against a set of 1,542 mutants manually classified in previous work as equivalents and non-equivalents. We notice that the approach is effective in suggesting equivalent mutants, reaching more than 96% of accuracy in five out of eight subjects studied. Compared with manual analysis of the surviving mutants, our approach takes a third of the time to suggest equivalents and is 25 times faster to indicate non-equivalents. (AU)

• FAPESP's process: 20/09560-2 - Development of a methodology for the definition of software appraisal proficiency tests
• Grantee: Simone Do Rocio Senger de Souza
• Support Opportunities: Regular Research Grants
• FAPESP's process: 19/23160-0 - Mutation-based software testing with high efficiency and low technical debt: automated process and free support environment prototype
• Grantee: Auri Marcelo Rizzo Vincenzi
• Support Opportunities: Regular Research Grants