Empirical research on concurrent software testing: A systematic mapping study

Autor(es)

Melo, Silvana M. ^[1] ; Carver, Jeffrey C. ^[2] ; Souza, Paulo S. L. ^[1] ; Souza, Simone R. S. ^[1]

Afiliação do(s) autor(es)

^[1] Univ Sao Paulo, Inst Math & Comp Sci, Ave Trabalhador Sao Carlense, 400 Ctr, BR-13566590 Sao Carlos, SP - Brazil

^[2] Univ Alabama, Dept Comp Sci, 3441 SEC, Box 870290, Tuscaloosa, AL 35487 - USA

• Tipo de documento: Artigo Científico
• Fonte: INFORMATION AND SOFTWARE TECHNOLOGY; v. 105, p. 226-251, JAN 2019.
• Citações Web of Science: 1

Resumo

Background: Concurrent software testing is a costly and difficult task, especially due to the exponential increase in the test sequences caused by non-determinism. Such an issue has motivated researchers to develop testing techniques that select a subset of the input domain that has a high probability of revealing faults. Academics and industrial practitioners rarely use most concurrent software testing techniques because of the lack of data about their applicability. Empirical evidence can provide an important scientific basis for the strengths and weaknesses of each technique to help researchers and practitioners choose concurrent testing techniques appropriate for their environments. Aim: This paper gathers and synthesizes empirical research on concurrent software testing to characterize the field and the types of empirical studies performed. Method: We performed a systematic mapping study to identify and analyze empirical research on concurrent software testing techniques. We provide a detailed analysis of the studies and their design choices. Results: The primary findings are: (1) there is a general lack of empirical validation of concurrent software testing techniques, (2) the type of evaluation method varies with the type of technique, (3) there are some key challenges to empirical study design in concurrent software testing, and (4) there is a dearth of controlled experiments in concurrent software testing. Conclusions: There is little empirical evidence available about some specific concurrent testing techniques like model-based testing and formal testing. Overall, researchers need to perform more empirical work, especially real-world case studies and controlled experiments, to validate properties of concurrent software testing techniques. In addition, researchers need to perform more analyses and synthesis of the existing evidence. This paper is a first step in that direction. (AU)

• Processo FAPESP: 15/23653-5 - Definição de um Design de Estudos Empíricos em Técnicas de Teste para Programas Concorrentes
• Beneficiário: Silvana Morita Melo
• Modalidade de apoio: Bolsas no Exterior - Estágio de Pesquisa - Doutorado
• Processo FAPESP: 13/05046-9 - Definição de um Framework para Avaliação Sistemática de Técnicas de Teste no Contexto de Programação Concorrente
• Beneficiário: Silvana Morita Melo
• Modalidade de apoio: Bolsas no Brasil - Doutorado