On Applying Metamorphic Testing: An Empirical Study On Academic Search Engines

Autor(es)

Andrade, Stevao ; Santos, Italo ; Brito Junior, Claudinei ; Junior, Misael ; de Souza, Simone R. S. ; Delamaro, Marcio E. ; IEEE

• Tipo de documento: Artigo Científico
• Fonte: 2019 IEEE/ACM 4TH INTERNATIONAL WORKSHOP ON METAMORPHIC TESTING (MET 2019); v. N/A, p. 8-pg., 2019-01-01.

Resumo

Software testing can be a helpful practice to certify the quality of a product. However, there are programs which are hard, if not impossible, to determine the expected outputs. This problem is called the "oracle problem". Metamorphic testing (MT) is an approach that aims to alleviate this problem by setting a series of relations, called metamorphic relations (MRs). This paper applies MT through a user-oriented approach and the following MRs: MPublished, MPTitle, MPShuffleJD and Top1Absent as a strategy for evaluating, verifying, and validating four Academic Search Engines (ASEs): ACM, IEEE, ScienceDirect, and Springer. Therefore, we conducted an experimental study to analyze how MRs can contribute to verifying the correctness of the properties of ASEs. Results indicate that the ASEs have performed differently in their execution for each MR analyzed. This shows that the algorithms of the scientific search engines have different behaviors among the characteristics analyzed. These behaviors may not be properly characterized by a possible failure, but by a simple design choice. On the other hand, the approach is useful for search engine users that need to be sure that the ASEs behave as expected during a search, as the results of secondary studies strongly depend on the correct behavior of the scientific search engine used. (AU)

• Processo FAPESP: 17/19492-1 - Teste de mutação aplicado a ambientes de realidade virtual modelados com grafo de cena
• Beneficiário: Stevão Alves de Andrade
• Modalidade de apoio: Bolsas no Brasil - Doutorado
• Processo FAPESP: 18/10183-9 - Estudo e definição de atributos de projeto para seleção de técnicas de teste de software concorrente
• Beneficiário: Italo de Oliveira Santos
• Modalidade de apoio: Bolsas no Brasil - Mestrado