Semi-automatic selection of primary studies in systematic literature reviews: is it reasonable?

Author(s)

Octaviano, Fabio R. ^{[1, 2]} ; Felizardo, Katia R. ^[3] ; Maldonado, Jose C. ^[4] ; Fabbri, Sandra C. P. F. ^[1]

Affiliation

^[1] Univ Fed Sao Carlos, Dept Comp, BR-13565905 Sao Carlos, SP - Brazil

^[2] Univ Fed Sao Carlos, Dept Comp Syst, BR-13565905 Sao Carlos, SP - Brazil

^[3] Fed Technol Univ Parana UTFPR, Dept Comp, BR-86300000 Cornelio Procopio, Parana - Brazil

^[4] Univ Sao Paulo, Dept Comp Syst, BR-86300000 Cornelio Procopio, Parana - Brazil

• Document type: Journal article
• Source: EMPIRICAL SOFTWARE ENGINEERING; v. 20, n. 6, p. 1898-1917, DEC 2015.
• Web of Science Citations: 4

Abstract

The systematic review (SR) is a methodology used to find and aggregate all relevant existing evidence about a specific research question of interest. One of the activities associated with the SR process is the selection of primary studies, which is a time consuming manual task. The quality of primary study selection impacts the overall quality of SR. The goal of this paper is to propose a strategy named ``Score Citation Automatic Selection{''} (SCAS), to automate part of the primary study selection activity. The SCAS strategy combines two different features, content and citation relationships between the studies, to make the selection activity as automated as possible. Aiming to evaluate the feasibility of our strategy, we conducted an exploratory case study to compare the accuracy of selecting primary studies manually and using the SCAS strategy. The case study shows that for three SRs published in the literature and previously conducted in a manual implementation, the average effort reduction was 58.2 % when applying the SCAS strategy to automate part of the initial selection of primary studies, and the percentage error was 12.98 %. Our case study provided confidence in our strategy, and suggested that it can reduce the effort required to select the primary studies without adversely affecting the overall results of SR. (AU)

• FAPESP's process: 12/02524-4 - Evidence-Based Software Engineering: Systematic Literature Review Process based on Visual Text Mining
• Grantee: Katia Romero Felizardo Scannavino
• Support Opportunities: Scholarships in Brazil - Post-Doctoral