A systematic review on the evaluation and characteristics of computer-aided diagnosis systems

Author(s)

Vagner Mendonça Gonçalves ; Márcio Eduardo Delamaro ; Fátima de Lourdes dos Santos Nunes

• Document type: Journal article
• Source: Rev. Bras. Eng. Bioméd.; v. 30, n. 4, p. 355-383, Dez. 2014.

Abstract

INTRODUCTION: One of the challenges in developing Computer-Aided Diagnosis (CAD) systems is their accurate and comprehensive assessment. This paper presents the conduction and results of a systematic review (SR) that aims to verify the state of the art regarding the assessment of CAD systems. This survey provides a general analysis of the current status of the design, development and assessment of such systems and includes discussions on the most used metrics and approaches that could be utilized to obtain more objective evaluation methods. METHODS: The SR was conducted using the scientific databases, ACM Digital Library, IEEE Xplore Digital Library, ScienceDirect and Web of Science. Inclusion and exclusion criteria were defined and applied to each retrieved work to select those of interest. From 156 studies retrieved, 100 studies were included. Results: There is a number of abnormalities that have been used for the development of CAD systems. Images from computed tomographies and mammographies are the most encountered types of medical images. Additionally, a number of studies used public databases for CAD evaluations. The main evaluation metrics and methods applied to CAD systems include sensitivity, accuracy, specificity and receiver operating characteristic (ROC) analyses. In the assessed CAD systems that used the segmentation method, 30.0% applied the overlap measure. DISCUSSION: There remain several topics to explore for the assessment of CAD schemes. While some evaluation metrics are traditionally used, they require a prior knowledge of case characteristics to test CAD systems. We were not able to identify articles that use software testing to evaluate CAD systems. Thus, we realize that there is a gap between CAD assessments and traditional practices of software engineering. However, the scope of this research is limited to scientific and academic works and excludes commercial interests. Finally, we discuss potential research studies within this scope to create a more objective and efficient evaluation of CAD systems. (AU)

• FAPESP's process: 10/01496-1 - Application of CBIR for evaluating computer-aided diagnosis systems
• Grantee: Vagner Mendonça Gonçalves
• Support type: Scholarships in Brazil - Scientific Initiation
• FAPESP's process: 10/15691-0 - Proposition, implementation and validation of techniques for virtual interactive medical training
• Grantee: Fátima de Lourdes dos Santos Nunes Marques
• Support type: Regular Research Grants
• FAPESP's process: 10/09806-0 - Software test and validation: theoretical studies and applications
• Grantee: Márcio Eduardo Delamaro
• Support type: Regular Research Grants