Advanced search
Start date
Betweenand
Related content
(Reference retrieved automatically from Web of Science through information on FAPESP grant and its corresponding number as mentioned in the publication by the authors.)

Measurement of the Inter-Rater Reliability Rate Is Mandatory for Improving the Quality of a Medical Database: Experience with the Paulista Lung Cancer Registry

Full text
Author(s):
Lauricella, Leticia L. [1, 2] ; Costa, Priscila B. [2] ; Salati, Michele [3] ; Pego-Fernandes, Paulo M. [2, 4] ; Terra, Ricardo M. [1, 2]
Total Authors: 5
Affiliation:
[1] Inst Canc Estado Sao Paulo, Sao Paulo - Brazil
[2] Univ Sao Paulo, Fac Med, Thorac Surg Div, 44 Dr Eneas de Carvalho Aguiar Ave, B 2, Room 9, BR-05403000 Sao Paulo, SP - Brazil
[3] Univ Ancona, Sch Med, Ancona - Italy
[4] Univ Sao Paulo, Fac Med, Hosp Clin, Thorac Surg Div, Inst Coracao, Sao Paulo - Brazil
Total Affiliations: 4
Document type: Journal article
Source: JOURNAL OF THE AMERICAN COLLEGE OF SURGEONS; v. 226, n. 6, p. 1128-1136, JUN 2018.
Web of Science Citations: 0
Abstract

BACKGROUND: Database quality measurement should be considered a mandatory step to ensure an adequate level of confidence in data used for research and quality improvement. Several metrics have been described in the literature, but no standardized approach has been established. We aimed to describe a methodological approach applied to measure the quality and inter-rater reliability of a regional multicentric thoracic surgical database (Paulista Lung Cancer Registry). STUDY DESIGN: Data from the first 3 years of the Paulista Lung Cancer Registry underwent an audit process with 3 metrics: completeness, consistency, and inter-rater reliability. The first 2 methods were applied to the whole data set, and the last method was calculated using 100 cases randomized for direct auditing. Inter-rater reliability was evaluated using percentage of agreement between the data collector and auditor and through calculation of Cohen's kappa and intraclass correlation. RESULTS: The overall completeness per section ranged from 0.88 to 1.00, and the overall consistency was 0.96. Inter-rater reliability showed many variables with high disagreement (>10%). For numerical variables, intraclass correlation was a better metric than inter-rater reliability. Cohen's k showed that most variables had moderate to substantial agreement. CONCLUSIONS: The methodological approach applied to the Paulista Lung Cancer Registry showed that completeness and consistency metrics did not sufficiently reflect the real quality status of a database. The inter-rater reliability associated with k and intraclass correlation was a better quality metric than completeness and consistency metrics because it could determine the reliability of specific variables used in research or benchmark reports. This report can be a paradigm for future studies of data quality measurement. (C) 2018 by the American College of Surgeons. Published by Elsevier Inc. All rights reserved. (AU)

FAPESP's process: 12/51764-8 - Implementation of the paulista registry of lung cancer surgical treatment
Grantee:Ricardo Mingarini Terra
Support Opportunities: Research Grants - Research in Public Policies for the National Health Care System (PP-SUS)