Development of an evidence-based computer-based assessment application to assess clinical competencies

• Grant number: 23/01393-8
• Support Opportunities: Scholarships abroad - Research
• Start date: September 01, 2023
• End date: August 31, 2024
• Field of knowledge: Health Sciences - Medicine - Medical Clinics
• Principal Investigator: Marcos de Carvalho Borges
• Grantee: Marcos de Carvalho Borges
• Host Investigator: Larry D. Gruppen
• Host Institution: Faculdade de Medicina de Ribeirão Preto (FMRP). Universidade de São Paulo (USP). Ribeirão Preto , SP, Brazil
• Institution abroad: University of Michigan, United States

Abstract

Student assessment has undergone significant advances in its fundamental concepts. Some changes are the realization of more formative and not only summative assessments and the need to carry out competency-based assessments. Then, it is necessary to use several assessment methods in the different stages of training, which can create operational difficulties. In recent decades, big data has improved several tasks and interpretations of clinical research results, which can also be applied in Medical Education. This project aims to develop an evidence-based computer-based assessment application (Mi-TRAC) to streamline and improve the assessment process of medical residents. If successful, the Mi-TRAC application will automatically generate predicted milestones along a resident clinical competency continuum overlayed on a modern measurement model. It will also be possible to compile performance ratings from various sources and raters and organize and standardize assessments to facilitate tracking medical residents' performance. This study will be carried out at the University of Michigan. Data from medical residents' assessments will be used to define the most appropriate model to generate estimated milestone ratings along a resident clinical competency continuum. Afterward, the functionality and acceptability of the MI-TRAC system will be evaluated. Additionally, the Mi-TRAC system will be compared to existing methods used to assess residents' clinical competencies at the University of Michigan. Finally, we will estimate the application's cost savings by calculating administration and faculty time and effort saved by the automatic calibration process and report generation. (AU)