Influence of examiner training on the perceptual assessment of hypernasality

Author(s)

Adriana Cristina de Almeida Santos Furlan de Oliveira

• Document type: Master's Dissertation
• Press: Bauru.
• Institution: Universidade de São Paulo (USP). Faculdade de Odontologia de Bauru (FOB/SDB)
• Defense date: 2014-07-18

Examining board members

Renata Paciello Yamashita; Trixy Cristina Niemeyer Vilela Alves; Cristina Guedes de Azevedo Bento Gonçalves

• Advisor: Renata Paciello Yamashita

Abstract

Introduction: A high rate of agreement in the perceptual assessment of hypernasality is hardly achieved among different examiners, due to the subjectivity of this type of analysis. Previous training of examiners and the establishment of analysis criteria may be an effective strategy to minimize the effect of subjectivity on the perceptual assessment and increase the agreement among examiners. Objective: To investigate the influence of previous training on the agreement among different examiners in the perceptual assessment of hypernasality. Material and method: Three experienced speech therapists individually analyzed 77 audio-recorded speech samples of individuals with repaired cleft palate, in two stages. In the first stage, the examiners classified hypernasality according to their own criteria in a 4-point scale: 1=absence of hypernasality, 2=mild hypernasality, 3=moderate and 4=severe. After 70 days, the examiners were submitted to training, in which speech samples representing the four categories in the scale were defined by consensus, and were used as reference models for the assessment of hypernasality in the following stage. On the second stage, the examiners individually analyzed the same samples and classified hypernasality in a 4-point scale, using as criteria the references defined during training. The inter- and intraexaminer agreements in the two stages were calculated by the Kappa coefficient. These values were statistically compared by the Z test. Results: The agreement concerning the degree of hypernasality achieved among the three examiners was 0.37 (regular) before training and 0.54 (moderate) after training. Statistical analysis revealed that the agreement after training was significantly higher than the agreement achieved before training (p=0.044). Analysis of intraexaminer agreement between the two stages revealed that, for examiner 1, the agreement was increased from 0.38 (regular) to 0.61 (substantial); for examiner 2, it was increased from 0.39 (regular) to 0.92 (almost perfect), and for examiner 3, it was reduced from 0.76 (substantial) to 0.50 (moderate). Statistically significant difference was only observed for examiner 2 (p=0.004). Conclusion: Examiner training and the definition of criteria for classification of hypernasality increased the inter- and intraexaminer agreement. These outcomes reinforce the need to establish standardized criteria to minimize the influence of individual internal standards on the perceptual assessment of speech. (AU)

• FAPESP's process: 13/06015-0 - Influence of training of evaluators in perceptual judgment of hipernasality
• Grantee: Adriana Cristina de Almeida Santos Furlan de Oliveira
• Support Opportunities: Scholarships in Brazil - Master