Acoustic-phonetic analysis of the expression of emotions in real life speech

Author(s)

Wellington da Silva

• Document type: Master's Dissertation
• Press: Campinas, SP.
• Institution: Universidade Estadual de Campinas (UNICAMP). Instituto de Estudos da Linguagem
• Defense date: 2014-04-07

Examining board members

Plinio Almeida Barbosa; Emilio Gozze Pagotto; Sandra Madureira

• Advisor: Plinio Almeida Barbosa

Abstract

This dissertation presents a cross-cultural study on the perception of real emotions expressed in speech conducted with Swedish and Brazilian listeners. The literature on the subject shows that listeners are capable of recognizing emotions expressed in speech in perception experiments with much higher percentage of success than that expected by chance, even when evaluating stimuli in a foreign language and of a different culture. The main objectives of the present study were to investigate which acoustic-phonetic parameters among those extracted are affected by the emotional state of the speakers, on which of them the listeners rely to judge the degree of expression of these emotions in the perception experiments, to investigate whether the subjects perform better in this task when evaluating emotional dimensions rather than discrete emotions, whether the listeners¿ culture and mother language affect the perception of the emotions and whether the Brazilian and the Swedish subjects rely on the same acoustic parameters to make the judgements. The corpora used consist of utterances of female speakers of Brazilian Portuguese and Swedish extracted from a documentary film as well as from radio and TV programs. The following acoustic parameters were measured automatically for these utterances with a script for the software PRAAT: fundamental frequency and its first derivative, global intensity, spectral tilt and Long-Term Average Spectrum (LTAS). A total of twelve statistical descriptors were computed for these parameters. We conducted two perception experiments with Brazilian and Swedish subjects. In the first one they evaluated the degree of expression of the discrete emotions described by eight adjectives (joyful, moved, surprised, sad, contented, anguished, distressed and enthusiastic) for the utterances in Brazilian Portuguese on scales ranging from 0 to 4. In the second experiment the subjects rated the degree of expression of five emotional dimensions (activation, fairness, valence, motivation and involvement) for the utterances in Swedish. The responses of the subjects in the experiments were analyzed by means of PCA. For experiment I, the PCA revealed that the listeners of both nationalities evaluated the emotions described by the eight adjectives jointly by means of two major emotional dimensions: happiness and neutrality. For experiment II, the two PCA components revealed that the listeners of both nationalities evaluated the five emotional dimensions distinguishing between a state of "calmness" and a state of higher emotional agitation. The PCA components were correlated with the acoustic parameters by means of linear regression analyses. These analyses showed that the subjects of both nationalities relied on the same acoustic parameters to make the judgements in both experiments. These acoustic parameters were also very robust in classifying the utterances in the LDA performed. The parameters which proved more relevant were the LTAS, fundamental frequency median and semi-interquartile range and spectral tilt mean and standard deviation. It is possible to conclude that the culture and the emotional experience of the Brazilian and the Swedish subjects did not affect the perception of the emotions expressed in both corpora (AU)

• FAPESP's process: 12/04254-4 - Acoustic-phonetic analysis of the expression of emotions in real life testimonies.
• Grantee: Wellington da Silva
• Support Opportunities: Scholarships in Brazil - Master