The impact of different facial expression intensities on the performance of pre-trained emotion recognition models

Author(s)

de Araujo, Hermon Faria ; Nunes, Fatima L. S. ; Machado-Lima, Ariane ; ACM

• Document type: Journal article
• Source: 37TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING; v. N/A, p. 8-pg., 2022-01-01.

Abstract

Facial Expression Recognition (FER) has improved a great deal with the advances of machine learning, mainly due to the development of deep learning methods for automatic facial expression classification. Due to the availability of big datasets required for the training processes, the amount of commercial and open-source solutions that use transfer learning and pre-trained machine learning models have increased. However, there is not enough information about the performance of these models in non-standard scenarios, in view of a set of images with variations of intensity in emotional facial expressions. In this article, an evaluation of the performance of pretrained open-source models used for facial expressions recognition of emotions was carried out on images with different facial expression intensities. A total of 1512 video frames from the ADFES-BIV dataset were submitted to five pre-trained machine learning models for performance evaluation. The ADFES-BIV dataset contains representations of seven facial expressions (anger, fear, disgust, happiness, neutral, sadness, surprise) considering three intensity levels (low, intermediate and high). The highest accuracy values (60% to 74%) were obtained for images of apex frames from high intensity facial expression videos. The lowest accuracy values (17% to 29%) were observed for low intensity frames. The results show that the tested pre-trained machine learning models are very susceptible to variations in intensity of facial expressions. In addition, a large recall variability rate was observed among different facial expressions. Therefore, before adopting pre-trained models, their performance should be carefully analyzed in order to meet the requirements of each specific application. (AU)

• FAPESP's process: 20/01992-0 - Computer aid system for the diagnosis of psychiatric disorders based on facial anthropometric measurements
• Grantee: Ariane Machado Lima
• Support Opportunities: Research Grants - eScience and Data Science Program - Regular Program Grants
• FAPESP's process: 14/50889-7 - National Institute of Science and Technology Medicine Assisted by Scientific Computing (INCT-MACC)
• Grantee: José Eduardo Krieger
• Support Opportunities: Research Projects - Thematic Grants