Combining Intelligence Artificial and Multimodal for Emotion Classification

• Grant number: 17/21054-2
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate
• Start date: February 01, 2018
• End date: January 31, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Jó Ueyama
• Grantee: Leandro Yukio Mano Alves
• Supervisor: Michael Biehl
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Institution abroad: University of Groningen, Netherlands
• Associated to the scholarship: 16/14267-7 - Exploiting the Internet of Things with Artificial Intelligence in the context of Health Smart Homes: a physical and emotional approach, BP.DR

Abstract

Current studies in the field of Human-Computer Interaction have drawn attention to the importance of emotional factors while interacting with computer systems. It is believed that if intelligent agents are employed to determine the nature of a user's emotions, they can induce and arouse them in a way that acts as a stimulus while interacting with computers. For example, if the computer finds out that the user is tired, under stress or depressed, then it attempts to ``persuade him'' to overcome this emotional state through a song, video and/or a game. A major challenge for researchers in human-computer interaction is to provide systems that are capable of recognizing, interpreting and reacting intelligently and sensitively to the emotions of the user, and thus meet the requirements of the largest possible number of individuals. One of the ways to achieve this in this project is to develop flexible systems that can handle a large number of emotions/types of behavior. The main purpose in carrying out this emotional interaction is to improve the coherence, consistency and credibility of the reactions and computational responses that occur during a human interaction via a human-computer interface. In this context, there is an opportunity to explore computational systems that are able to discover or infer the emotional state of the user at runtime; for example, if the software is aware of a "sad" emotional state of an elderly individual, it may suggest a song, a movie or an activity which can this change to a "happy" emotional state ,and thus determine and/or prevent emotional disorders. This project aims to develop and evaluate a model that can: i) find out the emotional state of elderly users; ii) explore multiple sensors for the classification of emotion, and; iii) to perform the relation of the responses of multiple sensors to determine the emotion of the user.

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

MANO, LEANDRO Y.; BARROS, VINICIUS A.; NUNES, LUIZ H.; SAWADA, LUANA O.; ESTRELLA, JULIO C.; UEYAMA, JO. ENLACE: A Combination of Layer-Based Architecture and Wireless Communication for Emotion Monitoring in Healthcare. MOBILE INFORMATION SYSTEMS, v. 2019, JUL 10 2019. (17/21054-2, 16/14267-7)

MANO, LEANDRO Y.; YILDIRIM, T; MANOLOPOULOS, Y; ANGELOV, P; ILIADIS, L. Emotional condition in the Health Smart Homes environment: emotion recognition using ensemble of classifiers. 2018 INNOVATIONS IN INTELLIGENT SYSTEMS AND APPLICATIONS (INISTA), v. N/A, p. 8-pg., 2018-01-01. (17/21054-2, 16/14267-7)