Augmented cognition using wearable computing

• Grant number: 15/26802-1
• Support Opportunities: Scholarships in Brazil - Doctorate (Direct)
• Start date: July 01, 2016
• End date: February 28, 2021
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Carlos Hitoshi Morimoto
• Grantee: Carlos Eduardo Leão Elmadjian
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Associated scholarship(s): 17/06933-0 - Augmented cognition through wearable computing, BE.EP.DD

Abstract

In this project we will investigate how to use wearable computers to enhance human cognition capacity, for example, by enhancing perception and improving memory performance. Such augmented cognition applications require information to be presented in a simple and efficient way, at appropriate times, with minimum user intervention. The motivation for this study comes from the current limitation of most wearable computing applications since they do not exploit the potential constancy associated to wearable devices. This property requires wearable applications to be designed for low and divided attention, not to interfere with other user tasks. We will use smart glasses with a Head-Up Display (HUD) for the development of this project. The HUD will show information about the scene or the task being executed by the user. Using a social scenario for example, the scene camera of the smart glasses can be used to detect and recognize faces, and automatically display information about these people on the HUD. Besides the problems of visualization and presentation of the information, we will design and investigate novel interaction methods using other emerging technologies such as eye tracking and RGBD cameras that will allow us to exploit head and gaze gestures to further enrich our wearable applications and the overall user experience. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

ELMADJIAN, CARLOS; MORIMOTO, CARLOS H.; ACM. GazeBar: Exploiting the Midas Touch in Gaze Interaction. EXTENDED ABSTRACTS OF THE 2021 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI'21), v. N/A, p. 7-pg., 2021-01-01. (15/26802-1, 16/10148-3)

SHUKLAR, PUSHKAR; ELMADJIAN, CARLOS; SHARAN, RICHIKA; KULKARNI, VIVEK; WANG, WILLIAM YANG; TURK, MATTHEW; ACL; KORHONEN, A; TRAUM, D; MARQUEZ, L. What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialogue. 57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), v. N/A, p. 10-pg., 2019-01-01. (15/26802-1)

CARNEIRO, ALEX TORQUATO S.; ELMADJIAN, CARLOS EDUARDO L.; GONZALES, CANDY; COUTINHO, FLAVIO L.; MORIMOTO, CARLOS H.; IEEE. PursuitPass: A Visual Pursuit-Based User Authentication System. 2019 32ND SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2019-01-01. (16/10148-3, 15/26802-1)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

ELMADJIAN, Carlos Eduardo Leão. Towards wearable gaze interaction. 2023. Doctoral Thesis - Universidade de São Paulo (USP). Instituto de Matemática e Estatística (IME/SBI) São Paulo.