Gaze-Based Interaction in Wearable Computing: an interface for the Internet of Things

• Grant number: 16/10148-3
• Support Opportunities: Regular Research Grants
• Start date: February 01, 2017
• End date: March 31, 2019
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Carlos Hitoshi Morimoto
• Grantee: Carlos Hitoshi Morimoto
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• City of the host institution: São Paulo
• Associated research grant(s): 17/50121-0 - Aretha:gaze interaction for the internet of things through augmented reality. (sprint 1/2017), AP.R SPRINT
• Associated scholarship(s): 18/12247-4 - Gaze interaction in wearable computing: an interface for the Internet of Things, BP.TT

Abstract

The Internet of Things (IoT) will put at our disposal an endless number of devices that can provide information and communicate with us, with applications in areas as diverse as education, smart cities, and industrial production. A major challenge of this emerging technology is how to create an effective, secure, and robust way to interact with these devices. In this project we propose to investigate the use of wearable computing platformsenhanced with eye trackers, for interaction with IoT devices.Despite recent advancements in wearable computing (WC), this technology has not yet achieved large commercial success, in part due to interaction problems and limited functionality of their apps. Gaze-based interaction also has restricted applications, despite being successfully used in interfaces for people with disabilities. One possible contribution of this project will be the development of a new interaction mode in wearable computing. In particular, for quick and simple interactions with IoT devices, gaze can provide the wearable platform important context information to be used by the wearable applications.One first objective of this project, is to create a wearable computing platform consisting basically of a notebook, a head-up display, and an eye-tracker. Two kinds of displays will be investigated, a monocular and a binocular. After we build this platform, we will investigate how gaze can be used for wearable interaction, by developing new augmented reality interfaces to IoT devices. The project will be developed with the participation of several graduate and undergraduate students from USP. (AU)

Scientific publications

(13)

(The scientific publications listed on this page originate from the Web of Science or SciELO databases. Their authors have cited FAPESP grant or fellowship project numbers awarded to Principal Investigators or Fellowship Recipients, whether or not they are among the authors. This information is collected automatically and retrieved directly from those bibliometric databases.)

BORSATO, FRANK H.; MORIMOTO, CARLOS H.; IEEE. Building structured lighting applications using low-cost cameras. 2017 30TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2017-01-01. (16/10148-3)

SOARES BORGES, ALINE DE FATIMA; MORIMOTO, CARLOS H.; IEEE. A Virtual Makeup Augmented Reality System. 2019 21ST SYMPOSIUM ON VIRTUAL AND AUGMENTED REALITY (SVR 2019), v. N/A, p. 9-pg., 2019-01-01. (16/10148-3)

VELLOSO, EDUARDO; MORIMOTO, CARLOS HITOSHI; ASSOC COMP MACHINERY. A Probabilistic Interpretation of Motion Correlation Selection Techniques. CHI '21: PROCEEDINGS OF THE 2021 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, v. N/A, p. 13-pg., 2021-01-01. (17/50121-0, 16/10148-3)

BORSATO, FRANK H.; MORIMOTO, CARLOS H.; IEEE. Asynchronous stroboscopic structured lighting image processing using low-cost cameras. PROCEEDINGS 2018 31ST SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2018-01-01. (16/10148-3)

ELMADJIAN, CARLOS; SHUKLA, PUSHKAR; TULA, ANTONIO DIAZ; MORIMOTO, CARLOS H.; SPENCER, SN. 3D gaze estimation in the scene volume with a head-mounted eye tracker. COMMUNICATION BY GAZE INTERACTION (COGAIN 2018), v. N/A, p. 9-pg., 2018-01-01. (17/06933-0, 16/10148-3)

GONZALES, CANDY T.; MORIMOTO, CARLOS H.. XTar: entropy-based target expansion for high density gaze-based interfaces. PROCEEDINGS OF THE ACM ON COMPUTER GRAPHICS AND INTERACTIVE TECHNIQUES, v. 8, n. 2, p. 15-pg., 2025-05-01. (16/10148-3)

CARNEIRO, ALEX TORQUATO S.; ELMADJIAN, CARLOS EDUARDO L.; GONZALES, CANDY; COUTINHO, FLAVIO L.; MORIMOTO, CARLOS H.; IEEE. PursuitPass: A Visual Pursuit-Based User Authentication System. 2019 32ND SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2019-01-01. (16/10148-3, 15/26802-1)

BORSATO, FRANK H.; MORIMOTO, CARLOS H.; SPENCER, SN. Towards a low cost and high speed mobile eye tracker. ETRA 2019: 2019 ACM SYMPOSIUM ON EYE TRACKING RESEARCH & APPLICATIONS, v. N/A, p. 9-pg., 2019-01-01. (16/10148-3)

BORSATO, FRANK H.; DIAZ-TULA, ANTONIO; MORIMOTO, CARLOS H.. xSDL: stroboscopic differential lighting eye tracker with extended temporal support. MACHINE VISION AND APPLICATIONS, v. 30, n. 4, p. 689-703, JUN 2019. (12/04426-0, 16/10148-3)

ELMADJIAN, CARLOS; MORIMOTO, CARLOS H.; ACM. GazeBar: Exploiting the Midas Touch in Gaze Interaction. EXTENDED ABSTRACTS OF THE 2021 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI'21), v. N/A, p. 7-pg., 2021-01-01. (15/26802-1, 16/10148-3)

MORIMOTO, CARLOS H.; LEYVA, JOSE A. T.; DIAZ-TULA, ANTONIO; SPENCER, SN. Context Switching Eye Typing Using Dynamic Expanding Targets. COMMUNICATION BY GAZE INTERACTION (COGAIN 2018), v. N/A, p. 9-pg., 2018-01-01. (16/10148-3)

CARNEIRO, ALEX TORQUATO S.; COUTINHO, FLAVIO LUIZ; MORIMOTO, CARLOS H.. Detection of visual pursuits using 1D convolutional neural networks. PATTERN RECOGNITION LETTERS, v. 179, p. 7-pg., 2024-02-03. (16/10148-3)

VELLOSO, EDUARDO; COUTINHO, FLAVIO LUIZ; KURAUCHI, ANDREW; MORIMOTO, CARLOS H.; SPENCER, SN. Circular Orbits Detection for Gaze Interaction Using 2D Correlation and Profile Matching Algorithms. 2018 ACM SYMPOSIUM ON EYE TRACKING RESEARCH & APPLICATIONS (ETRA 2018), v. N/A, p. 9-pg., 2018-01-01. (17/50121-0, 16/10148-3)