Towards wearable gaze interaction

Author(s)

Carlos Eduardo Leão Elmadjian

• Document type: Doctoral Thesis
• Press: São Paulo.
• Institution: Universidade de São Paulo (USP). Instituto de Matemática e Estatística (IME/SBI)
• Defense date: 2023-05-16

Examining board members

Carlos Hitoshi Morimoto; Roberto Hirata Junior; Enkelejda Kasneci; Thies Pfeiffer; Marco Porta

• Advisor: Carlos Hitoshi Morimoto

Abstract

Human-computer interaction has been facing a significant shift from the traditional paradigm of WIMP interfaces using desktop PCs and notebooks to a more distributed and heterogeneous one, where interaction occurs with multiple devices, including mobile phones, smart watches, smart glasses, and IoT-enabled devices. As a result, traditional means of interaction may not provide a satisfactory user experience in such a scenario. In this thesis, we look into gaze-based interaction as an alternative to comply with this new paradigm and some of its fundamental challenges. In particular, we investigate new techniques to recognize eye movement patterns, 3D gaze estimation, and gaze-based interactive methods for micro-interaction scenarios. Though our ultimate goal is to enhance gaze interaction in wearable computing, we broke down this objective into several fronts, namely improving gaze estimation for 3D environments using wearable eye-tracking devices, enhancing real-time eye movement pattern recognition to support context-aware systems using deep temporal convolutional models and scale-invariant methods, and creating novel gaze interaction methods with a just-look-to-select mechanism, in spite of the Midas touch problem. Our results demonstrate that a 3D wearable gaze estimation procedure is feasible, though still very challenging; that our techniques for eye pattern recognition are able to achieve the best of both worlds: state-of-the-art accuracy while being computationally lightweight; and that our interactive techniques, such as GazeBar and V-Switch, have the potential to improve micro-interactions without the common safety mechanisms deployed to gaze-based methods. Though there are still significant obstacles to be addressed in future work, including thorough user studies and complex context detection strategies, this thesis shines new light towards gaze-based interaction and related challenges in this paradigm-shifting environment. (AU)

• FAPESP's process: 15/26802-1 - Augmented cognition using wearable computing
• Grantee: Carlos Eduardo Leão Elmadjian
• Support Opportunities: Scholarships in Brazil - Doctorate (Direct)