Activity recognition and bioinspired approaches for robotics in intelligent environments

Author(s)

Caetano Mazzoni Ranieri

• Document type: Doctoral Thesis
• Press: São Carlos.
• Institution: Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB)
• Defense date: 2021-06-02

Examining board members

Roseli Aparecida Francelin Romero; Fernando Santos Osório; João Paulo Papa; Josue Junior Guimarães Ramos

• Advisor: Roseli Aparecida Francelin Romero; Patrícia Amâncio Vargas

Abstract

Home automation projects have been developed for some time, having evolved into the socalled smart environments. These environments are characterised by the presence of sets of sensors and actuators, connected in order to respond appropriately and proactively to different situations. The integration of intelligent environments with robots allows for the introduction of additional sensing capabilities, besides performing tasks with greater flexibility and less mechanical complexity than traditional monolithic robots. To endow such environments with truly autonomous behaviours, algorithms must extract semantically meaningful information from whichever sensor data is available. Human activity recognition is one of the most active fields of research within this context. In this project, the design and evaluations of learning techniques for human activity recognition was addressed, considering different sensor modalities. Two types of neural networks, based on combinations of Convolutional Neural Networks to Recurrent Networks with Long Short-Term Memory or Temporal Convolutional Networks, were proposed and evaluated on two public datasets for multimodal activity recognition from videos and inertial sensors. The resulting framework was then introduced to a new dataset, the HWU-USP activities dataset, collected as part of this work, in an actual environment endowed with videos, inertial units, and ambient sensors. This design allowed for assessing the influence of ambient sensors, synchronised to the inertial and video data, to the accuracy of the results, which has proven to be a promising approach. Also, the new dataset provided complex activities with long-term dependencies, evaluated through segment-wise classifiers simulating the results for real-time applications. In a second moment, works were developed on neurophysiological data from primates induced to Parkinsons disease. Those studies ranged from data analysis and classification, using neural networks, to the construction of a computational model of the affected structures within the brain. Although different from the studies on activity recognition and assistive technologies, which were the focus of this thesis, these works were related in the nature of the techniques used, and their results were part of the application scenario developed next. Finally, an application scenario was designed and implemented as a robot simulation, so that the developed module could be evaluated in practical situations. For the behaviour selection mechanism, a bioinspired approach based on computational models of the basal ganglia-thalamus-cortex circuit was evaluated and compared to non-bioinspired approaches based on simple heuristics. (AU)

• FAPESP's process: 17/02377-5 - Machine Learning and Applications for Robotics in Smart Environments
• Grantee: Caetano Mazzoni Ranieri
• Support Opportunities: Scholarships in Brazil - Doctorate