Restricted Boltzmann Machines applied to video-based action recognition

• Grant number: 19/15837-0
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: August 01, 2019
• End date: July 31, 2020
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Agreement: Microsoft Research
• Principal Investigator: João Paulo Papa
• Grantee: Pedro Lamkowski dos Santos
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Company: Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Instituto de Geociências e Ciências Exatas (IGCE)
• Associated research grant: 17/25908-6 - Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert, AP.PITE

Abstract

Restricted Boltzmann Machines (RBMs) are widely used in a number of applications in the literature, but they have been poorly considered in the context of action recognition in videos. The main problem concerns encoding the temporal dependence present in such multimedia data, which is of crucial importance to recognize the numerous situations that may occur in a sequence of frames. In this project, we propose to apply RBMs to analyze videos in order to recognize actions performed by people in different situations, as well as we also intend to evaluate temporal-based RBMs.