From Actions to Events: A Transfer Learning Approach Using Improved Deep Belief Networks

Author(s)

Roder, Mateus ; Almeida, Jurandy ; De Rosa, Gustavo H. ; Passos, Leandro A. ; Rossi, Andre L. D. ; Papa, Joao P. ; IEEE

• Document type: Journal article
• Source: 2021 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2021); v. N/A, p. 8-pg., 2021-01-01.

Abstract

In the last decade, exponential data growth supplied machine learning-based algorithms' capacity and enabled their usage in daily-life activities. Additionally, such an improvement is partially explained due to the advent of deep learning techniques, i.e., stacks of simple architectures that end up in more complex models. Although both factors produce outstanding results, they also pose drawbacks regarding the learning process as training complex models over large datasets are expensive and time-consuming. Such a problem is even more evident when dealing with video analysis. Some works have considered transfer learning or domain adaptation, i.e., approaches that map the knowledge from one domain to another, to ease the training burden, yet most of them operate over individual or small blocks of frames. This paper proposes a novel approach to map the knowledge from action recognition to event recognition using an energy-based model, denoted as Spectral Deep Belief Network. Such a model can process all frames simultaneously, carrying spatial and temporal information through the learning process. The experimental results conducted over two public video dataset, the HMDB-51 and the UCF-101, depict the effectiveness of the proposed model and its reduced computational burden when compared to traditional energy-based models, such as Restricted Boltzmann Machines and Deep Belief Networks. (AU)

• FAPESP's process: 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry
• Grantee: Francisco Louzada Neto
• Support Opportunities: Research Grants - Research, Innovation and Dissemination Centers - RIDC
• FAPESP's process: 19/07825-1 - Deep Boltzmann machines for event recognition in videos
• Grantee: Mateus Roder
• Support Opportunities: Scholarships in Brazil - Master
• FAPESP's process: 17/25908-6 - Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert
• Grantee: João Paulo Papa
• Support Opportunities: Research Grants - Research Partnership for Technological Innovation - PITE
• FAPESP's process: 19/07665-4 - Center for Artificial Intelligence
• Grantee: Fabio Gagliardi Cozman
• Support Opportunities: Research Grants - Research Program in eScience and Data Science - Research Centers in Engineering Program