Multi-stream Architecture with Symmetric Extended Visual Rhythms for Deep Learning Human Action Recognition

Autor(es):Mostrar menos -

Tacon, Hemerson ; Brito, Andre de Souza ; Chaves, Hugo de Lima ; Vieira, Marcelo Bernardes ; Villela, Saulo Moraes ; Maia, Helena de Almeida ; Concha, Darwin Ttito ; Pedrini, Helio ; Farinella, GM ; Radeva, P ; Braz, J

• Tipo de documento: Artigo Científico
• Fonte: VISAPP: PROCEEDINGS OF THE 15TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS, VOL 4: VISAPP; v. N/A, p. 8-pg., 2020-01-01.

Resumo

Despite the significant progress of Deep Learning models on the image classification task, it still needs enhancements for the Human Action Recognition task. In this work, we propose to extract horizontal and vertical Visual Rhythms as well as their data augmentations as video features. The data augmentation is driven by crops extracted from the symmetric extension of the time dimension, preserving the video frame rate, which is essential to keep motion patterns. The crops provide a 2D representation of the video volume matching the fixed input size of a 2D Convolutional Neural Network. In addition, multiple crops with stride guarantee coverage of the entire video. We verified that the combination of horizontal and vertical directions leads do better results than previous methods. A multi-stream strategy combining RGB and Optical Flow information is modified to include the additional spatiotemporal streams: one for the horizontal Symmetrically Extended Visual Rhythm (SEVR), and another for the vertical one. Results show that our method achieves accuracy rates close to the state of the art on the challenging UCF101 and HMDB51 datasets. Furthermore, we assessed the impact of data augmentations methods for Human Action Recognition and verified an increase of 10% for the UCF101 dataset. (AU)

• Processo FAPESP: 17/12646-3 - Déjà vu: coerência temporal, espacial e de caracterização de dados heterogêneos para análise e interpretação de integridade
• Beneficiário: Anderson de Rezende Rocha
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 17/09160-1 - Reconhecimento de Ações Humanas em Vídeos
• Beneficiário: Helena de Almeida Maia
• Modalidade de apoio: Bolsas no Brasil - Doutorado