Multi-Stream Convolutional Neural Networks for Action Recognition in Video Sequences Based on Adaptive Visual Rhythms

Autor(es):Mostrar menos -

Concha, Darwin Ttito ; Maia, Helena de Almeida ; Pedrini, Helio ; Tacon, Hemerson ; Brito, Andre de Souza ; Chaves, Hugo de Lima ; Vieira, Marcelo Bernardes ; Wani, MA ; Kantardzic, M ; Sayedmouchaweh, M ; Gama, J ; Lughofer, E

• Tipo de documento: Artigo Científico
• Fonte: 2018 17TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA); v. N/A, p. 8-pg., 2018-01-01.

Resumo

Advances in digital technology have increased event recognition capabilities through the development of devices with high resolution, small physical dimensions and high sampling rates. The recognition of complex events in videos has several relevant applications, particularly due to the large availability of digital cameras in environments such as airports, banks, roads, among others. The large amount of data produced is the ideal scenario for the development of automatic methods based on deep learning. Despite the significant progress achieved through image-based deep networks, video understanding still faces challenges in modeling spatio-temporal relations. In this work, we address the problem of human action recognition in videos. A multi-stream network is our architecture of choice to incorporate temporal information, since it may benefit from pre-trained deep networks for images and from handcrafted features for initialization. Furthermore, its training cost is usually lower than video-based networks. We explore visual rhythm images since they encode longer-term information when compared to still frames and optical flow. We propose a novel method based on point tracking for deciding the best visual rhythm direction for each video. Experiments conducted on the challenging UCF101 and HMDB51 data sets indicate that our proposed stream improves network performance, achieving accuracy rates comparable to the stateof- the-art approaches. (AU)

• Processo FAPESP: 17/09160-1 - Reconhecimento de Ações Humanas em Vídeos
• Beneficiário: Helena de Almeida Maia
• Modalidade de apoio: Bolsas no Brasil - Doutorado
• Processo FAPESP: 17/12646-3 - Déjà vu: coerência temporal, espacial e de caracterização de dados heterogêneos para análise e interpretação de integridade
• Beneficiário: Anderson de Rezende Rocha
• Modalidade de apoio: Auxílio à Pesquisa - Temático