Video event recognition with deep learning

• Grant number: 16/10703-7
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate
• Start date: October 01, 2016
• End date: September 30, 2017
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Agma Juci Machado Traina
• Grantee: Letrícia Pereira Soares Avalhais
• Supervisor: Larry Davis
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Institution abroad: University of Maryland, College Park, United States
• Associated to the scholarship: 13/27101-1 - An approach to semantic segmentation exploiting multimodality and temporal characteristics of digital videos, BP.DR

Abstract

The huge amount of videos made available online every day has created a demand for intelligent tools supporting basic multimedia systems needs as indexing, retrieval and automatic tagging. Robust event recognition strategy can directly aid such needs. A common approach for this task is to extract low-level features from the videos and to use classical machine learning techniques to model potential patterns from the features. Other approaches use multimodal features combined into a single representation. Deep learned high-level features is a recent trend in this subject, reporting promising results. In this project, we propose to develop techniques to integrate low-level features with high-level features automatically learned in multimodal representation aiming at improving the effectiveness of event recognition tasks. The internship project will be hosted at the University of Maryland, USA, under the supervision of Professor Larry Davis. (AU)