Visual attention models based on compressed domain video analysis techniques

• Grant number: 21/02739-0
• Support Opportunities: Scholarships in Brazil - Master
• Start date: April 01, 2021
• End date: January 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Agreement: Microsoft Research
• Principal Investigator: João Paulo Papa
• Grantee: Pedro Lamkowski dos Santos
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Company: Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Instituto de Geociências e Ciências Exatas (IGCE)
• Associated research grant: 17/25908-6 - Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert, AP.PITE

Abstract

Visual attention models take inspiration from how the human eye works, i.e., we can acquire a broad view of the scene we are looking at, but we tend to focus on a particular region only. Usually, such a region is the one that, for some reason, calls our attention to it. When we work with compressed images, e.g., JPG, PNG, and GIF, we first need to decode them before feeding Convolutional Neural Networks (CNNs) for proper feature extraction. Indeed, this operation demands additional computational costs. Therefore, the design of CNNs capable of learning directly from the compressed domain is highly desirable. In this proposal, we aim at combining attention models and deep networks to work on the compressed domain, whose outcomes will be evaluated in the context of weak video classification. We expect that the proposed models will turn into more efficient approaches. (AU)