Advanced search
Start date

Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert

Grant number: 17/25908-6
Support type:Research Grants - Research Partnership for Technological Innovation - PITE
Duration: February 01, 2019 - January 31, 2023
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Cooperation agreement: Microsoft Research
Principal Investigator:João Paulo Papa
Grantee:João Paulo Papa
Home Institution: Instituto de Geociências e Ciências Exatas (IGCE). Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Rio Claro , SP, Brazil
Company: Microsoft Informática Ltda
City: Rio Claro
Co-Principal Investigators:Daniel Carlos Guimarães Pedronette ; Fabio Augusto Faria ; João Paulo Papa ; Jurandy Gomes de Almeida Junior
Assoc. researchers:João Paulo Papa
Associated scholarship(s):19/11104-8 - A comparative analysis of rank correlation measures for weakly-supervised learning, BP.IC
19/15837-0 - Restricted Boltzmann Machines applied to video-based action recognition, BP.IC
19/10998-5 - Investigation of compressed domain video features based on deep neural networks, BP.IC
19/07825-1 - Deep Boltzmann machines for event recognition in videos, BP.MS
19/04754-6 - Weakly supervised learning strategies through Rank-based measures, BP.MS


Several machine learning techniques have relied on large labeled data sets to construct predictive models and solving supervised learning tasks. The use of deep learning techniques can be highlighted, since it have been broadly and successfully used in various domains. On the other hand, in many circumstances, the labeled sets are unavailable or insufficient to train effective supervised models. Such scenarios have been mainly addressed by unsupervised learning techniques, which consider the unlabeled data to learn about its structure. However, the use of completely unsupervised methods still remains a research challenge in many scenarios and situations. A promising solution is based on the use of weakly supervised approaches, capable of performing effective learning tasks based on incomplete or inaccurate labeled sets. In this project, we intend to investigate the analysis, retrieval, and classification of compressed video domain based on small training sets. The main object of the project consists in to investigate and propose methods capable of analyzing compressed video sequences and trigger alerts according to considered applications. Such approaches can be useful and relevant in several domains, ranging from surveillance, medical and industrial environments to smart homes. The fundamental research challenge consists in making use of different techniques in order to analyze, represent, and classification videos using restricted labeled data. The proposed approach aims at exploiting the maximum available information, in order to become the approach suitable for operating with small training datasets. We intend to exploit: (i) deep learning representations; (ii) contextual unsupervised measures and; (iii) fusion techniques, in order to extend the initial labeled sets. The first challenge to be addressed is to analyze and represent videos in the compressed domain using deep learning techniques. Based on such representations, we intend to investigate strategies for expanding the training sets using unsupervised contextual measures. Given the obtained labeled sets, fusion strategies will be used to combined diverse classification methods and triggering alerts. Although the methods which will investigated can be used in several domains, we intend to select domains to validate the proposed approaches. The selection will be performed considering the existence of public available datasets to conduct experimental evaluations. (AU)

Scientific publications (7)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
VALEM, LUCAS PASCOTTI; GUIMARAES PEDRONETTE, DANIEL CARLOS. Unsupervised selective rank fusion for image retrieval tasks. Neurocomputing, v. 377, p. 182-199, FEB 15 2020. Web of Science Citations: 0.
SANTANA, MARCOS C. S.; PASSOS, JR., LEANDRO APARECIDO; MOREIRA, THIERRY P.; COLOMBO, DANILO; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. A Novel Siamese-Based Approach for Scene Change Detection With Applications to Obstructed Routes in Hazardous Environments. IEEE INTELLIGENT SYSTEMS, v. 35, n. 1, p. 44-53, JAN-FEB 2020. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; VALEM, LUCAS PASCOTTI; ALMEIDA, JURANDY; TONES, RICARDO DA S. Multimedia Retrieval Through Unsupervised Hypergraph-Based Manifold Ranking. IEEE Transactions on Image Processing, v. 28, n. 12, p. 5824-5838, DEC 2019. Web of Science Citations: 0.
CAMPOS, VICTOR DE ABREU; GUIMARAES PEDRONETTE, DANIEL CARLOS. A framework for speaker retrieval and identification through unsupervised learning. COMPUTER SPEECH AND LANGUAGE, v. 58, p. 153-174, NOV 2019. Web of Science Citations: 0.
SCARPARO, DANIELE CRISTINA; PINHEIRO SALVADEO, DENIS HENRIQUE; GUIMARAES PEDRONETTE, DANIEL CARLOS; BARUFALDI, BRUNO; ARNOLD MAIDMENT, ANDREW DOUGLAS. Evaluation of denoising digital breast tomosynthesis data in both projection and image domains and a study of noise model on digital breast tomosynthesis image domain. JOURNAL OF MEDICAL IMAGING, v. 6, n. 3 JUL 2019. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; WENG, YING; BALDASSIN, ALEXANDRO; HOU, CHAOHUAN. Semi-supervised and active learning through Manifold Reciprocal kNN Graph for image retrieval. Neurocomputing, v. 340, p. 19-31, MAY 7 2019. Web of Science Citations: 0.
VALEM, LUCAS PASCOTTI; DE OLIVEIRA, CARLOS RENAN; GUIMARAES PEDRONETTE, DANIEL CARLOS; ALMEIDA, JURANDY. Unsupervised Similarity Learning through Rank Correlation and kNN Sets. ACM Transactions on Multimedia Computing Communications and Applications, v. 14, n. 4 NOV 2018. Web of Science Citations: 1.

Please report errors in scientific publications list by writing to: