Advanced search
Start date

Learning representations through deep generative models on video


Automatic media generation (or synthesis) is a field that had an incredible boost in recent years, with the advent of deep generative models. Nowadays, neural networks can create text, images and videos based on class labels or other media. The common task is to generate content. However, we can take advantage of the learned feature representations on these tasks to understand the relevant features and as a source of interpretability. That is, what features are relevant for the creation of different content, and how can we interpret what the models are learning or paying attention too. In this project, we propose to investigate how to learn efficient and rich representations for video data based on deep generative tasks. We focus on two particular problems for learning effective representations. The first one is semantic transfer between data modalities, in particular video and (written) language. And the second one is disentanglement within the same domain, that is, separate different variations and modalities of the data. The separation of semantics (intra and inter domain) will allow us to better understand the type of features that are learned by the different architectures on these tasks. Our objective is to train the deep generative models on different video reconstruction tasks and study their learning capabilities. We will perform experiments on the existing benchmark datasets for the particular problems. (AU)

Articles published in Agência FAPESP Newsletter about the research grant:
Articles published in other media outlets (0 total):
More itemsLess items

Scientific publications (5)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
ALBARRACIN, JUAN F. HERNANDEZ; RAMIREZ RIVERA, ADIN. Video Reenactment as Inductive Bias for Content-Motion Disentanglement. IEEE Transactions on Image Processing, v. 31, p. 10-pg., . (17/16144-2, 19/07257-3)
RIVERA, ADIN RAMIREZ; KHAN, ADIL; BEKKOUCH, IMAD EDDINE IBRAHIM; SHEIKH, TAIMOOR SHAKEEL. Anomaly Detection Based on Zero-Shot Outlier Synthesis and Hierarchical Feature Distillation. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, v. 33, n. 1, p. 281-291, . (19/07257-3)
SAIRE, DARWIN; RIVERA, ADIN RAMIREZ. Global and Local Features Through Gaussian Mixture Models on Image Semantic Segmentation. IEEE ACCESS, v. 10, p. 14-pg., . (19/07257-3, 17/16597-7, 19/18678-0)
SANTANDER, MIGUEL RODRIGUEZ; ALBARRACIN, JUAN HERNANDEZ; RIVERA, ADIN RAMIREZ. On the pitfalls of learning with limited data: A facial expression recognition case study. EXPERT SYSTEMS WITH APPLICATIONS, v. 183, . (19/07257-3, 16/19947-6, 17/16144-2)
SAIRE, DARWIN; RIVERA, ADIN RAMIREZ. Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task. IEEE ACCESS, v. 9, p. 80654-80670, . (19/18678-0, 19/07257-3, 17/16597-7)

Please report errors in scientific publications list using this form.