| Grant number: | 19/07257-3 |
| Support type: | Regular Research Grants |
| Duration: | September 01, 2020 - July 31, 2021 |
| Field of knowledge: | Physical Sciences and Mathematics - Computer Science |
| Principal researcher: | Gerberth Adín Ramírez Rivera |
| Grantee: | Gerberth Adín Ramírez Rivera |
| Home Institution: | Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil |
| Assoc. researchers: | Anderson de Rezende Rocha ; Hélio Pedrini ; Ricardo da Silva Torres |
Abstract
Automatic media generation (or synthesis) is a field that had an incredible boost in recent years, with the advent of deep generative models. Nowadays, neural networks can create text, images and videos based on class labels or other media. The common task is to generate content. However, we can take advantage of the learned feature representations on these tasks to understand the relevant features and as a source of interpretability. That is, what features are relevant for the creation of different content, and how can we interpret what the models are learning or paying attention too. In this project, we propose to investigate how to learn efficient and rich representations for video data based on deep generative tasks. We focus on two particular problems for learning effective representations. The first one is semantic transfer between data modalities, in particular video and (written) language. And the second one is disentanglement within the same domain, that is, separate different variations and modalities of the data. The separation of semantics (intra and inter domain) will allow us to better understand the type of features that are learned by the different architectures on these tasks. Our objective is to train the deep generative models on different video reconstruction tasks and study their learning capabilities. We will perform experiments on the existing benchmark datasets for the particular problems. (AU)