| Texto completo | |
| Autor(es): |
Albarracin, Juan F. Hernandez
;
Ramirez Rivera, Adin
Número total de Autores: 2
|
| Tipo de documento: | Artigo Científico |
| Fonte: | IEEE Transactions on Image Processing; v. 31, p. 10-pg., 2022-01-01. |
| Resumo | |
Independent components within low-dimensional representations are essential inputs in several downstream tasks, and provide explanations over the observed data. Video-based disentangled factors of variation provide low-dimensional representations that can be identified and used to feed task-specific models. We introduce MTC-VAE, a self-supervised motion-transfer VAE model to disentangle motion and content from videos. Unlike previous work on video content-motion disentanglement, we adopt a chunk-wise modeling approach and take advantage of the motion information contained in spatiotemporal neighborhoods. Our model yields independent per-chunk representations that preserve temporal consistency. Hence, we reconstruct whole videos in a single forward-pass. We extend the ELBO's log-likelihood term and include a Blind Reenactment Loss as an inductive bias to leverage motion disentanglement, under the assumption that swapping motion features yields reenactment between two videos. We evaluate our model with recently-proposed disentanglement metrics and show that it outperforms a variety of methods for video motion-content disentanglement. Experiments on video reenactment show the effectiveness of our disentanglement in the input space where our model outperforms the baselines in reconstruction quality and motion alignment. (AU) | |
| Processo FAPESP: | 17/16144-2 - Transferência de dinâmica de vídeo para vídeo com modelos generativos profundos |
| Beneficiário: | Juan Felipe Hernández Albarracín |
| Modalidade de apoio: | Bolsas no Brasil - Doutorado |
| Processo FAPESP: | 19/07257-3 - Aprendendo representações através de modelos generativos profundos em vídeo |
| Beneficiário: | Gerberth Adín Ramírez Rivera |
| Modalidade de apoio: | Auxílio à Pesquisa - Regular |