| Full text | |
| Author(s): |
Stival, Leandro
;
Torres, Ricardo da Silva
;
Pedrini, Helio
Total Authors: 3
|
| Document type: | Journal article |
| Source: | INTELLIGENT SYSTEMS AND APPLICATIONS, VOL 1, INTELLISYS 2024; v. 1065, p. 15-pg., 2024-01-01. |
| Abstract | |
Cutting-edge techniques are being employed by researchers to develop algorithms that have the capability to automatically add color to black-and-white videos. This advancement has the potential to revolutionize our experience of historical films and provide filmmakers and video producers with a powerful new tool. These algorithms employ sophisticated deep neural networks to analyze images, identifying patterns and offering a promising avenue for extracting meaning and insights from visual data in the field of computer vision. Although current studies primarily focus on image colorization, there is a noticeable gap when it comes to videos and movies in the realm of deep machine learning techniques. Our investigation aims to bridge this gap and demonstrate that the image colorization techniques used today can also be effectively applied to videos and match the current state of the art presented at NTIRE 2023 video colorization challenge. We explored the application of diffusion models, which have gained popularity due to their ability to generate images and text. Our implementation involves utilizing a diffusion model to introduce noise in the frames, while a U-Net with self-attention layers predicts the denoised frames, thereby predicting the color of the video frames. For training purposes, we utilized the DAVIS and LDV datasets. When comparing the colorized frames with the ground truth in the test set, we observed promising results under several quality metrics, such as PSNR, SSIM, FID, and CDC. (AU) | |
| FAPESP's process: | 23/11556-1 - Novel deep learning methods for remote sensing imagery |
| Grantee: | Leandro Stival |
| Support Opportunities: | Scholarships abroad - Research Internship - Doctorate |
| FAPESP's process: | 22/12294-8 - Convolutional Networks with Attention for Video Color Propagation |
| Grantee: | Leandro Stival |
| Support Opportunities: | Scholarships in Brazil - Doctorate |