| Grant number: | 22/12294-8 |
| Support Opportunities: | Scholarships in Brazil - Doctorate |
| Start date: | April 01, 2023 |
| End date: | August 08, 2026 |
| Field of knowledge: | Physical Sciences and Mathematics - Computer Science - Computer Systems |
| Principal Investigator: | Hélio Pedrini |
| Grantee: | Leandro Stival |
| Host Institution: | Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil |
| Associated scholarship(s): | 23/11556-1 - Novel deep learning methods for remote sensing imagery, BE.EP.DR |
Abstract The colorization process consists of reconstituting the colors of images or videos that did not capture this information at the time of obtaining the pixel information. In this sense, the color restoration of images and videos was originally done manually, which required an excess of hours in the coloring process and was something very prone to failure. With the emergence of digital images and videos, the colorization process was computerized, thus streamlining the process, however, the development remained expensive and manual.This way, this project will create a pipeline capable of predicting the color of a pixel in the original video when only a sample color image is available. This type of tool is very important for many practical applications, such as restoring old videos, remote sensing, x-rays, and coloring animations. And since there is still no optimal and defined solution to this problem, this research is more than necessary and current.The prediction method proposed in this project utilizes the latest techniques in machine learning, such as a U-net topology with an encoder and decoder, where the bottleneck will be implemented using a model based on ViT (Visual Transformer) to maintain the temporal context.The main difference between this project and the current works present in the literature goes beyond the ViT + U-net architecture. However, the choice to use keyframes allows for the identification of sudden changes in the scenes. This improves the temporal consistency of the process by individually coloring each scene based on its keyframe.The entire pipeline will be trained on a dataset created from videos from the Videvo website. Many short videos are available on this website without copywriting, which is a common technique among other works in the colorization literature.It is expected, at the end of the project, to obtain a robust model capable of making this type of prediction, as well as to advance the knowledge of ways to create loss functions for colorization methods. These are some of the most challenging aspects of the literature, together with maintaining color saturation, temporal consistency, and choosing the ideal color space for this type of method. | |
| News published in Agência FAPESP Newsletter about the scholarship: | |
| More itemsLess items | |
| TITULO | |
| Articles published in other media outlets ( ): | |
| More itemsLess items | |
| VEICULO: TITULO (DATA) | |
| VEICULO: TITULO (DATA) | |