Real-time adaptive streaming for VR applications using Self-Attention Generative Networks (SAGAN)

• Grant number: 25/01929-0
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: April 01, 2025
• End date: February 29, 2028
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Fabio Luciano Verdi
• Grantee: Públio Elon Correa da Silva
• Host Institution: Centro de Ciências em Gestão e Tecnologia (CCGT). Universidade Federal de São Carlos (UFSCAR). Campus de Sorocaba. Sorocaba , SP, Brazil
• Company: Universidade Estadual de Campinas (UNICAMP). Faculdade de Engenharia Elétrica e de Computação (FEEC)
• Associated research grant: 21/00199-8 - SMART NEtworks and ServiceS for 2030 (SMARTNESS), AP.PCPE

Abstract

The recent advancements in cellular networks, particularly during the COVID-19 pandemic, have significantly accelerated the development of virtual reality (VR) and augmented reality (AR) applications. This surge has been evident in fields such as tele-education, telemedicine, skills acquisition, and gamification, concurrently escalating the demand for high-resolution video content. This, in turn, increases the bandwidth requirements and the Quality of Experience (QoE), particularly highlighting the challenge of streaming 360-degree images where the spatial resolution and image size significantly strain bandwidth and latency. In this context, programmable and acceleration hardware such as FPGAs, GPUs, smartNICs and switches, located at the edge of the network, have gained considerable interest due to their ability to reduce end-to-end latency by handling compute-intensive tasks. Additionally, the integration of edge computing with Generative Artificial Intelligence (AI) has shown immense potential in creating data based on Machine Learning (ML) models. In this PhD project, we aim to utilize Generative Adversarial Networks (GANs) to selectively enhance image resolution at the network edge. Specifically, we plan to investigate Self-Attention GANs for Video Super-Resolution and develop a solution optimized for execution on accelerators for efficient AR/VR task processing.