Load-balancing function offloading on SmartNIC in Microservice frameworks using heterogeneous computing units

• Grant number: 23/05183-8
• Support Opportunities: Scholarships in Brazil - Post-Doctoral
• Effective date (Start): October 01, 2023
• Effective date (End): September 30, 2025
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Fabio Luciano Verdi
• Grantee: Alireza Shirmarz
• Host Institution: Centro de Ciências em Gestão e Tecnologia (CCGT). Universidade Federal de São Carlos (UFSCAR). Campus de Sorocaba. Sorocaba , SP, Brazil
• Host Company: Universidade Estadual de Campinas (UNICAMP). Faculdade de Engenharia Elétrica e de Computação (FEEC)
• Associated research grant: 21/00199-8 - SMART NEtworks and ServiceS for 2030 (SMARTNESS), AP.PCPE

Abstract

These days, fine-grained software applications motivate microservice frameworks to be adopted by a wide range of usages. The software load balancing function in the microservice framework allocates the server's incoming packets to each pod (node). We will develop the research considering three key aspects: (1) the load balancing function is implemented and deployed entirely in a SmartNIC; (2) there is a set of heterogeneous processing units, including mainly CPU and GPUs; (3) the load balancing function must be aware of two key issues: (a) the load balancer must know the load in the processing units so that the traffic is sent to the Pods based on that load and (b) must be aware of the type of traffic entering in the data center. This last aspect means that depending on the traffic, the load balancing may decide to send it to the CPU or GPU. Typically, XR/AR and cloud gaming types of traffic should be sent to the GPU. Otherwise, the traffic should be forwarded to the CPU units. The challenge here is how to identify the type of traffic considering that it is encrypted. To solve this problem, we intend to use Machine Learning (ML) algorithms to classify the type of traffic and jointly work with the load balancer to improve performance and save energy. In summary, in this work, we take a step further to make a smart load balancer to allocate the incoming packets to the appropriate processing unit in the microservice framework having heterogeneous processing units. (AU)