Algorithms, methods and tools to migrate high-performance computing science and engineering applications to the cloud

• Grant number: 19/12792-5
• Support Opportunities: Scholarships in Brazil - Post-Doctoral
• Start date: September 01, 2019
• End date: December 19, 2022
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Edson Borin
• Grantee: Rafael Keller Tesser
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated research grant: 13/08293-7 - CCES - Center for Computational Engineering and Sciences, AP.CEPID

Abstract

In the cloud computing scenario, the user may choose among several hardware configurations and prices to configure his/her high performing cluster of computers. This opens the opportunity for several optimizations, such as avoiding long waits on job queues and creating specialized clusters for each application. However, migrating the code to the cloud, selecting the most cost-effective set of resources for each application and dealing with performance fluctuations on virtual network infrastructures are still challenges that must be tackled [1].To mitigate these network performance problems, some providers are offering specialized HPC services on the cloud. These services provide guarantees on network performance, nonetheless, they are more expensive and usually limited to a maximum number of machines that can be rented. While some high-performance applications may benefit from these extra guarantees, others are tolerant of network performance variations. Hence, selecting the most cost-effective service for each application may be a challenge itself.High-performance science and engineering applications may depend on multiple software packages, including specialized libraries. However, in many cases, installing and configuring these packages on new systems may be challenging. In this sense, it is crucial to investigate technologies to ease the migration of high-performance programs from workstations and local clusters to the cloud.The main goal here is to investigate techniques and methods to ease the migration of existing high-performance computing applications to the cloud and to optimize the execution of high-performance applications on cloud resources. To facilitate code migration, we plan to investigate the use of container technologies [2].[1] Netto, M. A. S., Calheiros, R. N., Rodrigues, E. R., Cunha, R. L. F., Buyya, R. HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges. ACM Computing Surveys 51 (2018).[2] Pahl, C., Brogi, A., Soldani, J., and Jamshidi, P. Cloud Container Technologies: a State- of-the-Art Review. IEEE Transactions on Cloud Computing (2017).

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

TESSER, RAFAEL KELLER; BORIN, EDSON. Containers in HPC: a survey. JOURNAL OF SUPERCOMPUTING, v. 79, n. 5, p. 69-pg., 2022-10-27. (13/08293-7, 19/12792-5)

TESSER, RAFAEL KELLER; MARQUES, ALVARO; BORIN, EDSON; IEEE COMP SOC. Selecting efficient VM types to train deep learning models on Amazon SageMaker. 2021 IEEE 33RD INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE AND HIGH PERFORMANCE COMPUTING WORKSHOPS (SBAC-PADW 2021), v. N/A, p. 8-pg., 2021-01-01. (13/08293-7, 19/12792-5)