PLB-HAC: Dynamic Load-Balancing for Heterogeneous Accelerator Clusters

Author(s)

Sant'Ana, Luis ; Cordeiro, Daniel ; de Camargo, Raphael Y. ; Yahyapour, R

• Document type: Journal article
• Source: EURO-PAR 2019: PARALLEL PROCESSING; v. 11725, p. 13-pg., 2019-01-01.

Abstract

Efficient usage of Heterogeneous clusters containing combinations of CPUs and accelerators, such as GPUs and Xeon Phi boards requires balancing the computational load among them. Their relative processing speed for each target application is not available in advance and must be computed at runtime. Also, dynamic changes in the environment may cause these processing speeds to change during execution. We propose a Profile-based Load-Balancing algorithm for Heterogeneous Accelerator Clusters (PLB-HAC), which constructs a performance curve model for each resource at runtime and continuously adapt it to changing conditions. It dispatches execution blocks asynchronously, preventing synchronization overheads and other idleness periods due to imbalances. We evaluated the algorithm using data clustering, matrix multiplication, and bioinformatics applications and compared with existing load-balancing algorithms. PLB-HAC obtained the highest performance gains with more heterogeneous clusters and larger problems sizes, where a more refined load-distribution is required. (AU)

• FAPESP's process: 14/50937-1 - INCT 2014: on the Internet of the Future
• Grantee: Fabio Kon
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 13/26644-1 - Algorithms and programming models for efficient execution of parallel applications in heterogeneous clusters
• Grantee: Raphael Yokoingawa de Camargo
• Support Opportunities: Regular Research Grants
• FAPESP's process: 15/24485-9 - Future internet for smart cities
• Grantee: Fabio Kon
• Support Opportunities: Research Projects - Thematic Grants