Analysis of Potential Online Scheduling Improvements by Real-Time Strategy Selection

Author(s)

Sant'Ana, Luis Felipe ; Carastan-Santos, Danilo ; Cordeiro, Daniel ; de Camargo, Raphael Y. ; Bianchini, CD ; DeSouza, PSL ; DeBarros, COF ; Ferreira, RAC

• Document type: Journal article
• Source: 2018 SYMPOSIUM ON HIGH PERFORMANCE COMPUTING SYSTEMS (WSCAD 2018); v. N/A, p. 7-pg., 2018-01-01.

Abstract

Task Scheduling in large-scale HPC platforms is normally accomplished with simple heuristics combined with a backfilling algorithm. Some strategies, such as the First-Come First-Served (FCFS) with backfilling, provide reasonable results in a variety of scenarios, including different HPC platforms and task set characteristics. But for each scenario, a different strategy might be the most appropriate for minimizing some metric, such the as the average task waiting time or turnaround time. In this work, we evaluate the effects of choosing different scheduling strategies over sub-sequences of workload logs of 6 real HPC platforms, for periods from 1 to 24 hours. For each platform and workload period, we show that the performance of each scheduling strategy have large variations for different workload sub-sequences. Similarly, the best scheduling strategy for each sub-sequence also varied. Finally, we show that, if one could select the best strategy for each workload sub-sequence, it would significantly reduce the scheduling performance variations and improve the mean queue waiting time by more than 50% for most cases. These results indicate that the development of heuristics or machine learning algorithms for selecting the best scheduling strategy every 6 or 24 hours, can result in significant improvements in the mean queue waiting time of tasks in HPC platforms. (AU)

• FAPESP's process: 13/26644-1 - Algorithms and programming models for efficient execution of parallel applications in heterogeneous clusters
• Grantee: Raphael Yokoingawa de Camargo
• Support Opportunities: Regular Research Grants