Real-Time Scheduling Policy Selection from Queue and Machine States

Author(s)

Sant'Ana, Luis ; de Camargo, Raphael ; Carastan-Santos, Danilo ; Cordeiro, Daniel ; IEEE

• Document type: Journal article
• Source: 2019 19TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID); v. N/A, p. 10-pg., 2019-01-01.

Abstract

Task Scheduling in large-scale HPC platforms is normally accomplished with simple heuristics combined with a backfilling algorithm. Some strategies, such as the First-Come First-Serve (FCFS) with backfilling, provide reasonable results in a variety of scenarios, including different IIPC platforms and task set characteristics. But for each scenario, a different strategy might be the most appropriate for minimizing some metric, such as the average task waiting time or turnaround time. In this work, we present a real-time scheduling policy selection algorithm, which takes as input the running queue job characteristics and machine states. We evaluated the use of logistic regression and support-vector machines to perform the mapping from queue and machine state to selected scheduling policy. The machine learning algorithms are trained and evaluated using simulations configured using IIPC platform traces. When selecting among S (eight) scheduling policies, we obtained an accuracy above 80%, when compared to the best selection. When simulating the online real-time selection of policies for a period of one year, we obtained a reduction in the mean queue waiting time of tasks of up to 40% over using FCFS and 10% over randomly selecting policies. Moreover, the method performed close the best possible selection of policies, with a maximum of 9% increase in the mean queue waiting tune. (AU)

• FAPESP's process: 13/26644-1 - Algorithms and programming models for efficient execution of parallel applications in heterogeneous clusters
• Grantee: Raphael Yokoingawa de Camargo
• Support Opportunities: Regular Research Grants