Busca avançada
Ano de início
Entree


Towards a Scalable and Cost Efficient Serverless Scheduler for Dask

Texto completo
Autor(es):
Millani, Carlos Eduardo ; Astudillo, Carlos A. ; Borin, Edson
Número total de Autores: 3
Tipo de documento: Artigo Científico
Fonte: 2024 IEEE/ACM 17TH INTERNATIONAL CONFERENCE ON UTILITY AND CLOUD COMPUTING, UCC; v. N/A, p. 6-pg., 2024-01-01.
Resumo

HPC workflows run either on supercomputers, shared through queues, or Cloud virtual machines, allocated for the experiment. Both options are ill-suited for exploration scenarios, where the user explores the dataset by applying small computations to parts of a much bigger dataset and analyzing the result, either because of the queue time present on supercomputers or costs generated even when idle on traditional IaaS cloud offerings. We propose FaaS-Dask, a runtime for Dask that is capable of running on FaaS offerings, providing the same tailored experience as cloud virtual machines but billing only for executed computation. We will explore current FaaS challenges and optimizations available, and compare execution on our environment with traditional cloud Dask clusters, showing that FaaS-Dask is cheaper in some scenarios, but there is still work to be done to make it as fast as current options. (AU)

Processo FAPESP: 13/08293-7 - CECC - Centro de Engenharia e Ciências Computacionais
Beneficiário:Munir Salomao Skaf
Modalidade de apoio: Auxílio à Pesquisa - Centros de Pesquisa, Inovação e Difusão - CEPIDs