Energy-aware Coflow Scheduling for Sustainable Workload Management

Abstract

Handling High-Performance Computing (HPC) workflows often requires the orchestration of a collection of parallel flows. Traditional techniques to optimize flow-level metrics do not perform well in optimizing such collections because the network is usually agnostic to application requirements. A Coflow is a recently proposed abstraction that created new opportunities in network scheduling for datacenter networks. However, recent work on coflow scheduling has focused on merely two objectives: decreasing communication time of data-intensive jobs and guaranteeing predictable communication time. In this paper, we take a step further and propose some initial results towards the design of heuristics that optimize also the energy consumption of a data center that hosts HPC jobs. To this aim, we built and released an energy-aware coflow scheduling simulator to the community that helps analyze the tradeoff between energy efficiency and coflow completion time. We also propose two scheduling algorithms that consider coflow completion time, CPU utilization, and energy consumption efficiency. Our initial results using the simulator clarify how each policy should be tuned to the application needs and the computational resources available.