Dealing with Memory Constraints in a Non-Dedicated Linux Cluster

Abstract

Our research is focused on keeping both local and parallel jobs together in a non-dedicated cluster and scheduling them efficiently. In such a system, memory becomes a critical resource for both kinds of job. Thus, the minimization of the impact caused by the overflow of physical into virtual memory on the performance of distributed jobs together with the development of an efficient memory management adapted to the needs of NOWs (Networks of Workstations) are major issues in this kind of environment. An algorithm is presented to adjust time slices dynamically to the necessities of distributed and local tasks in order to reduce the number of page faults across the cluster. Additionally, a memory management policy aimed at preserving enough memory for both workloads is evaluated. Our proposals are implemented in a Linux cluster and compared with alternative algorithms.

This work was supported by the MCyT under contract TIC 2001-2592 and partially supported by the Generalitat de Catalunya – Grup de Recerca Consolidat 2001SGR- 00218.