Applying Deep-Learning Techniques to Parallel Processes Scheduling within Heterogeneous Distributed Computational Environments

Abstract

Many companies, organizations and/or universities have accumulated a large number of computing resources grouped in clusters. Cluster Federated Environments arise as a new architecture to take advantage of such resources by joining them and increasing the computing capacity. With this, a new problem appears, a high number of machines and computing resources implies a huge amount of energy consumption. The scheduling process, responsible for allocating the applications to the system resources, offers the possibility of improve the resource management increasing the system performance and energy efficiency. A considerably amount of research has been done in this field, but given the scheduling problem is classified as an NP problem it is difficult to determine which method is the best one for it. From the point of view of the artificial intelligence, the advances in hardware and the recent increasing research in this field and specifically in algorithms for deep learning, neural networks have become an essential tool for the research in other scopes such as pattern recognition, system identification or medical diagnosis. Even though some of those problems are too complex to get an optimal solution neural networks have demonstrated give a good enough solutions. In this Degrees Final Project (DFP) we will try to find if it is possible to use neural networks to find which is the best heuristic to allocate the applications arriving to the system, obtaining the best results in terms of time and energy consumption. To build the neural network we will make use of an open source library named Tensorflow developed by Google Brain Team and to execute the workloads with a variety of different techniques we will use ClusterGridSim, a Java application that simulates the allocation using the desired technique.