Title:
|
MapReduce performance models for Hadoop 2.x
|
Author:
|
Glushkova, Daria; Jovanovic, Petar; Abelló Gamazo, Alberto
|
Other authors:
|
Universitat Politècnica de Catalunya. Departament d'Enginyeria de Serveis i Sistemes d'Informació; Universitat Politècnica de Catalunya. inSSIDE - integrated Software, Service, Information and Data Engineering |
Abstract:
|
MapReduce is a popular programming model for distributed processing of large data sets. Apache Hadoop is one of the most common open-source implementations of such paradigm. Performance analysis of concurrent job executions has been recognized as a challenging problem, at the same time, that it may provide reasonably accurate job response time at significantly lower cost than experimental evaluation of real setups. In this paper, we tackle the challenge of defining MapReduce
performance models for Hadoop 2.x. While there are several efficient approaches for modeling the performance of MapReduce workloads in Hadoop 1.x, the fundamental architectural changes of Hadoop 2.x require that the cost models are also reconsidered. The proposed solution is based on
an existing performance model for Hadoop 1.x, but it takes into consideration the architectural changes of Hadoop 2.x and captures the execution flow of a MapReduce job by using queuing network model. This way the cost model adheres to the intra-job synchronization constraints that occur due the contention at shared resources. The accuracy of our solution is validated via comparison of our model estimates against measurements in a real Hadoop 2.x setup. According to our evaluation results, the proposed model produces estimates of average job response time with
error within the range of 11% - 13.5%. |
Abstract:
|
Peer Reviewed |
Subject(s):
|
-Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació -Electronic data processing -- Distributed processing -Cost effectiveness -Open source software -MapReduce performance models -Hadoop 2.x -Queuing theory -Mean value analysis -Processament distribuït de dades -Cost-eficàcia -Programari lliure |
Rights:
|
Attribution-NonCommercial-NoDerivs 3.0 Spain
http://creativecommons.org/licenses/by-nc-nd/3.0/es/ |
Document type:
|
Article - Published version Conference Object |
Published by:
|
CEUR-WS.org
|
Share:
|
|