Title:
|
Cost model for Pregel on GraphX
|
Author:
|
Kumar, Rohit; Abelló Gamazo, Alberto; Carders, Toon
|
Other authors:
|
Universitat Politècnica de Catalunya. Departament d'Enginyeria de Serveis i Sistemes d'Informació; Universitat Politècnica de Catalunya. inSSIDE - integrated Software, Service, Information and Data Engineering |
Abstract:
|
The graph partitioning strategy plays a vital role in the overall execution of an algorithm in a distributed graph processing system. Choosing the best strategy is very challenging, as no one strategy is always the best fit for all kinds of graphs or algorithms. In this paper, we help users choosing a suitable partitioning strategy for algorithms based on the Pregel model by providing a cost model for the Pregel implementation in Spark-GraphX. The cost model shows the relationship between four major parameters: (1) input graph (2) cluster configuration (3) algorithm properties and (4) partitioning strategy. We validate the accuracy of the cost model on 17 different combinations of input graph, algorithm, and partition strategy. As such, the cost model can serve as a basis for yet to be developed optimizers for Pregel. |
Abstract:
|
Peer Reviewed |
Subject(s):
|
-Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació -Àrees temàtiques de la UPC::Informàtica::Informàtica teòrica -Algorithms -Algorithm properties -Cluster configurations -Cost modeling -Graph partitioning -Graph processing -Input graphs -Overall execution -Partitioning strategies -Algorismes |
Rights:
|
|
Document type:
|
Article - Submitted version Conference Object |
Published by:
|
Springer
|
Share:
|
|