<abstract xmlns="http://www.w3.org/1999/xhtml"><p> Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such as locality of data, user share fairness, and resource awareness. This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave nodes. The experiments include variation in data sizes, use of two different data processing applications, and variation in the number of nodes used in processing. The article analyzes the performance of the job scheduling algorithms based on various relevant performance measures. The results of the experiments are evident of the performance being affected by the job scheduling parameters, the type of applications, the number of nodes in the cluster, and size of the input data.</p></abstract>

Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such as locality of data, user share fairness, and resource awareness. This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave nodes. The experiments include variation in data sizes, use of two different data processing applications, and variation in the number of nodes used in processing. The article analyzes the performance of the job scheduling algorithms based on various relevant performance measures. The results of the experiments are evident of the performance being affected by the job scheduling parameters, the type of applications, the number of nodes in the cluster, and size of the input data.

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Department of Information Technology, Dharmsinh Desai University, Nadiad,

Institute of Engineering & Technology, Ahmedabad University, Ahmedabad,

Cybernetics and Information Technologies

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce"}

Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such...

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Publié en ligne: 06 avr. 2017

Pages: 146 - 163

DOI: https://doi.org/10.1515/cait-2017-0012

Mots clés
Big Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation

© by Jyoti V. Gautam

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Publié en ligne: 06 avr. 2017

Pages: 146 - 163

DOI: https://doi.org/10.1515/cait-2017-0012

Mots clésBig Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation

© by Jyoti V. Gautam

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Mots clés
Big Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation