Apache Hadoop cluster configuration on Hadoop 2.3.0 with high availability, complete installation and configuration has been divided into three chapters as below.
Chapter 1 : Installing Hadoop 2.3.0
[button size=”large” type=”link” value=”Click here for Chapter 1″ href=”http://demo.com/installing-hadoop-2-3-0-chapter-1/”]
Chapter 2 : Configuring Hadoop cluster high availability.
[button size=”large” type=”link” value=”Click here for Chapter 2″ href=”http://demo.com/configuring-apache-hadoop-cluster-high-availability-chapter-2/”]
Chapter 3 : Test the failover.
[button size=”large” type=”link” value=”Click here for Chapter 3″ href=”http://demo.com/test-the-failover-chapter-3/”]
Quorum Journal Manager or Shared Storage(NFS share)
Method : Quorum Journal Manager
In Hadoop 1. X there was no automatic failover and no high availability for name node service, namenode service is responsible to maintain metadata. In Hadoop 2. x elimination of single point of failure , whereas in Hadoop 1.x the basic draw back was a single point of failure for namenode.
In our installation guide for Hadoop 2.3 with high availability which covers all steps required to install and configure Hadoop 2nd generation with high availability.
The method we are going to be showing using QuorumJournalManagar.
In our guide we are going configure 3 node cluster, the details are as follows.
server1 :mn1 (Master Node : namenode , datanode ,
resource manager, DFSZKFailoverController,JournalNode ,
ResourceManager,NodeManager & QuorumPeerMain)
server2 :mn2 (Slave Node: namenode , datanode ,
DFSZKFailoverController,JournalNode ,NodeManager & QuorumPeerMain)
server3 :dn1(datanode, QuorumPeerMain, & JournalNode)
Software packages and dependencies
hadoop-2.3.0 , zookeeper-3.4.5 & jdk-7u51-linux-i586.rpm
User account in all the cluster nodes
Network and Hostname details
mn1 : 192.168.1.39
mn2 : 192.168.1.57
dn1 : 192.168.1.72
Operating system and arch.
OS: Centos 6.5
For more Detail you can watch video and also subscribe for more Videos :