Installation of Apache Hadoop Cluster and High Availability

Apache Hadoop cluster configuration on Hadoop 2.3.0 with high availability,  complete installation and configuration has been divided into three  chapters as below.

Chapter 1            : Installing Hadoop 2.3.0

Click here for Chapter 1

Chapter 2            : Configuring Hadoop cluster high availability.

Click here for Chapter 2

Chapter 3            : Test the failover.

Click here for Chapter 3

Quorum Journal Manager or  Shared Storage(NFS share)

Method : Quorum Journal Manager

In Hadoop 1. X  there was no automatic failover and no high availability for name node service, namenode service is responsible to maintain metadata.  In Hadoop 2. x elimination of single point of failure , whereas in Hadoop 1.x the basic draw back was a single point of failure for namenode.

In our installation guide for Hadoop 2.3 with high availability which covers all steps required to install and configure Hadoop 2nd generation with high availability.

The method  we are going  to be showing using QuorumJournalManagar.

In our guide we are going configure 3 node cluster, the details are as follows.

server1        :mn1  (Master Node : namenode  , datanode ,

resource manager, DFSZKFailoverController,JournalNode ,

ResourceManager,NodeManager & QuorumPeerMain)

server2       :mn2 (Slave Node: namenode  , datanode ,

DFSZKFailoverController,JournalNode ,NodeManager & QuorumPeerMain)

server3       :dn1(datanode, QuorumPeerMain, & JournalNode)

 

Software packages and dependencies

hadoop-2.3.0 , zookeeper-3.4.5 &  jdk-7u51-linux-i586.rpm

 

User account in all the cluster nodes

hduser

Network and Hostname details

mn1    : 192.168.1.39

mn2    : 192.168.1.57

dn1    : 192.168.1.72

Operating system and arch.

OS: Centos 6.5

Arch: 32

Thank You.

For more Detail you can watch video and also subscribe for more Videos :

Both comments and pings are currently closed.

Comments are closed.

Copyright ©Solutions@Experts.com
Copyright © NewWpThemes Techmark Solutions - www.techmarksolutions.co.uk