Hive Database Installation on Centos

Hive is an Apache Hive data warehouse software for querying and managing a large dataset store in distributed storage  (hdfs, Hadoop). Hive provides a platform to Project structure onto this data and query the data using a language called HIVEQL.

Step 1: System Requirements and Dependencies

Step 2: Download and Install Hive

Step 3: Copy .xml files

Step 4: Download ODBC

Step 5: Create database and user permissions

Step 6: Modifying Hive Configuration files

Step 7: Start hadoop and mysql services

Step 8: Start Thrift Hive Server

To complete Hive Database Installation steps follow the below process:

Step 1: System Requirements and Dependencies

Hadoop single / multimode Cluster (started mode): For Hadoop Installation steps follow this hyperlink

Mysql-server (started mode)

Hive Installation and configuration guide has prepared on following platform and services.

Operating System        : CentOS release 6.4 (Final) 64bit

hadoop version          : hadoop-1.2.1

Mysql-server            : 5.1.71

JDK                     : java version “1.7.0_45″

Hadoop single / multimode Cluster (started mode)

Mysql-server (started mode)

Login to hadoop master / slave server using hadoop user account.

Step 2: Download and Install Hive

Download hive version as per your requirement

#wget http://archive.apache.org/dist/hive/hive-0.9.0/hive-0.9.0.tar.gz

Extract hive-0.9.0.tar.gz

#tar zxvf hive-0.9.0.tar.gz

Change directory structure to configure hive configuration files

#cd hive-0.9.0/conf

Rename / copy following sample configuration files

#pwd

#/home/hadoop/hive-0.9.0/conf

#ls

-rw-rw-r-- 1 hadoop hadoop 48930 Apr 25  2012 hive-default.xml.template
-rw-rw-r-- 1 hadoop hadoop  2378 Apr 25  2012 hive-env.sh.template
-rw-rw-r-- 1 hadoop hadoop  2422 Apr 25  2012 hive-exec-log4j.properties.template
-rw-rw-r-- 1 hadoop hadoop  2828 Apr 25  2012 hive-log4j.properties.template

Step 3: Copy  .xml files

#cp -rp hive-default.xml.template hive-site.xml

#cp -rp hive-env.sh.template hive-env.sh

Step 4: Download ODBC

Go back to lib folder structure to download mysql odbc drivers required to store metastore information in mysql

#cd ../lib/

#wget http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.1.27.tar.gz

Extract mysql drivers

#tar -zxvf mysql-connector-java-5.1.27.tar.gz

copy mysql-connector-java-5.1.27-bin.jar from mysql-connector-java-5.1.27 folder to lib/

#cp -rp mysql-connector-java-5.1.27 mysql-connector-java-5.1.27-bin.jar

#ll

login to mysql server terminal console to create database and user permissions for hive metastore

Step 5: Create database and user permissions

#mysql  -u root -p

mysql> create database hivedemo;

Query OK, 1 row affected (0.00 sec)

mysql>GRANT ALL PRIVILEGES ON hivedemo.* TO ‘hiveuser’@'%’ IDENTIFIED BY ‘hivepassword’;

mysql>Exit;

Test by login user hiveruser account created in above step

#mysql -u hiveuser -p

Step 6: Modifying Hive Configuration files 

Modifying Hive Configuration files located /home/hadoop/hive-0.9.0/conf

#cd /home/hadoop/hive-0.9.0/conf

Add hadoop home foler path  in hive-env.sh file at bottom of page

#vi hive-env.sh

export HADOOP_HOME=/home/hadoop/hadoop-1.2.1

Save and Exit.

“hive-site.xml is main configuration file to control hive, change the settings for  mysql database and other configuration in hive-site.xml file”

#vi hive-site.xml

Replace following context “from “<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>“Repalce to “<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hivedemo?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>Replace following context “from

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
<description>Driver class name for a JDBC metastore</description>
</property>

“Replace to “

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

“Modify mysql database user name”

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>username to use against metastore database</description>
</property>

“Modify mysql database password”

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hivepassword</value>
<description>password to use against metastore database</description>
</property>

Save and Exit.

Step 7: Start hive services

Before running ./hive script to invoke hive terminal console make sure to start hadoop and mysql services.

#cd /home/hadoop/hive-0.9.0/bin/

#./hive

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/home/hadoop/hive-0.9.0/lib/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/hadoop/hive_job_log_hadoop_201312121621_1494929084.txt
hive>
hive> show databases;
OK
default
Time taken: 2.798 seconds
hive> create database demo;
OK
Time taken: 2.659 seconds
hive>exit;

Step 8: start Thrift Hive Server

#/home/hadoop/hive-0.9.0/bin/hive –service hiveserver

Note : The HIVE_PORT is assumed to be 10000 on localhost if not specified.

Both comments and pings are currently closed.

Comments are closed.

Copyright ©Solutions@Experts.com
Copyright © NewWpThemes Techmark Solutions - www.techmarksolutions.co.uk