It is common that when we try out hive hbase integration it leads to lots of unexpected errors even though hive and hbase are running individually without any issues. Some common issues are
1) jars not available
The following jars should be available on hive auxpath
- usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar
- /usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar
- /usr/lib/hive/lib/zookeeper-3.3.1.jar
These jars vary with the hbase , hive and zookeeper version running on your cluster
2)zookeeper quorum not available for hive client
Should specify the zoo keeper server host names so that the leader hbase master server can be chosen for the hbase connection
hbase.zookeeper.quorum=zk1,zk2,zk3
Where zk1,zk2,zk3 should be the actual hostnames of the ZooKeeper servers
These values can be set either on your hive session or permanently on your hive configuration files
1) Setting in hive client session
$ hive -auxpath /usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar:/usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar:/usr/lib/hive/lib/zookeeper-3.3.1.jar -hiveconf hbase.zookeeper.quorum=zk1,zk2,zk3
2) Setting in hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar,file:///usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u2.jar</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>zk1,zk2,zk3</value>
</property>
Still hive hbase integration not working
Error thrown : Master not running
Check whether the HbaseMaster is really down. If it is fine there could be other possibilities , a few common ones being
- Firstly you need to check whether hbase.zookeeper.quorum is correctly set, it shouldn’t be localhost.
- If multiple hbase clusters share the same zookeeper quorum then the znode parent value will be different for each. If that is the case, then set ‘zookeeper.znode.parent’ also has to be set in hive configuration, to the correct value from hbase-site.xml.
I suppose I've selected an unbelievable and interesting blog. Big data analytics platform
ReplyDeleteHadoop is really a good booming technology in coming days. And good path for the people who are looking for the Good Hikes and long career. We also provide Hadoop online training
ReplyDeleteYes Hadoop is the buzzing technology in the present world.HBase is the data base used by Hadoop.One website named http://www.hadooponlinetutor.com is offering the hadoop complete videos at $20 only.Really the videos are good
ReplyDeleteIn coming days hadoop is one of the best technology to migrate for all java people who ever interested and eager to get more income .
ReplyDeleteHadoop Tutorial