文档章节

Eclipse调试HBase程序需要注意的地方

华兹格
 华兹格
发布于 2013/06/17 15:57
字数 2373
阅读 182
收藏 0

网上关于HBase的代码都比较多,但是我们在实际运行中多少会出现问题,现在把我遇到的问题总结一下

1.HBase所需jar的引入

从网上粘贴的代码,放在编辑器里会提示所需的包不存在。这个时候建议给HBase做成用户库,以便下次再使用的话直接使用。此库里的jar包包括: hbase下的hbase.jar,hbase-test.jar以及lib目录下的所有。

2.Eclipse调试时报:unknown host: Master.Hadoop错误

这个错误很明显,就是找不到Master.Hadoop主机,解决也挺简单,就是修改windows下的host文件。

一般host的位置在:C:\WINDOWS\system32\drivers\etc\hosts 

3.在网上看到的一种通过指定配置文件的调试方法:将自己所连接的HBase的配置文件hbase-site.xml添加到本工程的Classpath中,方法是:

先在工程下创建一个文件夹,可以命名为Conf,去Hbase的配置文件夹内将hbase-site.xml拷贝,粘贴至你所创建的Conf文件夹中。右击工程,Properties,Java Build Path, Libraries, Add Class Folder,勾选出Conf。

示图:

当然也可以通过硬编码的方式指定参数设置(方法见附录:一种Hbase操作的代码)。

实际运行中出现的问题:

执行代码无相应,只出现如下log信息:

13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:host.name=max-pc
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_21
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Program Files\Java\jdk1.7.0_21\jre
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.class.path=C:\Users\max\workspace\.metadata\.plugins\org.apache.hadoop.eclipse\hadoop-conf-1930663343015406040;C:\Users\max\workspace\WordCountProject\bin;E:\program\hbase\hbase-0.94.7.jar;E:\program\hbase\hbase-0.94.7-tests.jar;E:\program\hbase\lib\activation-1.1.jar;E:\program\hbase\lib\asm-3.1.jar;E:\program\hbase\lib\avro-1.5.3.jar;E:\program\hbase\lib\avro-ipc-1.5.3.jar;E:\program\hbase\lib\commons-beanutils-1.7.0.jar;E:\program\hbase\lib\commons-beanutils-core-1.8.0.jar;E:\program\hbase\lib\commons-cli-1.2.jar;E:\program\hbase\lib\commons-codec-1.4.jar;E:\program\hbase\lib\commons-collections-3.2.1.jar;E:\program\hbase\lib\commons-configuration-1.6.jar;E:\program\hbase\lib\commons-digester-1.8.jar;E:\program\hbase\lib\commons-el-1.0.jar;E:\program\hbase\lib\commons-httpclient-3.1.jar;E:\program\hbase\lib\commons-io-2.1.jar;E:\program\hbase\lib\commons-lang-2.5.jar;E:\program\hbase\lib\commons-logging-1.1.1.jar;E:\program\hbase\lib\commons-math-2.1.jar;E:\program\hbase\lib\commons-net-1.4.1.jar;E:\program\hbase\lib\core-3.1.1.jar;E:\program\hbase\lib\guava-11.0.2.jar;E:\program\hbase\lib\hadoop-core-1.0.4.jar;E:\program\hbase\lib\high-scale-lib-1.1.1.jar;E:\program\hbase\lib\httpclient-4.1.2.jar;E:\program\hbase\lib\httpcore-4.1.3.jar;E:\program\hbase\lib\jackson-core-asl-1.8.8.jar;E:\program\hbase\lib\jackson-jaxrs-1.8.8.jar;E:\program\hbase\lib\jackson-mapper-asl-1.8.8.jar;E:\program\hbase\lib\jackson-xc-1.8.8.jar;E:\program\hbase\lib\jamon-runtime-2.3.1.jar;E:\program\hbase\lib\jasper-compiler-5.5.23.jar;E:\program\hbase\lib\jasper-runtime-5.5.23.jar;E:\program\hbase\lib\jaxb-api-2.1.jar;E:\program\hbase\lib\jaxb-impl-2.2.3-1.jar;E:\program\hbase\lib\jersey-core-1.8.jar;E:\program\hbase\lib\jersey-json-1.8.jar;E:\program\hbase\lib\jersey-server-1.8.jar;E:\program\hbase\lib\jettison-1.1.jar;E:\program\hbase\lib\jetty-6.1.26.jar;E:\program\hbase\lib\jetty-util-6.1.26.jar;E:\program\hbase\lib\jruby-complete-1.6.5.jar;E:\program\hbase\lib\jsp-2.1-6.1.14.jar;E:\program\hbase\lib\jsp-api-2.1-6.1.14.jar;E:\program\hbase\lib\jsr305-1.3.9.jar;E:\program\hbase\lib\junit-4.10-HBASE-1.jar;E:\program\hbase\lib\libthrift-0.8.0.jar;E:\program\hbase\lib\log4j-1.2.16.jar;E:\program\hbase\lib\metrics-core-2.1.2.jar;E:\program\hbase\lib\netty-3.2.4.Final.jar;E:\program\hbase\lib\protobuf-java-2.4.0a.jar;E:\program\hbase\lib\servlet-api-2.5-6.1.14.jar;E:\program\hbase\lib\slf4j-api-1.4.3.jar;E:\program\hbase\lib\slf4j-log4j12-1.4.3.jar;E:\program\hbase\lib\snappy-java-1.0.3.2.jar;E:\program\hbase\lib\stax-api-1.0.1.jar;E:\program\hbase\lib\velocity-1.7.jar;E:\program\hbase\lib\xmlenc-0.52.jar;E:\program\hbase\lib\zookeeper-3.4.5.jar;E:\program\hadoop-1.0.0\lib\xmlenc-0.52.jar;E:\program\hadoop-1.0.0\lib\slf4j-log4j12-1.4.3.jar;E:\program\hadoop-1.0.0\lib\slf4j-api-1.4.3.jar;E:\program\hadoop-1.0.0\lib\servlet-api-2.5-20081211.jar;E:\program\hadoop-1.0.0\lib\oro-2.0.8.jar;E:\program\hadoop-1.0.0\lib\mockito-all-1.8.5.jar;E:\program\hadoop-1.0.0\lib\log4j-1.2.15.jar;E:\program\hadoop-1.0.0\lib\kfs-0.2.2.jar;E:\program\hadoop-1.0.0\lib\junit-4.5.jar;E:\program\hadoop-1.0.0\lib\jsch-0.1.42.jar;E:\program\hadoop-1.0.0\lib\jetty-util-6.1.26.jar;E:\program\hadoop-1.0.0\lib\jetty-6.1.26.jar;E:\program\hadoop-1.0.0\lib\jets3t-0.6.1.jar;E:\program\hadoop-1.0.0\lib\jersey-server-1.8.jar;E:\program\hadoop-1.0.0\lib\jersey-json-1.8.jar;E:\program\hadoop-1.0.0\lib\jersey-core-1.8.jar;E:\program\hadoop-1.0.0\lib\jdeb-0.8.jar;E:\program\hadoop-1.0.0\lib\jasper-runtime-5.5.12.jar;E:\program\hadoop-1.0.0\lib\jasper-compiler-5.5.12.jar;E:\program\hadoop-1.0.0\lib\jackson-mapper-asl-1.0.1.jar;E:\program\hadoop-1.0.0\lib\jackson-core-asl-1.0.1.jar;E:\program\hadoop-1.0.0\lib\hsqldb-1.8.0.10.jar;E:\program\hadoop-1.0.0\lib\hadoop-thriftfs-1.0.0.jar;E:\program\hadoop-1.0.0\lib\hadoop-fairscheduler-1.0.0.jar;E:\program\hadoop-1.0.0\lib\hadoop-capacity-scheduler-1.0.0.jar;E:\program\hadoop-1.0.0\lib\core-3.1.1.jar;E:\program\hadoop-1.0.0\lib\commons-net-1.4.1.jar;E:\program\hadoop-1.0.0\lib\commons-math-2.1.jar;E:\program\hadoop-1.0.0\lib\commons-logging-api-1.0.4.jar;E:\program\hadoop-1.0.0\lib\commons-logging-1.1.1.jar;E:\program\hadoop-1.0.0\lib\commons-lang-2.4.jar;E:\program\hadoop-1.0.0\lib\commons-httpclient-3.0.1.jar;E:\program\hadoop-1.0.0\lib\commons-el-1.0.jar;E:\program\hadoop-1.0.0\lib\commons-digester-1.8.jar;E:\program\hadoop-1.0.0\lib\commons-daemon-1.0.1.jar;E:\program\hadoop-1.0.0\lib\commons-configuration-1.6.jar;E:\program\hadoop-1.0.0\lib\commons-collections-3.2.1.jar;E:\program\hadoop-1.0.0\lib\commons-codec-1.4.jar;E:\program\hadoop-1.0.0\lib\commons-cli-1.2.jar;E:\program\hadoop-1.0.0\lib\commons-beanutils-core-1.8.0.jar;E:\program\hadoop-1.0.0\lib\commons-beanutils-1.7.0.jar;E:\program\hadoop-1.0.0\lib\aspectjtools-1.6.5.jar;E:\program\hadoop-1.0.0\lib\aspectjrt-1.6.5.jar;E:\program\hadoop-1.0.0\lib\asm-3.2.jar;E:\program\hadoop-1.0.0\hadoop-tools-1.0.0.jar;E:\program\hadoop-1.0.0\hadoop-core-1.0.0.jar;E:\program\hadoop-1.0.0\hadoop-ant-1.0.0.jar;C:\Users\max\workspace\WordCountProject\conf
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.library.path=C:\Program Files\Java\jdk1.7.0_21\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Common Files\Thunder Network\KanKan\Codecs;C:\Program Files\Java\jdk1.7.0_21\bin;E:\program\apache-maven-3.0.5\\bin;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;.
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\max\AppData\Local\Temp\
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 8
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:os.version=6.2
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\max
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Client environment:user.dir=C:\Users\max\workspace\WordCountProject
13/06/17 16:52:33 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.1.72:2181,192.168.1.71:2181,192.168.1.73:2181 sessionTimeout=180000 watcher=hconnection
13/06/17 16:52:33 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 8104@max-pc
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Opening socket connection to server 192.168.1.73/192.168.1.73:2181. Will not attempt to authenticate using SASL (unknown error)
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Socket connection established to 192.168.1.73/192.168.1.73:2181, initiating session
13/06/17 16:52:36 WARN zookeeper.ClientCnxnSocket: Connected to an old server; r-o mode will be unavailable
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Session establishment complete on server 192.168.1.73/192.168.1.73:2181, sessionid = 0x43f512f7ede0002, negotiated timeout = 40000
13/06/17 16:52:36 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.1.72:2181,192.168.1.71:2181,192.168.1.73:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@34e0907c
13/06/17 16:52:36 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 8104@max-pc
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Opening socket connection to server 192.168.1.73/192.168.1.73:2181. Will not attempt to authenticate using SASL (unknown error)
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Socket connection established to 192.168.1.73/192.168.1.73:2181, initiating session
13/06/17 16:52:36 WARN zookeeper.ClientCnxnSocket: Connected to an old server; r-o mode will be unavailable
13/06/17 16:52:36 INFO zookeeper.ClientCnxn: Session establishment complete on server 192.168.1.73/192.168.1.73:2181, sessionid = 0x43f512f7ede0003, negotiated timeout = 40000


附录:一种Hbase操作的代码

/**
 * @author MAX.HAN
 * @version 1.0
 * HbaseHelper.java Create on 2013-6-17 下午02:07:13 
 * Copyright (c) 2013 Company,Inc. All Rights Reserved.
 */
package org.apache.hadoop.examples;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;

/**
 * Description:
 *
 * @author MAX.HAN
 * @version 1.0
 * Create on 2013-6-17 下午02:07:13
 */
public class HbaseHelper {
	private static Configuration conf = null;
	// 初始化配置
	static {
		Configuration HBASE_CONFIG = new Configuration();  
        //与hbase/conf/hbase-site.xml中hbase.zookeeper.quorum配置的值相同   
        HBASE_CONFIG.set("hbase.master", "192.168.1.70:60000");
        HBASE_CONFIG.set("hbase.zookeeper.quorum",  "192.168.1.71, 192.168.1.72,192.168.1.73");  
        //与hbase/conf/hbase-site.xml中hbase.zookeeper.property.clientPort配置的值相同  
        HBASE_CONFIG.set("hbase.zookeeper.property.clientPort", "2181");  
        conf = HBaseConfiguration.create(HBASE_CONFIG);  
	}

	// 1、建表
	public static void createTable(String tablename, String[] cfs)
			throws IOException {
		HBaseAdmin admin = new HBaseAdmin(conf);
		if (admin.tableExists(tablename)) {
			System.out.println("表已经存在!");
		} else {
			HTableDescriptor tableDesc = new HTableDescriptor(tablename);
			for (int i = 0; i < cfs.length; i++) {
				// 表建好后,列簇不能动态增加,而列是可以动态增加的,这是hbase伸缩性的一个体现。
				tableDesc.addFamily(new HColumnDescriptor(cfs[i]));
			}
			admin.createTable(tableDesc);
			System.out.println("表创建成功!");
		}
	}

	// 2、插入数据
	public static void writeRow(String tablename, String[] cfs) {
		try {
			HTable table = new HTable(conf, tablename);
			Put put = new Put(Bytes.toBytes("rows1"));
			for (int j = 0; j < cfs.length; j++) {
				// 指定列簇// 指定列名// 指定列值
				put.add(Bytes.toBytes(cfs[j]), Bytes.toBytes(String.valueOf("列1")),	Bytes.toBytes("value_13"));
				put.add(Bytes.toBytes(cfs[j]), Bytes.toBytes(String.valueOf("lie2")),Bytes.toBytes("value_24"));
				table.put(put);
				System.out.println("插入数据成功");
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	// 3、删除一行数据
	public static void deleteRow(String tablename, String rowkey)
			throws IOException {
		HTable table = new HTable(conf, tablename);
		List list = new ArrayList();
		Delete d1 = new Delete(rowkey.getBytes());
		list.add(d1);
		table.delete(list);
		System.out.println("删除行成功!");
	}

	// 4、查找一行数据
	public static void selectRow(String tablename, String rowKey)
			throws IOException {
		HTable table = new HTable(conf, tablename);
		Get g = new Get(rowKey.getBytes());
		Result rs = table.get(g);
		for (KeyValue kv : rs.raw()) {
			System.out.print(new String(kv.getRow()) + "  ");// 行号
			System.out.print(new String(kv.getFamily()) + ":");// 列簇名
			System.out.print(new String(kv.getQualifier()) + "  ");// 列名
			System.out.print(kv.getTimestamp() + "  ");// 时间戳
			System.out.println(new String(kv.getValue()));// 单元格的值
		}
	}

	// 5、查找全部数据
	public static void scanerTable(String tablename) {
		try {
			HTable table = new HTable(conf, tablename);
			Scan s = new Scan();
			ResultScanner rs = table.getScanner(s);
			for (Result r : rs) {
				KeyValue[] kv = r.raw();
				for (int i = 0; i < kv.length; i++) {
					System.out.print(new String(kv[i].getRow()) + "  ");
					System.out.print(new String(kv[i].getFamily()) + ":");
					System.out.print(new String(kv[i].getQualifier()) + "  ");
					System.out.print(kv[i].getTimestamp() + "  ");
					System.out.println(new String(kv[i].getValue()));
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	// 6、删除表
	public static void deleteTable(String tablename) throws IOException {
		try {
			HBaseAdmin admin = new HBaseAdmin(conf);
			admin.disableTable(tablename);
			admin.deleteTable(tablename);
			System.out.println("表删除成功!");
		} catch (MasterNotRunningException e) {
			e.printStackTrace();
		} catch (ZooKeeperConnectionException e) {
			e.printStackTrace();
		}
	}

	public static void main(String[] args) throws IOException {

		String[] cfs = "a,b,c".split(",");
		createTable("test01", cfs);
		//writeRow("test01", cfs);
		// deleteRow("test01", "rows1");
		// selectRow("test01", "rows2");
		//scanerTable("test01");
		// deleteTable("test01");
	}
}

参考链接:

1.【Hbase】Java对hbase的操作   http://smallwildpig.iteye.com/blog/1695203

2. 用 Java 对 hbase 进行CRUD增删改查操作 http://my.oschina.net/leejun2005/blog/92775


 

© 著作权归作者所有

华兹格

华兹格

粉丝 163
博文 220
码字总数 82578
作品 0
海淀
高级程序员
私信 提问
加载中

评论(2)

华兹格
华兹格 博主

引用来自“小小晋”的评论

您好,想查看这篇文章,是否可以开权限了?
好的,很多年的了,现在也不再搞这个了
小小晋
小小晋
您好,想查看这篇文章,是否可以开权限了?
HBase 源码-下载, 编译

有些事情, 看起来很简单, 但是自己从头摸索时却是各种迷茫, 不知从何下手, 比如 HBase 源码的下载, 编译... 1. 软件安装 (1). 安装 git sudo apt-get install git (2). 安装 jdk7 sudo add-...

sailtseng
2013/12/04
3.8K
0
关于学习Hadoop中未总结的资料

1)Cygwin相关资料   (1)Cygwin上安装、启动ssh服务失败、ssh localhost失败的解决方案   地址:http://blog.163.com/pwcrab/blog/static/16990382220107267443810/   (2)windows...

Carl_
2015/06/25
49
0
大数据(HBase-分布式安装和基本命令使用)

分布式搭建 1:下载官网对应的hbase的版本 地址:http://archive.apache.org/dist/hbase/1.2.3/ 2:上传对应的版本到主节点并解压 3:配置hbase的环境变量 export HBASEHOME=/home/hadoop/bd...

这很耳东先生
2019/06/26
23
0
bulk-load装载hdfs数据到hbase小结

bulk-load的作用是用mapreduce的方式将hdfs上的文件装载到hbase中,对于海量数据装载入hbase非常有用,参考http://hbase.apache.org/docs/r0.89.20100621/bulk-loads.html: hbase提供了现成...

超人学院
2015/06/01
127
0
hadoop hive hbase 入门学习 (二)

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// hadoop 自学系列 hado......

curiousby
2015/03/08
0
0

没有更多内容

加载失败,请刷新页面

加载更多

应急广播户户通平台

一、平台概述 应急广播户户通平台为软硬一体化广播服务解决方案。实现了应急广播、视音频及图片文字信息、调频及数字广播FM、天气预报信息接收功能,以及视音频播放、智能机器人、电子日历等...

neocean
33分钟前
47
0
如何为Apache 2.2启用mod_rewrite

我已经在我的Vista机器上安装了新的Apache 2.2,一切正常,除了mod重写。 我没有注释 LoadModule rewrite_module modules/mod_rewrite.s 但是我的重写规则都没有,即使是简单的重写规则 Re...

javail
39分钟前
23
0
移除Python unicode字符串中的重音符号的最佳方法是什么?

我在Python中有一个Unicode字符串,我想删除所有的重音符号(变音符号)。 我在网上发现了一种用Java实现此目的的优雅方法: 将Unicode字符串转换为长规范化格式(带有单独的字母和变音符号)...

技术盛宴
55分钟前
48
0
ActiveMQ学习之SpringBoot整合ActiveMQ------>主题生产者和消费者

一、pom <!--聚合工程集成关系--> <!--统一整合第三方框架依赖信息--> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</a......

冥焱
今天
89
0
两周自制脚本语言-第11天 优化变量读写性能

第11天 优化变量读写性能 以变量值的读写为例,向读者介绍基于这种理念的语言处理器性能优化方式。 11.1 通过简单数组来实现环境 假如函数包含局部变量x与y,程序可以事先将x设为数组的第0个...

果汁分你一半
今天
58
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部