文档章节

Apache Slider编译安装

Yulong_
 Yulong_
发布于 2016/09/08 14:38
字数 1228
阅读 663
收藏 0

Apache Slider

简介

Apache Slider目前是Apache二级孵化项目,主要用途是在YARN上部署用户已有的服务或者组件,如:hbase,storm,kafka.

下载地址:Apache Slider Download,目前最新0.91.0版本

系统要求

需要安装软件

  • Apache Hadoop Hadoop 2.6+
  • Apache Zookeeper 3.4+
  • Oracle JDK 1.7 (64-bit)
  • Python 2.6+
  • openssl

需要启动服务

Zookeeper

提供slider应用所需的zookeeper数据存放目录,如hbase的zookeeper.znode.parent

/services/slider/users/{username}/{application_name}
#for example:
/services/slider/users/hadoop/cl1

将在slider中启动的应用所用container信息注册在此位置

/registry/users/{username}/services/org-apache-slider/{application_name}
#for example:
/registry/users/hadoop/services/org-apache-slider/cl1/components/container-1473130308719-0016-01-000002

HDFS

提供slider应用包所需的hdfs数据存放目录

 /user/{username}/.slider/package/{package_name}/slider-{package_name}-app-package-{version}.zip
 #for example:
 /user/hadoop/.slider/package/HBASE/slider-hbase-app-package-0.98.21-hadoop2.zip

提供slider应用所需的hdfs数据存放目录,包括一些配置以及hdfs数据

 /user/{username}/.slider/cluster/{application_name}/*
 #for example:
 /user/hadoop/.slider/cluster/cl1/app_config.json
 /user/hadoop/.slider/cluster/cl1/confdir
 /user/hadoop/.slider/cluster/cl1/database
 /user/hadoop/.slider/cluster/cl1/generated
 /user/hadoop/.slider/cluster/cl1/history
 /user/hadoop/.slider/cluster/cl1/internal.json
 /user/hadoop/.slider/cluster/cl1/resources.json
 /user/hadoop/.slider/cluster/cl1/snapshot
 /user/hadoop/.slider/cluster/cl1/tmp

slider应用的全局配置文件app_config.json,可以通过修改进行相关服务配置的修改

 /user/{username}/.slider/cluster/{application_name}/app_config.json
 #for example:
 /user/hadoop/.slider/cluster/cl1/app_config.json

Yarn

提供Slider启动时所需要的container.yarn-site.xml需要修改以下配置

<property>
  <name>yarn.scheduler.minimum-allocation-mb</name>
  <value>256</value>
  <description>more than 256 (ensure that YARN can allocate sufficient number of containers),general set to more than 1000 for production environment</description>
</property>
<property>
  <name>yarn.nodemanager.delete.debug-delay-sec</name>
  <value>3600</value>
  <description>Number of seconds after an application finishes before the nodemanager's DeletionService will delete the application's localized file directory and log directory. To diagnose Yarn application problems, set this property's value large enough (for example, to 600 = 10 minutes) to permit examination of these directories. After changing the property's value, you must restart the nodemanager in order for it to have an effect. The roots of Yarn applications' work directories is configurable with the yarn.nodemanager.local-dirs property (see below), and the roots of the Yarn applications' log directories is configurable with the yarn.nodemanager.log-dirs property (see also below).</description>
</property>

##编译

下载源码包解压至某目录${BUILD_DIR},maven项目,编译命令

mvn clean site:site site:stage package -DskipTests

执行包生成在

${BUILD_DIR}/apache-slider-0.91.0-incubating/slider-assembly/target/

压缩包为tar.gz和zip格式,任选其一

slider-0.91.0-incubating-all.tar.gz
slider-0.91.0-incubating-all.zip

##安装

选择安装目录${SLIDER_INSTALL_DIR}

cp slider-0.91.0-incubating-all.zip ${SLIDER_INSTALL_DIR}
cd ${SLIDER_INSTALL_DIR}
unzip slider-0.91.0-incubating-all.zip
ln -s slider-0.91.0-incubating slider

设置环境变量,根据你自己的需要修改/etc/profile或者~/.bash_profile

export SLIDER_HOME=${SLIDER_INSTALL_DIR}/slider
export PATH=$SLIDER_HOME/bin:$PATH

修改配置文件$SLIDER_HOME/conf/slider-env.sh: export JAVA_HOME=${JAVA_HOME} export HADOOP_HOME=${HADOOP_HOME} export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop

##测试 通过slider命令验证一些基本功能。 ###slider help 测试slider help命令查看是否正常,正常说明PATH等环境变量是没有问题的。

[hadoop[@breath](http://my.oschina.net/liu1994) ~]$ slider help
2016-09-08 14:29:58,200 [main] INFO  client.SliderClient - 
Usage: slider COMMAND [options]
where COMMAND is one of
	am-suicideTell the Slider Application Master to simulate a process failure by terminating itself
	build Build a Slider cluster specification, but do not start it
	clientInstall the application client in the specified directory or obtain a client keystore or truststore
	createCreate a live Slider application
	dependencySlider AM and agent dependency (libraries) management
	destroy   Destroy a stopped Slider application
	diagnostics   Diagnose the configuration of the running slider application and slider client
	existsProbe for an application running
	flex  Flex a Slider application
	stop  Stop a running application
	help  Print help information
	install-keytabInstall the Kerberos keytab. Deprecated, use 'keytab --install'.
	install-package   Install application package. Deprecated, use 'package --install'.
	kdiag Diagnose Kerberos problems
	keytabManage a Kerberos keytab file (install, delete, list) in the sub-folder 'keytabs' of the user's Slider base directory
	kill-containerKill a container in the application
	list  List running Slider applications
	lookuplook up a YARN application
	nodes List the node information for the YARN cluster or a running application
	package   Install/list/delete application packages and list app instances that use the packages
	registry  Query the registry of a YARN application
	resolve   Query the registry of a YARN application
	statusGet the status of an application
	start Start a stopped application
	tokenssave tokens to a file or list tokens in a file
	updateUpdate template for a Slider application
	upgrade   Rolling upgrade/downgrade the application to a newer/previous version
	version   Print the Slider version information
Most commands print help when invoked without parameters or with --help
2016-09-08 14:29:58,244 [main] INFO  util.ExitUtil - Exiting with status 0

###slider package --list 测试slider package --list命令查看是否正常,正常说明HDFS、YARN等服务配置是正常的。我这里上传了一个HBASE的app包,所以显示HBASE。

[hadoop[@breath](http://my.oschina.net/liu1994) conf]$ slider package --list
2016-09-08 14:28:49,187 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/192.168.246.136:23140
2016-09-08 14:28:50,102 [main] INFO  client.SliderClient - Package install path : hdfs://breath:9000/user/hadoop/.slider/package
List of installed packages:
	HBASE

2016-09-08 14:28:51,068 [main] INFO  util.ExitUtil - Exiting with status 0

###slider list 测试slider package --list命令查看是否正常,正常说明HDFS、YARN等服务配置是正常的。我这里已经通过HBASE已经启动了一个cluster名为cl1,所以显示cl1状态为RUNNING.

[hadoop[@breath](http://my.oschina.net/liu1994) conf]$ slider list
2016-09-08 14:28:24,088 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/192.168.246.136:23140
cl1RUNNING  application_1473130308719_0016  http://breath:23188/proxy/application_1473130308719_0016/
2016-09-08 14:28:26,187 [main] INFO  util.ExitUtil - Exiting with status 0

© 著作权归作者所有

Yulong_
粉丝 10
博文 145
码字总数 253510
作品 0
朝阳
部门经理
私信 提问
Apache Slider + Hbase

Apache Slider + Hbase 系统环境 安装如下组件,部署可用环境 JDK 1.7.0_79 Apache Zookeeper 3.4.* Apache Zookeeper Apache Hadoop 2.6.* Apache Hadoop Apache Hbase 0.98.21-hadoop Apa......

Yulong_
2016/09/08
184
0
Apache Slider + Storm

Apache Slider + Storm 系统环境 安装如下组件,部署可用环境 JDK 1.7.0_79 Apache Zookeeper 3.4.* Apache Zookeeper Apache Hadoop 2.6.* Apache Hadoop Apache Storm 0.9.4 Apache Storm......

Yulong_
2016/09/21
457
0
基于QtAV的简易播放器(开源)

这个开源代码,是我利用QtAV源码,提取其中一部分代码,进行整合到我自己项目中,做的一个小型播放器测试,至于怎么安装一些环境以及QtAV源码编译在我以前写的一篇博客中可以看到(Qt第三方库...

longbei9029
2017/02/26
0
0
Apache Slider 0.60.0-incubating 发布

Apache Slider 0.60.0-incubating 发布,下载地址: apache-slider-0.60.0-incubating-source-release.tar.gz apache-slider-0.60.0-incubating-source-release.zip Apache Slider 是一个 YA......

oschina
2014/11/25
1K
0
Apache Slider 0.91.0-incubating 发布

Apache Slider 0.91.0-incubating 发布了。 Apache Slider 是一个 Yarn 应用,它可以用来在 Yarn 上部署并监控分布式应用。Slider 可以在应用运行期随意扩展或者收缩应用。 新特性 & 提升: ...

oschina
2016/06/30
844
0

没有更多内容

加载失败,请刷新页面

加载更多

Kylin构建Cube过程详解

1 前言 在使用Kylin的时候,最重要的一步就是创建cube的模型定义,即指定度量和维度以及一些附加信息,然后对cube进行build,当然我们也可以根据原始表中的某一个string字段(这个字段的格式...

大数据技术进阶
15分钟前
4
0
Git保存密码

保存密码 $ git config --global credential.helper store 参数 --global 设置全局,如果用 --local 则只设置当前库 要注意保存时是用明文保存的,所以不要在公用电脑使用...

编程老陆
17分钟前
4
0
ofcms 说明文档

一、模板说明 项目概述 java 版CMS系统、基于java技术研发的内容管理系统、功能:栏目模板自定义、内容模型自定义、多个站点管理、在线模板页面编辑等功能、代码完全开源、MIT授权协议。 技术...

kuchawyz
23分钟前
4
0
理解CSS相对定位和固定定位

本文转载于:专业的前端网站➦理解CSS相对定位和固定定位 前面的话   一般地,说起定位元素是指position不为static的元素,包括relative、absolute和fixed。前面已经详细介绍过absolute绝对...

前端老手
34分钟前
4
0
iOS Xcode升级包地址(感谢大神)

下载地址:DeviceSupport

_____1____
48分钟前
6
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部