azkaban3.x exec-server之jobtype
azkaban3.x exec-server之jobtype
你我他有个梦 发表于4个月前
azkaban3.x exec-server之jobtype
  • 发表于 4个月前
  • 阅读 12
  • 收藏 0
  • 点赞 0
  • 评论 0

腾讯云 新注册用户 域名抢购1元起>>>   

1.复制编译后的jobtype到exec服务的插件下,

cp dist/jobtype/packages/azkaban-jobtype-${git.tag}.tar.gz
/home/app/azkaban/azkaban-exec-server/plugins/jobtypes
cd /home/app/azkaban/azkaban-exec-server/plugins/jobtypes
tar -zxvf azkaban-jobtype-${git.tag}.tar.gz
解压之后将azkaban-jobtype-${git.tag}目录下的内容移动到/home/app/azkaban/azkaban-exec-server/plugins/jobtypes目录,然后rm -rf azkaban-jobtype-${git.tag}

2.在/home/app/azkaban/azkaban-exec-server/conf/azkaban.properties配置文件里增加一行语句:

azkaban.jobtype.plugin.dir=plugins/jobtypes

3.vim plugins/jobtypes/common.properties

## everything that the user job can know
##配置hadoop、hive、spark home
hadoop.home=/home/app/hadoop-2.7.2
hive.home=/home/app/hive-2.1.1
#pig.home=
spark.home=/home/app/spark2.1

azkaban.should.proxy=false

4.vim plugins/jobtypes/commonprivate.properties

## hadoop security manager setting common to all hadoop jobs
hadoop.security.manager.class=azkaban.security.HadoopSecurityManager_H_2_0

## hadoop security related settings

# proxy.keytab.location=
# proxy.user=

# azkaban.should.proxy=true
 azkaban.should.proxy=false
# obtain.binary.token=true
 obtain.binary.token=false
# obtain.namenode.token=true
# obtain.jobtracker.token=true

# global classpath items for all jobs. e.g. hadoop-core jar, hadoop conf
#jobtype.global.classpath=${hadoop.home}/*,${hadoop.home}/conf
# global jvm args for all jobs. e.g. java.io.temp.dir, java.library.path
#jobtype.global.jvm.args=

# hadoop
#hadoop.home=/home/app/hadoop-2.7.2
#pig.home=
hadoop.home=/home/app/hadoop-2.7.2
hive.home=/home/app/hive-2.1.1
#hive.home=
spark.home=/home/app/spark2.1
hadoop.classpath=${hadoop.home}/etc/hadoop,${hadoop.home}/share/hadoop/common/*,${hadoop.home}/share/hadoop/common/lib/*,${hadoop.home}/share/hadoop/hdfs/*,${hadoop.home}/share/hadoop/hdfs/lib/*,${hadoop.home}/share/hadoop/yarn/*,${hadoop.home}/share/hadoop/yarn/lib/*,${hadoop.home}/share/hadoop/mapreduce/*,${hadoop.home}/share/hadoop/mapreduce/lib/*
jobtype.global.classpath=${hadoop.home}/etc/hadoop,${hadoop.home}/share/hadoop/common/*,${hadoop.home}/share/hadoop/common/lib/*,${hadoop.home}/share/hadoop/hdfs/*,${hadoop.home}/share/hadoop/hdfs/lib/*,${hadoop.home}/share/hadoop/yarn/*,${hadoop.home}/share/hadoop/yarn/lib/*,${hadoop.home}/share/hadoop/mapreduce/*,${hadoop.home}/share/hadoop/mapreduce/lib/*
# configs for jobtype security settings
execute.as.user=tru

5.复制一份azkaban-jobtype-${git.tag}.jar到azkaban-exec-server/lib目录下

cp azkaban-exec-server/plugins/jobtypes/hdfsToTeradata/azkaban-exec-server/lib azkaban-exec-server/lib

 

共有 人打赏支持
你我他有个梦
粉丝 91
博文 109
码字总数 98629
×
你我他有个梦
如果觉得我的文章对您有用,请随意打赏。您的支持将鼓励我继续创作!
* 金额(元)
¥1 ¥5 ¥10 ¥20 其他金额
打赏人
留言
* 支付类型
微信扫码支付
打赏金额:
已支付成功
打赏金额: