文档章节

Apache Slider + Storm

Yulong_
 Yulong_
发布于 2016/09/21 16:54
字数 3171
阅读 458
收藏 0

#Apache Slider + Storm

##系统环境

安装如下组件,部署可用环境

Apache storm配置相关目录及环境变量,不启动服务,主要作用为client连接slider storm需要使用

Apache Slider部署参照Apache Slider编译安装

##Storm App Package编译

###下载Strom可执行包 下载软件包apache-storm-0.9.4.tar.gz至/home/hadoop/Downloads目录下。

cd /home/hadoop/Downloads
ls apache-storm-0.9.4.tar.gz

###编译环境

Slider编译环境参照Apache Slider编译安装

Storm Slider编译命令

mvn clean package -DskipTests -Pstorm-app-package -Dpkg.version=0.9.4 -Dpkg.name=apache-storm-0.9.4.tar.gz -Dpkg.src=/home/hadoop/Downloads

生成Slider-storm-app-package-${version}.zip。

${BUILD_DIR}/apache-slider-0.91.0-incubating/app-packages/storm/target/slider-storm-app-package-0.9.4.zip

##创建package 创建package

slider package --install --name STORM --package slider-storm-app-package-0.9.4.zip

查看package是否已经上传

hadoop fs -ls /user/hadoop/.slider/package/STORM/slider-storm-app-package-0.9.4.zip

替换新的package

slider package --install  --name STORM --package slider-storm-app-package-0.9.4.zip --relpacepkg

删除package

slider package --delete --name STORM 

##创建slider app ###slider app配置文件 需要配置如下两个配置文件

${BUILD_DIR}/apache-slider-0.91.0-incubating1/app-packages/storm/appConfig-default.json
${BUILD_DIR}/apache-slider-0.91.0-incubating1/app-packages/storm/resources-default.json

拷贝以供修改使用

cd ${BUILD_DIR}/apache-slider-0.91.0-incubating1/app-packages/storm
cp appConfig-default.json appConfig.json
cp resources-default.json resources.json

####appConfig.json appConfig.json配置为Hbase相关服务的参数配置,可以理解为通过此文件生成服务启动时的hbase-site.xml和hbase-env.sh

#####默认配置如下

{
  "schema": "http://example.org/specification/v2.0.0",
  "metadata": {
  },
  "global": {
    "application.def": ".slider/package/STORM/slider-storm-app-package-${pkg.version}.zip",
    "java_home": "/usr/jdk64/jdk1.7.0_67",
    "create.default.zookeeper.node": "true",

    "site.global.app_user": "${USER_NAME}",
    "site.global.app_version": "${storm.version}",
    "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.version}",
    "site.global.user_group": "hadoop",

    "site.global.metric_collector_host": "${NN_HOST}",
    "site.global.metric_collector_port": "6188",
    "site.global.metric_collector_lib": "",

    "site.storm-site.metrics.reporter.register" : "org.apache.hadoop.metrics2.sink.storm.StormTimelineMetricsReporter",
    "site.storm-site.storm.log.dir" : "${AGENT_LOG_ROOT}",
    "site.storm-site.storm.zookeeper.servers": "['${ZK_HOST}']",
    "site.storm-site.nimbus.thrift.port": "${NIMBUS.ALLOCATED_PORT}",
    "site.storm-site.storm.local.dir": "${AGENT_WORK_ROOT}/app/tmp/storm",
    "site.storm-site.transactional.zookeeper.root": "/transactional",
    "site.storm-site.storm.zookeeper.port": "2181",
    "site.storm-site.nimbus.childopts": "-Xmx1024m -javaagent:${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=${@//site/global/metric_collector_host},port=${@//site/global/metric    _collector_port},wireformat31x=true,mode=multicast,config=${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/conf/jmxetric-conf.xml,process=Nimbus_JVM",
    "site.storm-site.supervisor.childopts": "-Xmx256m -javaagent:${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=${@//site/global/metric_collector_host},port=${@//site/global/metric    _collector_port},wireformat31x=true,mode=multicast,config=${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/conf/jmxetric-conf.xml,process=Supervisor_JVM",
    "site.storm-site.ui.childopts": "-Xmx768m",
    "site.storm-site.worker.childopts": "-Xmx768m -javaagent:${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/lib/jmxetric-1.0.4.jar=host=${@//site/global/metric_collector_host},port=${@//site/global/metric    _collector_port},wireformat31x=true,mode=multicast,config=${AGENT_WORK_ROOT}/app/install/apache-storm-${storm.    version}/external/storm-jmxetric/conf/jmxetric-conf.xml,process=Worker_%ID%_JVM",
    "site.storm-site.dev.zookeeper.path": "${AGENT_WORK_ROOT}/app/tmp/dev-storm-zookeeper",
    "site.storm-site.drpc.invocations.port": "0",
    "site.storm-site.storm.zookeeper.root": "${DEFAULT_ZK_PATH}",
    "site.storm-site.transactional.zookeeper.port": "null",
    "site.storm-site.nimbus.host": "${NIMBUS_HOST}",
    "site.storm-site.ui.port": "${STORM_UI_SERVER.ALLOCATED_PORT}",
    "site.storm-site.supervisor.slots.ports": "[${SUPERVISOR.ALLOCATED_PORT}{PER_CONTAINER},${SUPERVISOR.    ALLOCATED_PORT}{PER_CONTAINER}]",
    "site.storm-site.drpc.port": "0",
    "site.storm-site.logviewer.port": "${SUPERVISOR.ALLOCATED_PORT}{PER_CONTAINER}"
  },
  "components": {
    "slider-appmaster": {
      "jvm.heapsize": "256M"
    }
  }
}

#####根据测试集群调整配置如下

{
  "schema": "http://example.org/specification/v2.0.0",
  "metadata": {
  },
  "global": {
    "application.def": ".slider/package/STORM/slider-storm-app-package-0.9.4.zip",
    "java_home": "/opt/beh/core/jdk",
    "create.default.zookeeper.node": "true",

    "site.global.app_user": "${USER_NAME}",
    "site.global.app_version": "0.9.4",
    "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/apache-storm-0.9.4",
    "site.global.user_group": "hadoop",

    "site.global.metric_collector_host": "${NN_HOST}",
    "site.global.metric_collector_port": "6188",
    "site.global.metric_collector_lib": "",

    "site.storm-site.metrics.reporter.register" : "org.apache.hadoop.metrics2.sink.storm.StormTimelineMetricsReporter",
    "site.storm-site.storm.log.dir" : "${AGENT_LOG_ROOT}",
    "site.storm-site.storm.zookeeper.servers": "[breath]",
    "site.storm-site.nimbus.thrift.port": "${NIMBUS.ALLOCATED_PORT}",
    "site.storm-site.storm.local.dir": "${AGENT_WORK_ROOT}/app/tmp/storm",
    "site.storm-site.transactional.zookeeper.root": "/transactional",
    "site.storm-site.storm.zookeeper.port": "2181",
    "site.storm-site.nimbus.childopts": "-Xmx1024m ",
    "site.storm-site.supervisor.childopts": "-Xmx256m",
    "site.storm-site.ui.childopts": "-Xmx768m",
    "site.storm-site.worker.childopts": "-Xmx768m",
    "site.storm-site.dev.zookeeper.path": "${AGENT_WORK_ROOT}/app/tmp/dev-storm-zookeeper",
    "site.storm-site.drpc.invocations.port": "0",
    "site.storm-site.storm.zookeeper.root": "${DEFAULT_ZK_PATH}",
    "site.storm-site.transactional.zookeeper.port": "null",
    "site.storm-site.nimbus.host": "${NIMBUS_HOST}",
    "site.storm-site.ui.port": "${STORM_UI_SERVER.ALLOCATED_PORT}",
    "site.storm-site.supervisor.slots.ports": "[${SUPERVISOR.ALLOCATED_PORT}{PER_CONTAINER},${SUPERVISOR.    ALLOCATED_PORT}{PER_CONTAINER}]",
    "site.storm-site.drpc.port": "0",
    "site.storm-site.logviewer.port": "${SUPERVISOR.ALLOCATED_PORT}{PER_CONTAINER}"
  },
  "components": {
    "slider-appmaster": {
      "jvm.heapsize": "256M"
    }
  }
}

其中测试时如下几项修改比较重要:

  • JAVA_HOME
    "java_home": "/opt/beh/core/jdk",

  • Zookeeper集群主机列表,主机名逗号隔开
    "site.storm-site.storm.zookeeper.servers": "[breath]",

  • 在JVM参数中移除了jmxetric的相关参数,只为测试使用 "site.storm-site.nimbus.childopts" "site.storm-site.supervisor.childopts" "site.storm-site.worker.childopts"

####resource.json resource.json的配置主要是设置Storm中不同服务启动时的container相关参数

#####默认配置如下

  {
  "schema" : "http://example.org/specification/v2.0.0",
  "metadata" : {
  },
  "global" : {
    "yarn.log.include.patterns": "",
    "yarn.log.exclude.patterns": ""
  },
  "components": {
    "slider-appmaster": {
      "yarn.memory": "512"
    },
    "NIMBUS": {
      "yarn.role.priority": "1",
      "yarn.component.instances": "1",
      "yarn.memory": "2048"
    },
    "STORM_UI_SERVER": {
      "yarn.role.priority": "2",
      "yarn.component.instances": "1",
      "yarn.memory": "1278"
    },
    "DRPC_SERVER": {
      "yarn.role.priority": "3",
      "yarn.component.instances": "1",
      "yarn.memory": "1278"
    },
    "SUPERVISOR": {
      "yarn.role.priority": "4",
      "yarn.component.instances": "1",
      "yarn.memory": "3072"
    }
  }
}  

###通过slider启动storm 此处cl1为为启动的slider-storm的一个实例名称。后文用${slider_application_name}表示

  • 创建

slider create cl1 --template appConfig.json --resources resources.json

2016-10-21 16:56:05,292 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/172.16.13.69:23140
2016-10-21 16:56:06,516 [main] INFO  zk.BlockingZKWatcher - waiting for ZK event
2016-10-21 16:56:06,614 [main-EventThread] INFO  zk.BlockingZKWatcher - ZK binding callback received
2016-10-21 16:56:07,098 [main] INFO  agent.AgentClientProvider - Validating app definition .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:56:07,100 [main] INFO  agent.AgentUtils - Reading metainfo at .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:56:07,409 [main] WARN  hdfs.DFSClient - DFSInputStream has been closed already
2016-10-21 16:56:07,409 [main] INFO  agent.AgentUtils - Got metainfo from summary file
2016-10-21 16:56:07,903 [main] INFO  client.SliderClient - No credentials requested
2016-10-21 16:56:08,094 [main] INFO  agent.AgentUtils - Reading metainfo at .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:56:08,113 [main] WARN  hdfs.DFSClient - DFSInputStream has been closed already
2016-10-21 16:56:08,114 [main] INFO  agent.AgentUtils - Got metainfo from summary file
2016-10-21 16:56:08,155 [main] INFO  launch.AbstractLauncher - Setting yarn.resourcemanager.am.retry-count-window-ms to 300000
2016-10-21 16:56:08,155 [main] INFO  launch.AbstractLauncher - Log include patterns: 
2016-10-21 16:56:08,155 [main] INFO  launch.AbstractLauncher - Log exclude patterns: 
2016-10-21 16:56:08,155 [main] INFO  launch.AbstractLauncher - Modified log include patterns: .*
2016-10-21 16:56:08,155 [main] INFO  launch.AbstractLauncher - Modified log exclude patterns: 
2016-10-21 16:56:08,892 [main] INFO  slideram.SliderAMClientProvider - Loading all dependencies for AM.
2016-10-21 16:56:08,905 [main] INFO  tools.SliderUtils - Loading all dependencies from /opt/beh/core/slider-0.91.0-incubating/lib
2016-10-21 16:56:12,171 [main] INFO  agent.AgentClientProvider - Automatically uploading the agent tarball at hdfs://breath:9000/user/hadoop/.slider/cluster/storm2/tmp/application_1476173557981_0003/agent
2016-10-21 16:56:12,216 [main] INFO  agent.AgentClientProvider - Validating app definition .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:56:12,224 [main] INFO  tools.SliderUtils - For faster submission of apps, upload dependencies using cmd dependency --upload
2016-10-21 16:56:12,231 [main] INFO  client.SliderClient - Submitting application application_1476173557981_0003
2016-10-21 16:56:12,234 [main] INFO  launch.AppMasterLauncher - Submitting application to Resource Manager
2016-10-21 16:56:12,265 [main] INFO  impl.YarnClientImpl - Submitted application application_1476173557981_0003
2016-10-21 16:56:12,268 [main] INFO  util.ExitUtil - Exiting with status 0
  • 启动

slider start cl1

2016-10-21 16:57:54,576 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/172.16.13.69:23140
2016-10-21 16:57:56,374 [main] INFO  agent.AgentUtils - Reading metainfo at .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:57:56,642 [main] WARN  hdfs.DFSClient - DFSInputStream has been closed already
2016-10-21 16:57:56,642 [main] INFO  agent.AgentUtils - Got metainfo from summary file
2016-10-21 16:57:56,717 [main] INFO  launch.AbstractLauncher - Setting yarn.resourcemanager.am.retry-count-window-ms to 300000
2016-10-21 16:57:56,717 [main] INFO  launch.AbstractLauncher - Log include patterns: 
2016-10-21 16:57:56,717 [main] INFO  launch.AbstractLauncher - Log exclude patterns: 
2016-10-21 16:57:56,717 [main] INFO  launch.AbstractLauncher - Modified log include patterns: .*
2016-10-21 16:57:56,717 [main] INFO  launch.AbstractLauncher - Modified log exclude patterns: 
2016-10-21 16:57:57,517 [main] INFO  slideram.SliderAMClientProvider - Loading all dependencies for AM.
2016-10-21 16:57:57,540 [main] INFO  tools.SliderUtils - Loading all dependencies from /opt/beh/core/slider-0.91.0-incubating/lib
2016-10-21 16:58:00,721 [main] INFO  agent.AgentClientProvider - Automatically uploading the agent tarball at hdfs://breath:9000/user/hadoop/.slider/cluster/storm2/tmp/application_1476173557981_0004/agent
2016-10-21 16:58:00,765 [main] INFO  agent.AgentClientProvider - Validating app definition .slider/package/STORM/slider-storm-app-package-0.9.4.zip
2016-10-21 16:58:00,784 [main] INFO  tools.SliderUtils - For faster submission of apps, upload dependencies using cmd dependency --upload
2016-10-21 16:58:00,790 [main] INFO  client.SliderClient - Submitting application application_1476173557981_0004
2016-10-21 16:58:00,793 [main] INFO  launch.AppMasterLauncher - Submitting application to Resource Manager
2016-10-21 16:58:01,030 [main] INFO  impl.YarnClientImpl - Submitted application application_1476173557981_0004
2016-10-21 16:58:01,033 [main] INFO  util.ExitUtil - Exiting with status 0
  • 关闭

slider stop cl1

2016-10-21 16:57:38,433 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/172.16.13.69:23140
2016-10-21 16:57:40,596 [main] INFO  util.ExitUtil - Exiting with status 0
  • 摧毁

slider destroy --force cl1

###通过YARN WEB端页面查看应用是否正常启动

Yarn中的Application

Storm的服务分布

配置Url

Storm集群的web访问url

Storm的web访问页面

###登录Slider Storm ####客户端程序 解压slider-storm-app-package-0.9.4.zip包含两个python脚本,可以将这两个脚本放置在$STORM_HOME/bin或者$SLIDER_HOME/bin

storm-slider.py
storm-slider

#####storm-slider测试

编译Storm自带的测试样例:${STORM_HOME}/examples/storm-starter生成jar包storm-starter-0.9.6-jar-with-dependencies.jar,并进行测试。

[hadoop@breath Documents]storm-slider --app cl1 jar storm-starter-0.9.6-jar-with-dependencies.jar storm.starter.RollingTopWords     production-topology remote
2016-09-21 16:05:36,276 [main] INFO  impl.TimelineClientImpl - Timeline service address: http://breath:8188/ws/v1/timeline/
2016-09-21 16:05:36,788 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/192.168.134.171:23140
2016-09-21 16:05:38,182 [main] INFO  util.ExitUtil - Exiting with status 0
Running: java -client -Dstorm.options=nimbus.host=breath,nimbus.thrift.port=43669,storm.thrift.transport=backtype.storm.security.auth    .SimpleTransportPlugin -Dstorm.home=/opt/beh/core/storm -cp /opt/beh/core/storm/lib/logback-core-1.0.13.    jar:/opt/beh/core/storm/lib/slf4j-api-1.7.5.jar:/opt/beh/core/storm/lib/jgrapht-core-0.9.0.jar:/opt/beh/core/storm/lib/jline-2.11.    jar:/opt/beh/core/storm/lib/reflectasm-1.07-shaded.jar:/opt/beh/core/storm/lib/compojure-1.1.3.jar:/opt/beh/core/storm/lib/core.    incubator-0.1.0.jar:/opt/beh/core/storm/lib/carbonite-1.4.0.jar:/opt/beh/core/storm/lib/commons-fileupload-1.2.1.    jar:/opt/beh/core/storm/lib/minlog-1.2.jar:/opt/beh/core/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/beh/core/storm/lib/tools.    logging-0.2.3.jar:/opt/beh/core/storm/lib/snakeyaml-1.11.jar:/opt/beh/core/storm/lib/tools.cli-0.2.4.    jar:/opt/beh/core/storm/lib/clojure-1.5.1.jar:/opt/beh/core/storm/lib/clj-time-0.4.1.jar:/opt/beh/core/storm/lib/ring-core-1.1.5.    jar:/opt/beh/core/storm/lib/commons-lang-2.5.jar:/opt/beh/core/storm/lib/commons-exec-1.1.jar:/opt/beh/core/storm/lib/commons-logging    -1.1.3.jar:/opt/beh/core/storm/lib/joda-time-2.0.jar:/opt/beh/core/storm/lib/json-simple-1.1.jar:/opt/beh/core/storm/lib/log4j-over-    slf4j-1.6.6.jar:/opt/beh/core/storm/lib/ring-devel-0.3.11.jar:/opt/beh/core/storm/lib/servlet-api-2.5.    jar:/opt/beh/core/storm/lib/ring-servlet-0.3.11.jar:/opt/beh/core/storm/lib/objenesis-1.2.jar:/opt/beh/core/storm/lib/tools.macro-0.1    .0.jar:/opt/beh/core/storm/lib/logback-classic-1.0.13.jar:/opt/beh/core/storm/lib/disruptor-2.10.1.    jar:/opt/beh/core/storm/lib/commons-codec-1.6.jar:/opt/beh/core/storm/lib/chill-java-0.3.5.jar:/opt/beh/core/storm/lib/asm-4.0.    jar:/opt/beh/core/storm/lib/jetty-util-6.1.26.jar:/opt/beh/core/storm/lib/hiccup-0.3.6.jar:/opt/beh/core/storm/lib/clout-1.0.1.    jar:/opt/beh/core/storm/lib/kryo-2.21.jar:/opt/beh/core/storm/lib/math.numeric-tower-0.0.1.jar:/opt/beh/core/storm/lib/clj-stacktrace    -0.2.2.jar:/opt/beh/core/storm/lib/commons-io-2.4.jar:/opt/beh/core/storm/lib/storm-core-0.9.5.jar:/opt/beh/core/storm/lib/jetty-6.1.    26.jar:storm-starter-0.9.6-jar-with-dependencies.jar:/home/hadoop/.storm:/opt/beh/core/storm/bin -Dstorm.jar=storm-starter-0.9.6-jar-    with-dependencies.jar storm.starter.RollingTopWords production-topology remote
152  [main] INFO  storm.starter.RollingTopWords - Topology name: production-topology
195  [main] INFO  storm.starter.RollingTopWords - Running in remote (cluster) mode
360  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
367  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar storm-starter-0.9.6-jar-with-dependencies.jar to assigned     location: /opt/beh/data/tmp/yarn/local/usercache/hadoop/appcache/application_1474435760880_0009/container_1474435760880_0009_01_00000    2/app/tmp/storm/nimbus/inbox/stormjar-07dea68c-8ae8-4ca3-8cd2-6762b3b17aca.jar
391  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /opt/beh/data/tmp/yarn/loc    al/usercache/hadoop/appcache/application_1474435760880_0009/container_1474435760880_0009_01_000002/app/tmp/storm/nimbus/inbox/stormja    r-07dea68c-8ae8-4ca3-8cd2-6762b3b17aca.jar
391  [main] INFO  backtype.storm.StormSubmitter - Submitting topology production-topology in distributed mode with conf {"nimbus.    host":"breath","storm.thrift.transport":"backtype.storm.security.auth.SimpleTransportPlugin","topology.debug":true,"nimbus.thrift.    port":43669}
568  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: production-topolog。

查看storm的WEBUI

输入图片说明

输入图片说明

查看storm-slider集群cl1中运行的topology,可使用如下命令: storm-slider --app cl1 list

[hadoop@breath hadoop]$ storm-slider --app cl1 list
2016-09-21 18:02:57,665 [main] INFO  impl.TimelineClientImpl - Timeline service address: http://breath:8188/ws/v1/timeline/
2016-09-21 18:02:58,260 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/192.168.134.171:23140
2016-09-21 18:02:59,688 [main] INFO  util.ExitUtil - Exiting with status 0
Running: java -client -Dstorm.options=nimbus.host=breath,nimbus.thrift.port=43669,storm.thrift.transport=backtype.storm.security.auth    .SimpleTransportPlugin -Dstorm.home=/opt/beh/core/storm -cp /opt/beh/core/storm/lib/logback-core-1.0.13.    jar:/opt/beh/core/storm/lib/slf4j-api-1.7.5.jar:/opt/beh/core/storm/lib/jgrapht-core-0.9.0.jar:/opt/beh/core/storm/lib/jline-2.11.    jar:/opt/beh/core/storm/lib/reflectasm-1.07-shaded.jar:/opt/beh/core/storm/lib/compojure-1.1.3.jar:/opt/beh/core/storm/lib/core.    incubator-0.1.0.jar:/opt/beh/core/storm/lib/carbonite-1.4.0.jar:/opt/beh/core/storm/lib/commons-fileupload-1.2.1.    jar:/opt/beh/core/storm/lib/minlog-1.2.jar:/opt/beh/core/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/beh/core/storm/lib/tools.    logging-0.2.3.jar:/opt/beh/core/storm/lib/snakeyaml-1.11.jar:/opt/beh/core/storm/lib/tools.cli-0.2.4.    jar:/opt/beh/core/storm/lib/clojure-1.5.1.jar:/opt/beh/core/storm/lib/clj-time-0.4.1.jar:/opt/beh/core/storm/lib/ring-core-1.1.5.    jar:/opt/beh/core/storm/lib/commons-lang-2.5.jar:/opt/beh/core/storm/lib/commons-exec-1.1.jar:/opt/beh/core/storm/lib/commons-logging    -1.1.3.jar:/opt/beh/core/storm/lib/joda-time-2.0.jar:/opt/beh/core/storm/lib/json-simple-1.1.jar:/opt/beh/core/storm/lib/log4j-over-    slf4j-1.6.6.jar:/opt/beh/core/storm/lib/ring-devel-0.3.11.jar:/opt/beh/core/storm/lib/servlet-api-2.5.    jar:/opt/beh/core/storm/lib/ring-servlet-0.3.11.jar:/opt/beh/core/storm/lib/objenesis-1.2.jar:/opt/beh/core/storm/lib/tools.macro-0.1    .0.jar:/opt/beh/core/storm/lib/logback-classic-1.0.13.jar:/opt/beh/core/storm/lib/disruptor-2.10.1.    jar:/opt/beh/core/storm/lib/commons-codec-1.6.jar:/opt/beh/core/storm/lib/chill-java-0.3.5.jar:/opt/beh/core/storm/lib/asm-4.0.    jar:/opt/beh/core/storm/lib/jetty-util-6.1.26.jar:/opt/beh/core/storm/lib/hiccup-0.3.6.jar:/opt/beh/core/storm/lib/clout-1.0.1.    jar:/opt/beh/core/storm/lib/kryo-2.21.jar:/opt/beh/core/storm/lib/math.numeric-tower-0.0.1.jar:/opt/beh/core/storm/lib/clj-stacktrace    -0.2.2.jar:/opt/beh/core/storm/lib/commons-io-2.4.jar:/opt/beh/core/storm/lib/storm-core-0.9.5.jar:/opt/beh/core/storm/lib/jetty-6.1.    26.jar:/home/hadoop/.storm:/opt/beh/core/storm/bin backtype.storm.command.list
803  [main] INFO  backtype.storm.thrift - Connecting to Nimbus at breath:43669
Topology_name        Status     Num_tasks  Num_workers  Uptime_secs
-------------------------------------------------------------------
production-topology  ACTIVE     15         1            7043      

将storm-slider集群cl1中运行的topology名为production-topology的任务杀死,可使用如下命令: storm-slider --app cl1 kill production-topology

[hadoop@breath hadoop]$ storm-slider --app cl1 kill production-topology
2016-09-21 18:03:32,549 [main] INFO  impl.TimelineClientImpl - Timeline service address: http://breath:8188/ws/v1/timeline/
2016-09-21 18:03:33,052 [main] INFO  client.RMProxy - Connecting to ResourceManager at breath/192.168.134.171:23140
2016-09-21 18:03:34,467 [main] INFO  util.ExitUtil - Exiting with status 0
Running: java -client -Dstorm.options=nimbus.host=breath,nimbus.thrift.port=43669,storm.thrift.transport=backtype.storm.security.auth    .SimpleTransportPlugin -Dstorm.home=/opt/beh/core/storm -cp /opt/beh/core/storm/lib/logback-core-1.0.13.    jar:/opt/beh/core/storm/lib/slf4j-api-1.7.5.jar:/opt/beh/core/storm/lib/jgrapht-core-0.9.0.jar:/opt/beh/core/storm/lib/jline-2.11.    jar:/opt/beh/core/storm/lib/reflectasm-1.07-shaded.jar:/opt/beh/core/storm/lib/compojure-1.1.3.jar:/opt/beh/core/storm/lib/core.    incubator-0.1.0.jar:/opt/beh/core/storm/lib/carbonite-1.4.0.jar:/opt/beh/core/storm/lib/commons-fileupload-1.2.1.    jar:/opt/beh/core/storm/lib/minlog-1.2.jar:/opt/beh/core/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/beh/core/storm/lib/tools.    logging-0.2.3.jar:/opt/beh/core/storm/lib/snakeyaml-1.11.jar:/opt/beh/core/storm/lib/tools.cli-0.2.4.    jar:/opt/beh/core/storm/lib/clojure-1.5.1.jar:/opt/beh/core/storm/lib/clj-time-0.4.1.jar:/opt/beh/core/storm/lib/ring-core-1.1.5.    jar:/opt/beh/core/storm/lib/commons-lang-2.5.jar:/opt/beh/core/storm/lib/commons-exec-1.1.jar:/opt/beh/core/storm/lib/commons-logging    -1.1.3.jar:/opt/beh/core/storm/lib/joda-time-2.0.jar:/opt/beh/core/storm/lib/json-simple-1.1.jar:/opt/beh/core/storm/lib/log4j-over-    slf4j-1.6.6.jar:/opt/beh/core/storm/lib/ring-devel-0.3.11.jar:/opt/beh/core/storm/lib/servlet-api-2.5.    jar:/opt/beh/core/storm/lib/ring-servlet-0.3.11.jar:/opt/beh/core/storm/lib/objenesis-1.2.jar:/opt/beh/core/storm/lib/tools.macro-0.1    .0.jar:/opt/beh/core/storm/lib/logback-classic-1.0.13.jar:/opt/beh/core/storm/lib/disruptor-2.10.1.    jar:/opt/beh/core/storm/lib/commons-codec-1.6.jar:/opt/beh/core/storm/lib/chill-java-0.3.5.jar:/opt/beh/core/storm/lib/asm-4.0.    jar:/opt/beh/core/storm/lib/jetty-util-6.1.26.jar:/opt/beh/core/storm/lib/hiccup-0.3.6.jar:/opt/beh/core/storm/lib/clout-1.0.1.    jar:/opt/beh/core/storm/lib/kryo-2.21.jar:/opt/beh/core/storm/lib/math.numeric-tower-0.0.1.jar:/opt/beh/core/storm/lib/clj-stacktrace    -0.2.2.jar:/opt/beh/core/storm/lib/commons-io-2.4.jar:/opt/beh/core/storm/lib/storm-core-0.9.5.jar:/opt/beh/core/storm/lib/jetty-6.1.    26.jar:/home/hadoop/.storm:/opt/beh/core/storm/bin backtype.storm.command.kill_topology production-topology
614  [main] INFO  backtype.storm.thrift - Connecting to Nimbus at breath:43669
646  [main] INFO  backtype.storm.command.kill-topology - Killed topology: production-topology

© 著作权归作者所有

Yulong_
粉丝 10
博文 145
码字总数 253510
作品 0
朝阳
部门经理
私信 提问
Apache Storm 1.2.3 发布,分布式实时计算

Apache Storm 1.2.3 发布了,更新内容如下: 新特性 [STORM-3233] - zookeeper 客户端升级到最新版本 (3.4.13) 改进 [STORM-3077] - Disruptor 升级至 3.3.11 [STORM-3083] - HikariCP 升级至...

xplanet
07/20
817
0
Apache Storm 0.9.7 发布,分布式实时计算

Apache Storm 0.9.7 发布了,Apache Storm 的前身是 Twitter Storm 平台,目前已经归于 Apache 基金会管辖。 Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像...

开源中国股侠
2016/09/08
796
1
Apache Storm 0.10.2 发布,分布式实时计算

Apache Storm 0.10.2 发布了,Apache Storm 的前身是 Twitter Storm 平台,目前已经归于 Apache 基金会管辖。 Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像...

局长
2016/09/15
889
0
Apache Storm 1.0.3 发布,分布式实时计算

Apache Storm 1.0.3 发布了,Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像 Hadoop 一样实现实时批处理。Storm 很简单,可用于任意编程语言。Apache Storm ...

王练
2017/02/05
1K
0
Apache Storm 1.0.5 发布,分布式实时计算

Apache Storm 1.0.4 已发布,Apache Storm 是一个免费开源的分布式实时计算系统。简化了流数据的可靠处理,像 Hadoop 一样实现实时批处理。Storm 很简单,可用于任意编程语言。Apache Storm ...

王练
2017/09/16
863
5

没有更多内容

加载失败,请刷新页面

加载更多

Activity启动模式二

上篇文章Activity启动模式一主要介绍了Activity的四种启动模式,这些启动模式都是在AndroidManifest中进行配置的。除此之外,Android系统还通过Intent类提供了一些标志位,同样可以指定Activ...

ltlovezh
15分钟前
4
0
三原色还原

1、Color Filter Array — CFA 随着数码相机、手机的普及,CCD/CMOS 图像传感器近年来得到广泛的关注和应用。 图像传感器一般都采用一定的模式来采集图像数据,常用的有 BGR 模式和 CFA 模式...

天王盖地虎626
26分钟前
3
0
kubernetes pod exec接口调用

正文 一般生产环境上由于网络安全策略,大多数端口是不能为集群外部访问的。多个集群之间一般都是通过k8s的ApiServer组件提供的接口通信,如https://192.168.1.101:6443。所以在做云平台时,...

码农实战
今天
8
0
3_数组

3_数组

行者终成事
今天
8
0
经典系统设计面试题解析:如何设计TinyURL(二)

原文链接:https://www.educative.io/courses/grokking-the-system-design-interview/m2ygV4E81AR 编者注:本文以一道经典的系统设计面试题:《如何设计TinyURL》的参考答案和解析为例,帮助...

APEMESH
今天
7
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部