kafka sink

原创
2017/04/21 13:11
阅读数 94

flume版本1.6 kafka版本0.8.2

创建配置目录

mkdir /opt/beh/core/flume/conf.d

创建采集数据目录

mkdir /opt/beh/data/flume/kafka-agent

创建配置文件 /opt/beh/core/flume/conf.d/kafka-agent.conf

kafka-agent.sources = src1
kafka-agent.sinks = sink1
kafka-agent.channels = ch1

kafka-agent.channels.ch1.type = memory

kafka-agent.sources.src1.type = spooldir
kafka-agent.sources.src1.spoolDir = /opt/beh/data/flume/kafka-agent
kafka-agent.sources.src1.channels = ch1

kafka-agent.sinks.sink1.channel = ch1  
kafka-agent.sinks.sink1.type = org.apache.flume.sink.kafka.KafkaSink
kafka-agent.sinks.sink1.topic = test
kafka-agent.sinks.sink1.brokerList = localhost:9092
kafka-agent.sinks.sink1.requiredAcks = 1
kafka-agent.sinks.sink1.batchSize = 20

flume agent启动命令

bin/flume-ng agent --conf-file /opt/beh/core/flume/conf.d/kafka-agent.conf --name kafka-agent -Dflum.root.logger=INFO,console
展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部