文档章节

Linux系统Kafka集群搭建与简单测试

L
 Listen_ing
发布于 2016/06/02 18:23
字数 3144
阅读 852
收藏 2

Kafka安装

  • Zookeeper集群搭建,可参考Linux系统Zookeeper集群配置
  • 上传kafka安装包并解压
  • [root@localhost ~]# cd /home/listen
    [root@localhost listen]# ll
    total 8
    drwxr-xr-x.  3 listen listen   23 Apr 27 01:48 codis
    drwxr-xr-x.  3 root   root     19 May 18 19:37 dubbox
    drwxrwxr-x.  3 listen listen   48 Apr 26 20:16 go
    drwxrwxr-x.  4 listen listen   64 May 13 03:14 gradle
    drwxrwxr-x.  2 listen listen   35 Apr 26 19:53 jdk
    drwxr-xr-x.  3 root   root     64 May 18 18:46 maven
    drwxrwxr-x.  4 listen listen  100 May  3 00:52 nginx
    drwxrwxr-x.  4 listen listen   66 Apr 26 19:31 redis
    drwxrwxr-x.  3 listen listen   94 May 10 00:42 session
    drwxrwxr-x.  4 listen listen 4096 May 10 17:58 tomcat
    drwxrwxr-x. 10 listen listen 4096 May 11 03:22 zookeeper
    [root@localhost listen]# mkdir kafka
    [root@localhost listen]# cd kafka/
    [root@localhost kafka]# ll
    total 30840
    -rw-r--r--. 1 root root 31579147 May 30 00:36 kafka_2.11-0.10.0.0.tgz
    [root@localhost kafka]# tar -xvf kafka_2.11-0.10.0.0.tgz
    
  • 编辑配置文件

  • [root@localhost config]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0/config
    [root@localhost config]# ls
    connect-console-sink.properties    connect-file-sink.properties    connect-standalone.properties  producer.properties     zookeeper.properties
    connect-console-source.properties  connect-file-source.properties  consumer.properties            server.properties
    connect-distributed.properties     connect-log4j.properties        log4j.properties               tools-log4j.properties
    [root@localhost config]# vi server.properties
    
     这个是server.properties文件内容
    # Licensed to the Apache Software Foundation (ASF) under one or more
    # contributor license agreements.  See the NOTICE file distributed with
    # this work for additional information regarding copyright ownership.
    # The ASF licenses this file to You under the Apache License, Version 2.0
    # (the "License"); you may not use this file except in compliance with
    # the License.  You may obtain a copy of the License at
    #
    #    http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    # see kafka.server.KafkaConfig for additional details and defaults
    
    ############################# Server Basics #############################
    
    # The id of the broker. This must be set to a unique integer for each broker.
    broker.id=0
    
    ############################# Socket Server Settings #############################
    
    # The address the socket server listens on. It will get the value returned from 
    # java.net.InetAddress.getCanonicalHostName() if not configured.
    #   FORMAT:
    #     listeners = security_protocol://host_name:port
    #   EXAMPLE:
    #     listeners = PLAINTEXT://your.host.name:9092
    listeners=PLAINTEXT://:9092
    
    # Hostname and port the broker will advertise to producers and consumers. If not set, 
    # it uses the value for "listeners" if configured.  Otherwise, it will use the value
    # returned from java.net.InetAddress.getCanonicalHostName().
    #advertised.listeners=PLAINTEXT://your.host.name:9092
    advertised.listeners=PLAINTEXT://192.168.75.141:9092
    
    port=9092
    host.name=192.168.75.141
    
    advertised.host.name=192.168.75.141
    advertised.port=9092
    # The number of threads handling network requests
    num.network.threads=3
    
    # The number of threads doing disk I/O
    num.io.threads=8
    
    # The send buffer (SO_SNDBUF) used by the socket server
    socket.send.buffer.bytes=102400
    
    # The receive buffer (SO_RCVBUF) used by the socket server
    socket.receive.buffer.bytes=102400
    
    # The maximum size of a request that the socket server will accept (protection against OOM)
    socket.request.max.bytes=104857600
    
    
    ############################# Log Basics #############################
    
    # A comma seperated list of directories under which to store log files
    log.dirs=/home/listen/kafka/kafka_2.11-0.10.0.0/logs/server0
    
    # The default number of log partitions per topic. More partitions allow greater
    # parallelism for consumption, but this will also result in more files across
    # the brokers.
    num.partitions=1
    
    # The number of threads per data directory to be used for log recovery at startup and flushing at shutdown.
    # This value is recommended to be increased for installations with data dirs located in RAID array.
    num.recovery.threads.per.data.dir=1
    
    ############################# Log Flush Policy #############################
    
    # Messages are immediately written to the filesystem but by default we only fsync() to sync
    # the OS cache lazily. The following configurations control the flush of data to disk.
    # There are a few important trade-offs here:
    #    1. Durability: Unflushed data may be lost if you are not using replication.
    #    2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush.
    #    3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to exceessive seeks.
    # The settings below allow one to configure the flush policy to flush data after a period of time or
    # every N messages (or both). This can be done globally and overridden on a per-topic basis.
    
    # The number of messages to accept before forcing a flush of data to disk
    #log.flush.interval.messages=10000
    
    # The maximum amount of time a message can sit in a log before we force a flush
    #log.flush.interval.ms=1000
    
    ############################# Log Retention Policy #############################
    
    # The following configurations control the disposal of log segments. The policy can
    # be set to delete segments after a period of time, or after a given size has accumulated.
    # A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
    # from the end of the log.
    
    # The minimum age of a log file to be eligible for deletion
    log.retention.hours=168
    
    # A size-based retention policy for logs. Segments are pruned from the log as long as the remaining
    # segments don't drop below log.retention.bytes.
    #log.retention.bytes=1073741824
    
    # The maximum size of a log segment file. When this size is reached a new log segment will be created.
    log.segment.bytes=1073741824
    
    # The interval at which log segments are checked to see if they can be deleted according
    # to the retention policies
    log.retention.check.interval.ms=300000
    
    ############################# Zookeeper #############################
    
    # Zookeeper connection string (see zookeeper docs for details).
    # This is a comma separated host:port pairs, each corresponding to a zk
    # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
    # You can also append an optional chroot string to the urls to specify the
    # root directory for all kafka znodes.
    zookeeper.connect=192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185
    
    # Timeout in ms for connecting to zookeeper
    zookeeper.connection.timeout.ms=6000
    
    

     

  • 增加集群的配置文件server1.properties
  • [root@localhost config]# cp server.properties server1.properties 
    [root@localhost config]# vi server1.properties 
     修改配置
    broker.id=1
    listeners=PLAINTEXT://:9093
    advertised.listeners=PLAINTEXT://192.168.75.141:9093
    port=9093
    host.name=192.168.75.141
    advertised.host.name=192.168.75.141
    advertised.port=9093
    log.dirs=/home/listen/kafka/kafka_2.11-0.10.0.0/logs/server1
    #其他配置和server.properties一样

     

  •  增加集群的配置文件server2.properties
  • [root@localhost config]# cp server.properties server2.properties 
    [root@localhost config]# vi server2.properties 

    修改配置

  • broker.id=2
    listeners=PLAINTEXT://:9094
    advertised.listeners=PLAINTEXT://192.168.75.141:9094
    port=9094
    host.name=192.168.75.141
    advertised.host.name=192.168.75.141
    advertised.port=9094
    log.dirs=/home/listen/kafka/kafka_2.11-0.10.0.0/logs/server2
    #其他配置和server.properties一样

     

  • 创建日志文件目录
  • [root@localhost ~]# cd /home/listen/kafka/
    [root@localhost kafka]# ll
    total 30840
    drwxr-xr-x. 6 root root       83 May 17 21:32 kafka_2.11-0.10.0.0
    -rw-r--r--. 1 root root 31579147 May 30 00:36 kafka_2.11-0.10.0.0.tgz
    [root@localhost kafka]# cd kafka_2.11-0.10.0.0/
    [root@localhost kafka_2.11-0.10.0.0]# ll
    total 48
    drwxr-xr-x. 3 root root  4096 May 17 21:32 bin
    drwxr-xr-x. 2 root root  4096 May 30 00:39 config
    drwxr-xr-x. 2 root root  4096 May 30 00:36 libs
    -rw-r--r--. 1 root root 28824 May 17 21:26 LICENSE
    -rw-r--r--. 1 root root   336 May 17 21:26 NOTICE
    drwxr-xr-x. 2 root root    46 May 17 21:32 site-docs
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# mkdir logs
    [root@localhost kafka_2.11-0.10.0.0]# cd logs
    [root@localhost logs]# mkdir server0
    [root@localhost logs]# mkdir server1
    [root@localhost logs]# mkdir server2
    [root@localhost logs]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0/logs
    [root@localhost logs]#
    
  • 先启动Zookeeper集群

  • 具体操作方法请查看Linux系统Zookeeper集群配置
  • 然后启动Kafka集群

  • [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh -daemon config/server.properties 
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh -daemon config/server1.properties 
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh -daemon config/server2.properties 
    #以上是在后台启动,如果需要在console控制台启动看到运行信息,可参考下面的方式,这种方式需要开多窗口,否则无法启动第二个第三个节点
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh config/server.properties 
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh config/server1.properties 
    [root@localhost kafka_2.11-0.10.0.0]# pwd
    /home/listen/kafka/kafka_2.11-0.10.0.0
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-server-start.sh config/server2.properties 
    
  • 启动成功,连接Zookeeper查看Kafka相关信息
  • [root@localhost zookeeper]# pwd
    /home/listen/zookeeper
    [root@localhost zookeeper]# zookeeper-2181/bin/zkCli.sh -server 192.168.75.141:2181
    [zk: 192.168.75.141:2181(CONNECTED) 0] ls /
    [dubbo, isr_change_notification, admin, zookeeper, consumers, config, controller, zk, brokers, controller_epoch]
    [zk: 192.168.75.141:2181(CONNECTED) 1] ls /brokers
    [seqid, topics, ids]
    [zk: 192.168.75.141:2181(CONNECTED) 2] ls /brokers/seqid
    []
    [zk: 192.168.75.141:2181(CONNECTED) 3] ls /brokers/topics
    []
    [zk: 192.168.75.141:2181(CONNECTED) 4] ls /brokers/ids   
    [2, 1, 0]
    [zk: 192.168.75.141:2181(CONNECTED) 5] 
    
      
  • Kafka常用命令操作
  • #手动创建topic,默认的topic只有一个分区,所以手动创建最佳
    [root@localhost ~]# cd /home/listen/kafka/kafka_2.11-0.10.0.0/
    #创建备份2个,分区2个,名称为meng的topic
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-topics.sh --create --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --replication-factor 2 --partitions 2 --topic meng
    Created topic "meng".
    [root@localhost kafka_2.11-0.10.0.0]#
    #创建备份2个,分区8个,名称为t-meng的topic
    [root@localhost bin]# ./kafka-topics.sh --create --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --replication-factor 2 --partitions 8 --topic t-meng
    Created topic "t-meng".
    [root@localhost bin]#
    #将刚才的t-meng topic的分区修改为10个,即增加2个
    [root@localhost bin]# ./kafka-topics.sh --alter --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --partitions 10 --topic t-meng
    
    WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
    Adding partitions succeeded!
    [root@localhost bin]#
    #减少分区,报错,分区只能增加不能减少
    [root@localhost bin]# ./kafka-topics.sh --alter --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --partitions 6 --topic t-meng
    WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
    Error while executing topic command : The number of partitions for a topic can only be increased
    [2016-06-02 00:58:31,143] ERROR kafka.admin.AdminOperationException: The number of partitions for a topic can only be increased
    	at kafka.admin.AdminUtils$.addPartitions(AdminUtils.scala:262)
    	at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:148)
    	at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:125)
    	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    	at kafka.admin.TopicCommand$.alterTopic(TopicCommand.scala:125)
    	at kafka.admin.TopicCommand$.main(TopicCommand.scala:63)
    	at kafka.admin.TopicCommand.main(TopicCommand.scala)
     (kafka.admin.TopicCommand$)
    [root@localhost bin]#
    #查看分区信息
    [root@localhost kafka_2.11-0.10.0.0]# bin/kafka-topics.sh --describe --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185  --topic meng
    Topic:meng	PartitionCount:2	ReplicationFactor:2	Configs:
    	Topic: meng	Partition: 0	Leader: 1	Replicas: 1,2	Isr: 1,2
    	Topic: meng	Partition: 1	Leader: 2	Replicas: 2,0	Isr: 2,0
    [root@localhost kafka_2.11-0.10.0.0]#
    
    #列出所有的topic
    [root@localhost bin]# ./kafka-topics.sh --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --list
    __consumer_offsets
    bar
    foo
    meng
    my-topic
    qiang
    t-zhou
    t-zhou-2
    t-zhou-3
    test
    zhou
    [root@localhost bin]#
    #删除一个topic,慎用
    [root@localhost bin]# ./kafka-topics.sh --delete --topic t-meng --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185
    Topic t-meng is marked for deletion.
    Note: This will have no impact if delete.topic.enable is not set to true.
    [root@localhost bin]# ./kafka-topics.sh --list --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185
    __consumer_offsets
    bar
    foo
    meng
    my-topic
    qiang
    t-meng - marked for deletion
    t-zhou
    t-zhou-2
    t-zhou-3
    test
    zhou
    #marked for deletion标示为删除的标记,意思为逻辑删除,真正需要删除需要在server.properties文件中配置,标记为删除的topic其实还可以使用,没有任何的区别。
    #delete.topic.enable=true #是否真正的删除指定的topic,默认为false,可选配置
    
    #查看所有topic的分布描述
    [root@localhost bin]# ./kafka-topics.sh  --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --describe
    Topic:__consumer_offsets	PartitionCount:50	ReplicationFactor:3	Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
    	Topic: __consumer_offsets	Partition: 0	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 1	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 2	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 3	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 4	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 5	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 6	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 7	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 8	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 9	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 10	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 11	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 12	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 13	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 14	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 15	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 16	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 17	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 18	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 19	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 20	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 21	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 22	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 23	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 24	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 25	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 26	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 27	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 28	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 29	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 30	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 31	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 32	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 33	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 34	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 35	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 36	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 37	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 38	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 39	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 40	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 41	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 42	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 43	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 44	Leader: 0	Replicas: 0,1,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 45	Leader: 1	Replicas: 1,0,2	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 46	Leader: 2	Replicas: 2,1,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 47	Leader: 0	Replicas: 0,2,1	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 48	Leader: 1	Replicas: 1,2,0	Isr: 0,1,2
    	Topic: __consumer_offsets	Partition: 49	Leader: 2	Replicas: 2,0,1	Isr: 0,1,2
    Topic:bar	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: bar	Partition: 0	Leader: 2	Replicas: 2	Isr: 2
    Topic:foo	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: foo	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
    Topic:meng	PartitionCount:2	ReplicationFactor:2	Configs:
    	Topic: meng	Partition: 0	Leader: 1	Replicas: 1,2	Isr: 1,2
    	Topic: meng	Partition: 1	Leader: 2	Replicas: 2,0	Isr: 0,2
    Topic:my-topic	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: my-topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
    Topic:qiang	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: qiang	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
    Topic:t-meng	PartitionCount:10	ReplicationFactor:2	Configs:
    	Topic: t-meng	Partition: 0	Leader: 0	Replicas: 0,2	Isr: 0,2
    	Topic: t-meng	Partition: 1	Leader: 1	Replicas: 1,0	Isr: 1,0
    	Topic: t-meng	Partition: 2	Leader: 2	Replicas: 2,1	Isr: 2,1
    	Topic: t-meng	Partition: 3	Leader: 0	Replicas: 0,1	Isr: 0,1
    	Topic: t-meng	Partition: 4	Leader: 1	Replicas: 1,2	Isr: 1,2
    	Topic: t-meng	Partition: 5	Leader: 2	Replicas: 2,0	Isr: 2,0
    	Topic: t-meng	Partition: 6	Leader: 0	Replicas: 0,2	Isr: 0,2
    	Topic: t-meng	Partition: 7	Leader: 1	Replicas: 1,0	Isr: 1,0
    	Topic: t-meng	Partition: 8	Leader: 2	Replicas: 2,0	Isr: 2,0
    	Topic: t-meng	Partition: 9	Leader: 0	Replicas: 0,2	Isr: 0,2
    Topic:t-zhou	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: t-zhou	Partition: 0	Leader: 2	Replicas: 2	Isr: 2
    Topic:t-zhou-2	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: t-zhou-2	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
    Topic:t-zhou-3	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: t-zhou-3	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
    Topic:test	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: test	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
    Topic:zhou	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: zhou	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
    [root@localhost bin]#
    
    #查看指定的topic信息
    [root@localhost bin]# ./kafka-topics.sh  --zookeeper 192.168.75.141:2181,192.168.75.141:2182,192.168.75.141:2183,192.168.75.141:2184,192.168.75.141:2185 --describe --topic t-meng
    Topic:t-meng	PartitionCount:10	ReplicationFactor:2	Configs:
    	Topic: t-meng	Partition: 0	Leader: 0	Replicas: 0,2	Isr: 0,2
    	Topic: t-meng	Partition: 1	Leader: 1	Replicas: 1,0	Isr: 1,0
    	Topic: t-meng	Partition: 2	Leader: 2	Replicas: 2,1	Isr: 2,1
    	Topic: t-meng	Partition: 3	Leader: 0	Replicas: 0,1	Isr: 0,1
    	Topic: t-meng	Partition: 4	Leader: 1	Replicas: 1,2	Isr: 1,2
    	Topic: t-meng	Partition: 5	Leader: 2	Replicas: 2,0	Isr: 2,0
    	Topic: t-meng	Partition: 6	Leader: 0	Replicas: 0,2	Isr: 0,2
    	Topic: t-meng	Partition: 7	Leader: 1	Replicas: 1,0	Isr: 1,0
    	Topic: t-meng	Partition: 8	Leader: 2	Replicas: 2,0	Isr: 2,0
    	Topic: t-meng	Partition: 9	Leader: 0	Replicas: 0,2	Isr: 0,2
    [root@localhost bin]#
    
  • 客户端测试Kafka

  • 可参考 Kafka集群简单测试

© 著作权归作者所有

L
粉丝 13
博文 28
码字总数 53225
作品 0
宝山
程序员
私信 提问
2.1、Linux下Kafka环境搭建及基本使用

一、kafka单节点单broker环境搭建 系统环境:centos 64位 mini 虚拟机:Virtual Box 如果你不熟悉virtual box的使用,不知道如何安装centos系统,可以参考之前发布的文章: 快速掌握基于Vir...

极客慧
2018/11/26
1K
0
Kafka1.1.0集群的简单使用(java)

Kafka本地集群搭建完成,简单入门Kafka集群的shell操作后,开始Java代码实现简单功能。 kafka版本说明:此处使用最新版本(现在)—1.1.0版本 1.1.0 is the latest release. The current sta...

tian330726
2018/04/19
0
0
filebeat+kafka+logstash+elasticsearch

服务架构 由于博主是在本机测试,所以的filebeat和logstash是用的传统方式安装,其他组件用的docker容器。 除了filebeat,其他组件会用到java虚拟机,所以为了方便,最好都走docker。我这为了...

xiaoxlm
2018/07/05
1K
0
Kafka集群简单测试

接上篇Linux系统Kafka集群搭建与简单测试,做简单的测试 新建common工程 新建listen-kafka-common工程,主要放置生产端和消费端公用部分代码 工程目录截图如下图 pom.xml文件如下 编写代码 ...

Listen_ing
2016/06/02
1K
1
Zookeeper+Kafka集群搭建

Zookeeper集群搭建 Kafka集群是把状态保存在Zookeeper中的,首先要搭建Zookeeper集群。 1、软件环境 (3台服务器-我的测试) 192.168.30.204 server1 192.168.30.205 server2 192.168.30.206...

qianghong000
2018/06/26
0
0

没有更多内容

加载失败,请刷新页面

加载更多

java通过ServerSocket与Socket实现通信

首先说一下ServerSocket与Socket. 1.ServerSocket ServerSocket是用来监听客户端Socket连接的类,如果没有连接会一直处于等待状态. ServetSocket有三个构造方法: (1) ServerSocket(int port);...

Blueeeeeee
今天
6
0
用 Sphinx 搭建博客时,如何自定义插件?

之前有不少同学看过我的个人博客(http://python-online.cn),也根据我写的教程完成了自己个人站点的搭建。 点此:使用 Python 30分钟 教你快速搭建一个博客 为防有的同学不清楚 Sphinx ,这...

王炳明
昨天
5
0
黑客之道-40本书籍助你快速入门黑客技术免费下载

场景 黑客是一个中文词语,皆源自英文hacker,随着灰鸽子的出现,灰鸽子成为了很多假借黑客名义控制他人电脑的黑客技术,于是出现了“骇客”与"黑客"分家。2012年电影频道节目中心出品的电影...

badaoliumang
昨天
15
0
很遗憾,没有一篇文章能讲清楚线程的生命周期!

(手机横屏看源码更方便) 注:java源码分析部分如无特殊说明均基于 java8 版本。 简介 大家都知道线程是有生命周期,但是彤哥可以认真负责地告诉你网上几乎没有一篇文章讲得是完全正确的。 ...

彤哥读源码
昨天
17
0
jquery--DOM操作基础

本文转载于:专业的前端网站➭jquery--DOM操作基础 元素的访问 元素属性操作 获取:attr(name);$("#my").attr("src"); 设置:attr(name,value);$("#myImg").attr("src","images/1.jpg"); ......

前端老手
昨天
7
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部