redis集群部署 两台服务器25节点

2019/05/09 15:43
阅读数 65

安装环境与版本

  • redis版本信息     Redis server v=3.2.12
  • 服务器操作系统 CentOS Linux release 7.4.1708 (Core)
  • 服务器IP信息      192.168.100.201(15个节点)      192.168.100.243(10个节点)  根据需求所有节点都为master节点,没有slave节点。
  • 安装redis

1. 下载并解压

cd /app
wget http: //download .redis.io /releases/redis-3 .2.12. tar .gz
tar -zxvf redis-3.2.12. tar .gz 

2. 编译安装

cd redis-3.2.12
make && make install
cd ..
mv redis-3.2.12 redis

3. 将 redis-trib.rb 复制到 /usr/local/bin 目录下

cd src
cp redis-trib.rb /usr/local/bin/  

4. 创建 Redis 节点

首先在 192.168.100.201 机器上 /app/redis 目录下创建 redis_cluster 目录;

mkdir redis_cluster  

在 redis_cluster 目录下,创建名为conf,data,logs的目录,并将 redis.conf 拷贝到conf目录中,从redis-6385.conf一直到redis-6399.conf,按顺序排列一共十五个节点,192.168.100.243节点从redis-6380.conf到redis-6389.conf,其他操作与192.168.100.201相同(以下安装步骤以201服务器为例)。

mkdir {data,logs,conf}
cp redis.conf redis_cluster/conf/redis_6385.conf cp redis.conf redis_cluster/conf/redis_6386.conf
cp redis.conf redis_cluster/conf/redis_6387.conf

分别修改配置文件,修改如下内容(由于本次生产环境数据量比较大,差不多200亿左右,故内存每个节点设定为20G,可根据实际需求做相应调整)

port (端口) //端口6385,6386 ...... 6399 
bind 本机ip //默认ip为127.0.0.1 需要改为其他节点机器可访问的ip 否则创建集群时无法访问对应的端口,无法创建集群
daemonize yes //redis后台运行
pidfile /var/run/redis_(端口).pid    //pidfile文件对应各节点端口
logfile /app/redis/redis_cluster/logs/redis-(端口).log //指定对应节点日志文件名及生成位置
dir /app/redis/redis_cluster/data //指定持久化文件存放位置
dbfilename dump-(端口).rdb   //RDB持久化文件名(以端口相对应做区分)
maxmemory 20GB //设定单节点占用服务器内存为20GB
hash-max-ziplist-entries 50000 //指定在超过一定的数量或者最大的元素超过某一临界值时,采用一种特殊的哈希算法 
hash-max-ziplist-value 1024
cluster-enabled yes //开启集群 把注释#去掉
cluster-config-file nodes_端口.conf //集群的配置 配置文件首次启动自动生成 
cluster-node-timeout 15000 //请求超时 默认15秒,可自行设置
appendonly yes //AOF日志开启,它会每次写操作都记录一条日志 
appendfilename appendonly-6385.aof //AOF持久化文件名(以端口相对应做区分) 
appendfsync no //由于数据量大导致磁盘io写入频繁导致线程阻塞,后续追加修改此参数由原先
appendfsync everysec修改为appendfsync no

 

5.启动redis集群各节点

/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6385.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6386.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6387.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6388.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6389.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6390.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6391.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6392.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6393.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6394.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6395.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6396.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6397.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6398.conf
/usr/local/bin/redis-server /app/redis/redis_cluster/conf/redis-6399.conf

 

6.查看服务启动状态。(此服务器有两个redis集群。6385...6399是本次新建的集群,6380...6384忽略)

[root@localhost ~]# ps -ef | grep redis
root       1174      1  0 4月28 ?       02:12:25 /usr/local/bin/redis-server 192.168.100.201:6380 [cluster]
root       1181      1  0 4月28 ?       02:09:54 /usr/local/bin/redis-server 192.168.100.201:6381 [cluster]
root       1200      1  0 4月28 ?       02:27:34 /usr/local/bin/redis-server 192.168.100.201:6382 [cluster]
root       1211      1  0 4月28 ?       02:12:10 /usr/local/bin/redis-server 192.168.100.201:6383 [cluster]
root       1228      1  0 4月28 ?       02:16:12 /usr/local/bin/redis-server 192.168.100.201:6384 [cluster]
root       2338      1  0 4月29 ?       00:57:55 /usr/local/bin/redis-server 192.168.100.201:6385 [cluster]
root       2342      1  0 4月29 ?       00:52:24 /usr/local/bin/redis-server 192.168.100.201:6386 [cluster]
root       2349      1  0 4月29 ?       00:57:33 /usr/local/bin/redis-server 192.168.100.201:6387 [cluster]
root      44721      1  0 4月29 ?       00:34:12 /usr/local/bin/redis-server 192.168.100.201:6388 [cluster]
root      75723      1  0 4月30 ?       00:32:53 /usr/local/bin/redis-server 192.168.100.201:6389 [cluster]
root      75742      1  0 4月30 ?       00:31:11 /usr/local/bin/redis-server 192.168.100.201:6390 [cluster]
root      75755      1  0 4月30 ?       00:30:59 /usr/local/bin/redis-server 192.168.100.201:6391 [cluster]
root      75765      1  0 4月30 ?       00:33:40 /usr/local/bin/redis-server 192.168.100.201:6392 [cluster]
root      75782      1  0 4月30 ?       00:31:11 /usr/local/bin/redis-server 192.168.100.201:6393 [cluster]
root      75796      1  0 4月30 ?       00:31:36 /usr/local/bin/redis-server 192.168.100.201:6394 [cluster]
root      75801      1  0 4月30 ?       00:33:13 /usr/local/bin/redis-server 192.168.100.201:6395 [cluster]
root      75810      1  0 4月30 ?       00:33:30 /usr/local/bin/redis-server 192.168.100.201:6396 [cluster]
root      75825      1  0 4月30 ?       00:33:27 /usr/local/bin/redis-server 192.168.100.201:6397 [cluster]
root      75837      1  0 4月30 ?       00:31:26 /usr/local/bin/redis-server 192.168.100.201:6398 [cluster]
root      75848      1  0 4月30 ?       00:31:18 /usr/local/bin/redis-server 192.168.100.201:6399 [cluster]
root     194925 193708  0 11:43 pts/0    00:00:00 grep --color=auto redis

 

7.创建集群

将所有的25个节点全部加入集群。依次将所有节点都加入集群,192.168.100.201的6385....6399     192.168.100.243的6380....6389    通过cluster  meet   IP  端口指令将节点全部添加入集群。

[root@localhost ~]# redis-cli -c -h 192.168.100.201 -p 6385
192.168.100.201:6385> CLUSTER MEET 192.168.100.201 6386
192.168.100.201:6385> CLUSTER MEET 192.168.100.201 6387
192.168.100.201:6385> CLUSTER MEET 192.168.100.201 6388

 

8.安装redis-trib所需的 ruby脚本

 安装ruby环境:   

[root@localhost redis-cluster]# yum install ruby
[root@localhost redis-cluster]# yum install rubygems

 

  安装redis-trib.rb运行依赖的ruby的包redis-3.2.2.gem,下载, 我的系统默认自带的版本是4.0的,4.0版本会在后续分配节点槽执行指令redis-trib.rb reshard时会出现报错:报错内容为:Syntax error ,try CLIENT (LIST|KILL|GETNAME|SETNAME|PAUSE|REPLY)

[root@localhost redis-cluster]# gem install redis-3.2.2.gem

 

9.对集群节点进行slot分配

目前所有的slot都在192.168.100.201服务器6385节点上,通过redis-trib.rb reshard 192.168.100.201:6385指令对所有节点重新分片,Redis集群把所有的数据映射到16384个槽中。每个key会映射为一个固定的槽,只有当节点分配了槽,才能响应和这些槽关联的键命令。我的是25个节点,平均分配到每个节点的slot数量是655个。

可以把分配的过程理解成打扑克牌,all表示大家重新洗牌;输入某个主节点的node id,然后在输入done的话,就好比从某个节点,抽牌

[root@localhost ~]# redis-trib.rb reshard 192.168.100.201:6385
>>> Performing Cluster Check (using node 192.168.100.201:6385)
M: a074fd7bca0aba40dcf93f950b6e5b6dcf1ea1e6 192.168.100.201:6385
   slots:15721-16383 (663 slots) master
   0 additional replica(s)
M: d6816a49d32f1554e0f2387ff3e774098a8a5244 192.168.100.201:6388
   slots:1311-1965 (655 slots) master
   0 additional replica(s)
M: e4e76f560773f1236963542b8da04c11e4fd02f1 192.168.100.201:6392
   slots:3931-4585 (655 slots) master
   0 additional replica(s)
M: 33c0707466932a97a9669af1b61b08f2966a4f2e 192.168.100.201:6396
   slots:6551-7205 (655 slots) master
   0 additional replica(s)
M: 518cc0289753643c55294326a35628d526653f09 192.168.100.243:6381
   slots:9826-10480 (655 slots) master
   0 additional replica(s)
M: 015bc42fc3daedc70ebf54754a1424d9c376410a 192.168.100.201:6390
   slots:2621-3275 (655 slots) master
   0 additional replica(s)
M: b1695bcdb9a1b52a3f21d288c54422f2d2bdb75e 192.168.100.201:6398
   slots:7861-8515 (655 slots) master
   0 additional replica(s)
M: 7cceae4e94af404e472117171761e5768f3b88dc 192.168.100.201:6399
   slots:8516-9170 (655 slots) master
   0 additional replica(s)
M: fd0466bddc86d534c41e7fd2ca2985cbf88b5096 192.168.100.243:6380
   slots:9171-9825 (655 slots) master
   0 additional replica(s)
M: b79687a3df98612dcce6cf300690692d489950e7 192.168.100.243:6389
   slots:15066-15720 (655 slots) master
   0 additional replica(s)
M: fc49900a75e0421c672fa3454eeba73b1bd3f6d9 192.168.100.243:6384
   slots:11791-12445 (655 slots) master
   0 additional replica(s)
M: 2e05e53de44be23ae05469be8e7886ae2393e166 192.168.100.201:6389
   slots:1966-2620 (655 slots) master
   0 additional replica(s)
M: 3b7c109dc3ad1bf9321174a8aec4571de63fa011 192.168.100.201:6395
   slots:5896-6550 (655 slots) master
   0 additional replica(s)
M: 9f967a3fe006472e25b551926face47d235ebb3b 192.168.100.243:6385
   slots:12446-13100 (655 slots) master
   0 additional replica(s)
M: 1223247236cf41fe4d38f097823af04d59af41f2 192.168.100.201:6386
   slots:0-655 (656 slots) master
   0 additional replica(s)
M: a958e8c84774e420d7a9b6ce3c40d45dab2c7676 192.168.100.243:6382
   slots:10481-11135 (655 slots) master
   0 additional replica(s)
M: 07c1c44110f915b56f68174fa1a827bde995c4b1 192.168.100.201:6387
   slots:656-1310 (655 slots) master
   0 additional replica(s)
M: 6f4273bceb9377d128419e99cd63b1b585715c92 192.168.100.243:6388
   slots:14411-15065 (655 slots) master
   0 additional replica(s)
M: 9b555a4dde023a7795df8c4fd1b146de7ca4d182 192.168.100.243:6386
   slots:13101-13755 (655 slots) master
   0 additional replica(s)
M: 8c304a6ce758ae40d0b17b018fe012cdf73b1674 192.168.100.201:6393
   slots:4586-5240 (655 slots) master
   0 additional replica(s)
M: 297044caa3e6154f9254545a042d6de520a94147 192.168.100.201:6391
   slots:3276-3930 (655 slots) master
   0 additional replica(s)
M: c3b0951bdd85665de39c844bd6834415adca0de4 192.168.100.201:6397
   slots:7206-7860 (655 slots) master
   0 additional replica(s)
M: a397bf8e21e25154800308e2035f1ae6b7c3cc77 192.168.100.201:6394
   slots:5241-5895 (655 slots) master
   0 additional replica(s)
M: 051dc42e4ca9db071c8e7e1b836aac934824294b 192.168.100.243:6383
   slots:11136-11790 (655 slots) master
   0 additional replica(s)
M: 07d33dd05095ad88ac60985e854d4e632effa2e5 192.168.100.243:6387
   slots:13756-14410 (655 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 655
What is the receiving node ID? 4458ef594b310e61bf5f249e8efd707d0da880b0          此处提示我们需要用哪个节点的nodeId来接收这些slots
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:      a074fd7bca0aba40dcf93f950b6e5b6dcf1ea1e6            此处提示我们需要从哪些节点迁移出这些slots  输入节点ID,表示从6385节点中抽取655个哈希槽转移分配给指定的节点ID。
Source node #2:done                                                      输入done结束。

......
Do you want to proceed with the proposed reshard plan (yes/no)? yes       #输入yes,redis集群就开始分配哈希槽了

 

以上节点的slots已经分配完毕,只做操作示例

分完slots以后的效果如下

[root@localhost ~]# redis-cli -c -h 192.168.100.201 -p 6385
192.168.100.201:6385> cluster nodes
d6816a49d32f1554e0f2387ff3e774098a8a5244 192.168.100.201:6388 master - 0 1557387391240 42 connected 1311-1965
e4e76f560773f1236963542b8da04c11e4fd02f1 192.168.100.201:6392 master - 0 1557387389737 46 connected 3931-4585
33c0707466932a97a9669af1b61b08f2966a4f2e 192.168.100.201:6396 master - 0 1557387387735 50 connected 6551-7205
518cc0289753643c55294326a35628d526653f09 192.168.100.243:6381 master - 0 1557387389738 55 connected 9826-10480
015bc42fc3daedc70ebf54754a1424d9c376410a 192.168.100.201:6390 master - 0 1557387386734 44 connected 2621-3275
b1695bcdb9a1b52a3f21d288c54422f2d2bdb75e 192.168.100.201:6398 master - 0 1557387388736 52 connected 7861-8515
7cceae4e94af404e472117171761e5768f3b88dc 192.168.100.201:6399 master - 0 1557387391740 53 connected 8516-9170
fd0466bddc86d534c41e7fd2ca2985cbf88b5096 192.168.100.243:6380 master - 0 1557387387735 54 connected 9171-9825
b79687a3df98612dcce6cf300690692d489950e7 192.168.100.243:6389 master - 0 1557387388736 63 connected 15066-15720
fc49900a75e0421c672fa3454eeba73b1bd3f6d9 192.168.100.243:6384 master - 0 1557387391741 58 connected 11791-12445
2e05e53de44be23ae05469be8e7886ae2393e166 192.168.100.201:6389 master - 0 1557387387735 43 connected 1966-2620
3b7c109dc3ad1bf9321174a8aec4571de63fa011 192.168.100.201:6395 master - 0 1557387392741 49 connected 5896-6550
a074fd7bca0aba40dcf93f950b6e5b6dcf1ea1e6 192.168.100.201:6385 myself,master - 0 0 30 connected 15721-16383
9f967a3fe006472e25b551926face47d235ebb3b 192.168.100.243:6385 master - 0 1557387386233 59 connected 12446-13100
1223247236cf41fe4d38f097823af04d59af41f2 192.168.100.201:6386 master - 0 1557387392741 40 connected 0-655
a958e8c84774e420d7a9b6ce3c40d45dab2c7676 192.168.100.243:6382 master - 0 1557387390740 56 connected 10481-11135
07c1c44110f915b56f68174fa1a827bde995c4b1 192.168.100.201:6387 master - 0 1557387388236 41 connected 656-1310
6f4273bceb9377d128419e99cd63b1b585715c92 192.168.100.243:6388 master - 0 1557387392240 62 connected 14411-15065
9b555a4dde023a7795df8c4fd1b146de7ca4d182 192.168.100.243:6386 master - 0 1557387389236 60 connected 13101-13755
8c304a6ce758ae40d0b17b018fe012cdf73b1674 192.168.100.201:6393 master - 0 1557387390238 47 connected 4586-5240
297044caa3e6154f9254545a042d6de520a94147 192.168.100.201:6391 master - 0 1557387389738 45 connected 3276-3930
c3b0951bdd85665de39c844bd6834415adca0de4 192.168.100.201:6397 master - 0 1557387387235 51 connected 7206-7860
a397bf8e21e25154800308e2035f1ae6b7c3cc77 192.168.100.201:6394 master - 0 1557387387735 48 connected 5241-5895
051dc42e4ca9db071c8e7e1b836aac934824294b 192.168.100.243:6383 master - 0 1557387388237 57 connected 11136-11790
07d33dd05095ad88ac60985e854d4e632effa2e5 192.168.100.243:6387 master - 0 1557387388736 61 connected 13756-14410
192.168.100.201:6385>

 

到此我们的redis集群可以正常使用了。废话不多说,就写这么多吧。

 

如有不到之处欢迎大家指点,谢谢!!!

 

展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部