[Redis] redis-cluster集群部署


本文总阅读量次

0、Redis 集群功能限制

1
2
3
4
5
6
Redis集群相对单机在功能上有一定限制。
key批量操作支持有限。如：MSET``MGET，目前只支持具有相同slot值的key执行批量操作。
key事务操作支持有限。支持多key在同一节点上的事务操作，不支持分布在多个节点的事务功能。
key作为数据分区的最小粒度，因此不能将一个大的键值对象映射到不同的节点。如：hash、list。
不支持多数据库空间。单机下Redis支持16个数据库，集群模式下只能使用一个数据库空间，即db 0。
复制结构只支持一层，不支持嵌套树状复制结构。

1、服务器三台，在每台服务器上部署两个redis服务器，节点信息如下：

1
2
3
主1：10.0.7.53-6379, 10.0.7.53-6380
主2：10.0.7.50-6379, 10.0.7.50-6380
主3：10.0.7.51-6379, 10.0.7.51-6380

2、修改redis.conf配置文件

cat /data/redis/etc/redis6379.conf(不同服务器，修改bind对应的ip即可)

1
2
3
4
5
6
7
8
9
10
11
12
pidfile /data/redis6379/log/redis.pid
bind 10.0.7.53
port 6379
daemonize yes
logfile /data/redis6379/log/redis.log
dir /data/redis6379/
databases 16
maxmemory 1g
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

cat /data/redis/etc/redis6380.conf(不同服务器，修改bind对应的ip即可)

1
2
3
4
5
6
7
8
9
10
11
12
pidfile /data/redis6380/log/redis.pid
bind 10.0.7.53
port 6380
daemonize yes
logfile /data/redis6380/log/redis.log
dir /data/redis6380/
databases 16
maxmemory 1g
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

创建指定的文件目录(指定目录不存在，启动会报错)

1
2
mkdir -p /data/redis6379/log/
mkdir -p /data/redis6380/log/

3、启动redis服务

3.1启动服务之前，修改内核参数：

1
2
3
4
5
6
7
8
9
10
echo never > /sys/kernel/mm/transparent_hugepage/enabled
添加到开机自动启动/etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
chmod +x /etc/rc.d/rc.local

echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
使修改环境变量生效
sysctl vm.overcommit_memory=1

echo 511 > /proc/sys/net/core/somaxconn

3.2在三个服务器上，分别启动redis服务

1
2
/data/redis/src/redis-server /data/redis/etc/redis6379.conf
/data/redis/src/redis-server /data/redis/etc/redis6380.conf

4、启动成功之后，在指定的dir目录下回生成一个nodes.conf的文件

1
2
3
cat /data/redis6379/nodes.conf
4933ef69f73b390098a4431449aeef2e83dacfc0 :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0

该文件记录了节点信息id，不随主机名和端口的改变而改变。

5、开始配置集群

5.1配置集群需要安装ruby

1
2
3
4
5
6
7
8
9
10
yum install -y ruby
安装完成之后运行：
gem install redis
提示报错：
----------
Fetching: redis-4.0.1.gem (100%)
ERROR:  Error installing redis:
	redis requires Ruby version >= 2.2.2.
------------
默认yum安装ruby版本为2.0.0，但是安装redis-trib.rb运行的环境，最少需要ruby-2.2.2版本

5.2需要先安装rvm，升级ruby版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
安装curl：
sudo yum install curl
安装RVM
curl -L get.rvm.io | bash -s stable
安装如果提示没有public key,按提示在命令行输入信息：
shell> gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
-----
gpg: Can't check signature: No public key
Warning, RVM 1.26.0 introduces signed releases and automated check of signatures when GPG software found. Assuming you trust Michal Papis import the mpapis public key (downloading the signatures).

GPG signature verification failed for '/usr/local/rvm/archives/rvm-1.29.4.tgz' - 'https://github.com/rvm/rvm/releases/download/1.29.4/1.29.4.tar.gz.asc'! Try to install GPG v2 and then fetch the public key:

    gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
------------------------------
使rvm环境变量生效
source /usr/local/rvm/scripts/rvm
查看rvm库中已知的ruby版本
rvm list known
安装一个ruby版本
rvm install  2.5.1
使用一个ruby版本
rvm use  2.5.1
卸载已安装的旧版本
rvm remove 2.0.0
查看版本
ruby  --version

5.3再安装redis就可以了

1
gem install redis

5.4安装完成之后，执行以下命令创建一个新的集群

1
/data/redis/src/redis-trib.rb create --replicas 1 10.0.7.53:6379 10.0.7.53:6380 10.0.7.50:6379 10.0.7.50:6380 10.0.7.51:6379 10.0.7.51:6380

–replicas 1：该参数代表为每一个主节点增加一个从节点

1
2
3
>>> Creating cluster
[ERR] Sorry, can't connect to node 10.0.7.53:6379
创建集群报错，确认gem版本没有问题，查看配置文件是否存在requirpass等参数

重新执行创建集群

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
10.0.7.53:6379
10.0.7.50:6379
10.0.7.51:6379
Adding replica 10.0.7.50:6380 to 10.0.7.53:6379
Adding replica 10.0.7.51:6380 to 10.0.7.50:6379
Adding replica 10.0.7.53:6380 to 10.0.7.51:6379
M: 4933ef69f73b390098a4431449aeef2e83dacfc0 10.0.7.53:6379
   slots:0-5460 (5461 slots) master
S: 3f757295e082a5fbe237015862d0a502779f3adf 10.0.7.53:6380
   replicates 3e31b1ceded27ad5b4114beaebb559d7f5ca465d
M: 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4 10.0.7.50:6379
   slots:5461-10922 (5462 slots) master
S: 5b3aa1532d911e1a9b966ec53e3203a3221056e3 10.0.7.50:6380
   replicates 4933ef69f73b390098a4431449aeef2e83dacfc0
M: 3e31b1ceded27ad5b4114beaebb559d7f5ca465d 10.0.7.51:6379
   slots:10923-16383 (5461 slots) master
S: 30aabebd554d34e59584774feebd472f622f1cc2 10.0.7.51:6380
   replicates 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 10.0.7.53:6379)
M: 4933ef69f73b390098a4431449aeef2e83dacfc0 10.0.7.53:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4 10.0.7.50:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 3e31b1ceded27ad5b4114beaebb559d7f5ca465d 10.0.7.51:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 30aabebd554d34e59584774feebd472f622f1cc2 10.0.7.51:6380
   slots: (0 slots) slave
   replicates 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4
S: 3f757295e082a5fbe237015862d0a502779f3adf 10.0.7.53:6380
   slots: (0 slots) slave
   replicates 3e31b1ceded27ad5b4114beaebb559d7f5ca465d
S: 5b3aa1532d911e1a9b966ec53e3203a3221056e3 10.0.7.50:6380
   slots: (0 slots) slave
   replicates 4933ef69f73b390098a4431449aeef2e83dacfc0
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
----------------------------------

5.5集群安装完成，查看集群状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
/data/redis/src/redis-trib.rb check 10.0.7.53:6379
（或者 进入控制台/data/redis/src/redis-cli -h  10.0.7.53，输入 CLUSTER nodes)
>>> Performing Cluster Check (using node 10.0.7.53:6379)
M: 4933ef69f73b390098a4431449aeef2e83dacfc0 10.0.7.53:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4 10.0.7.50:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 3e31b1ceded27ad5b4114beaebb559d7f5ca465d 10.0.7.51:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 30aabebd554d34e59584774feebd472f622f1cc2 10.0.7.51:6380
   slots: (0 slots) slave
   replicates 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4
S: 3f757295e082a5fbe237015862d0a502779f3adf 10.0.7.53:6380
   slots: (0 slots) slave
   replicates 3e31b1ceded27ad5b4114beaebb559d7f5ca465d
S: 5b3aa1532d911e1a9b966ec53e3203a3221056e3 10.0.7.50:6380
   slots: (0 slots) slave
   replicates 4933ef69f73b390098a4431449aeef2e83dacfc0
[OK] All nodes agree about slots configuration.

6、集群安装完成，测试集群是否正常运行

6.1杀掉任意一个主节点，从节点自动切换为主，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/data/redis/src/redis-trib.rb check 10.0.7.53:6380
>>> Performing Cluster Check (using node 10.0.7.53:6380)
S: 3f757295e082a5fbe237015862d0a502779f3adf 10.0.7.53:6380
   slots: (0 slots) slave
   replicates 3e31b1ceded27ad5b4114beaebb559d7f5ca465d
M: 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4 10.0.7.50:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 5b3aa1532d911e1a9b966ec53e3203a3221056e3 10.0.7.50:6380
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
S: 30aabebd554d34e59584774feebd472f622f1cc2 10.0.7.51:6380
   slots: (0 slots) slave
   replicates 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4
M: 3e31b1ceded27ad5b4114beaebb559d7f5ca465d 10.0.7.51:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)

6.2重新启动别杀掉的节点之后，该节点自动变为从：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/data/redis/src/redis-trib.rb check 10.0.7.53:6380
>>> Performing Cluster Check (using node 10.0.7.53:6380)
S: 3f757295e082a5fbe237015862d0a502779f3adf 10.0.7.53:6380
   slots: (0 slots) slave
   replicates 3e31b1ceded27ad5b4114beaebb559d7f5ca465d
M: 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4 10.0.7.50:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 4933ef69f73b390098a4431449aeef2e83dacfc0 10.0.7.53:6379
   slots: (0 slots) slave
   replicates 5b3aa1532d911e1a9b966ec53e3203a3221056e3
M: 5b3aa1532d911e1a9b966ec53e3203a3221056e3 10.0.7.50:6380
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 30aabebd554d34e59584774feebd472f622f1cc2 10.0.7.51:6380
   slots: (0 slots) slave
   replicates 27eb4895dc9369adcd81d9a00709dfbf6fbbd0d4
M: 3e31b1ceded27ad5b4114beaebb559d7f5ca465d 10.0.7.51:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
redis
目录
  1. 1. 0、Redis 集群功能限制
  2. 2. 1、服务器三台，在每台服务器上部署两个redis服务器，节点信息如下：
  3. 3. 2、修改redis.conf配置文件
  4. 4. 3、启动redis服务
  5. 5. 4、启动成功之后，在指定的dir目录下回生成一个nodes.conf的文件
  6. 6. 5、开始配置集群
  7. 7. 6、集群安装完成，测试集群是否正常运行

Proudly powered by Hexo and Theme by Lap
本站访客数人次
© 2020 zeven0707's blog