Redis集群詳解
Redis有三種集群模式,分別是:
* 主從模式
* Sentinel模式
* Cluster模式
三種集群模式各有特點(diǎn),關(guān)于Redis介紹可以參考這里:NoSQL(二)——Redis
Redis官網(wǎng):https://redis.io/ ,最新版本6.0.5
主從模式
主從模式介紹
主從模式是三種模式中最簡單的,在主從復(fù)制中,數(shù)據(jù)庫分為兩類:主數(shù)據(jù)庫(master)和從數(shù)據(jù)庫(slave)。
其中主從復(fù)制有如下特點(diǎn):
* 主數(shù)據(jù)庫可以進(jìn)行讀寫操作,當(dāng)讀寫操作導(dǎo)致數(shù)據(jù)變化時(shí)會(huì)自動(dòng)將數(shù)據(jù)同步給從數(shù)據(jù)庫
* 從數(shù)據(jù)庫一般都是只讀的,并且接收主數(shù)據(jù)庫同步過來的數(shù)據(jù)
* 一個(gè)master可以擁有多個(gè)slave,但是一個(gè)slave只能對應(yīng)一個(gè)master
* slave掛了不影響其他slave的讀和master的讀和寫,重新啟動(dòng)后會(huì)將數(shù)據(jù)從master同步過來
* master掛了以后,不影響slave的讀,但redis不再提供寫服務(wù),master重啟后redis將重新對外提供寫服務(wù)
* master掛了以后,不會(huì)在slave節(jié)點(diǎn)中重新選一個(gè)master
工作機(jī)制:
當(dāng)slave啟動(dòng)后,主動(dòng)向master發(fā)送SYNC命令。master接收到SYNC命令后在后臺保存快照(RDB持久化)和緩存保存快照這段時(shí)間的命令,然后將保存的快照文件和緩存的命令發(fā)送給slave。slave接收到快照文件和命令后加載快照文件和緩存的執(zhí)行命令。
復(fù)制初始化后,master每次接收到的寫命令都會(huì)同步發(fā)送給slave,保證主從數(shù)據(jù)一致性。
安全設(shè)置:
當(dāng)master節(jié)點(diǎn)設(shè)置密碼后,
客戶端訪問master需要密碼
啟動(dòng)slave需要密碼,在配置文件中配置即可
客戶端訪問slave不需要密碼
缺點(diǎn):
從上面可以看出,master節(jié)點(diǎn)在主從模式中唯一,若master掛掉,則redis無法對外提供寫服務(wù)。
主從模式搭建
環(huán)境準(zhǔn)備:
master節(jié)點(diǎn) 192.168.30.128
slave節(jié)點(diǎn) 192.168.30.129
slave節(jié)點(diǎn) 192.168.30.130
全部下載安裝:
# cd /software
# wget http://download.redis.io/releases/redis-6.0.5.tar.gz
# tar zxf redis-5.0.4.tar.gz mv redis-6.0.5/ /usr/local/redis
# cd /usr/local/redis make make install
# echo $?
0
全部配置成服務(wù):
服務(wù)文件
# vim /usr/lib/systemd/system/redis.service
[Unit]
Description=Redis persistent key-value database
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/usr/local/bin/redis-server /usr/local/redis/redis.conf --supervised systemd
ExecStop=/usr/libexec/redis-shutdown
Type=notify
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755
[Install]
WantedBy=multi-user.target
shutdown腳本
# vim /usr/libexec/redis-shutdown
#!/bin/bash
#
# Wrapper to close properly redis and sentinel
test x"$REDIS_DEBUG" != x set -x
REDIS_CLI=/usr/local/bin/redis-cli
# Retrieve service name
SERVICE_NAME="$1"
if [ -z "$SERVICE_NAME" ]; then
SERVICE_NAME=redis
fi
# Get the proper config file based on service name
CONFIG_FILE="/usr/local/redis/$SERVICE_NAME.conf"
# Use awk to retrieve host, port from config file
HOST=`awk '/^[[:blank:]]*bind/ { print $2 }' $CONFIG_FILE | tail -n1`
PORT=`awk '/^[[:blank:]]*port/ { print $2 }' $CONFIG_FILE | tail -n1`
PASS=`awk '/^[[:blank:]]*requirepass/ { print $2 }' $CONFIG_FILE | tail -n1`
SOCK=`awk '/^[[:blank:]]*unixsocket\s/ { print $2 }' $CONFIG_FILE | tail -n1`
# Just in case, use default host, port
HOST=${HOST:-127.0.0.1}
if [ "$SERVICE_NAME" = redis ]; then
PORT=${PORT:-6379}
else
PORT=${PORT:-26739}
fi
# Setup additional parameters
# e.g password-protected redis instances
[ -z "$PASS" ] || ADDITIONAL_PARAMS="-a $PASS"
# shutdown the service properly
if [ -e "$SOCK" ] ; then
$REDIS_CLI -s $SOCK $ADDITIONAL_PARAMS shutdown
else
$REDIS_CLI -h $HOST -p $PORT $ADDITIONAL_PARAMS shutdown
fi
# chmod +x /usr/libexec/redis-shutdown
# useradd -s /sbin/nologin redis
# chown -R redis:redis /usr/local/redis
# chown -R reids:redis /data/redis
# yum install -y bash-completion source /etc/profile #命令補(bǔ)全
# systemctl daemon-reload
# systemctl enable redis
修改配置:
192.168.30.128
# mkdir -p /data/redis
# vim /usr/local/redis/redis.conf
bind 192.168.30.128 #監(jiān)聽ip,多個(gè)ip用空格分隔
daemonize yes #允許后臺啟動(dòng)
logfile "/usr/local/redis/redis.log" #日志路徑
dir /data/redis #數(shù)據(jù)庫備份文件存放目錄
masterauth 123456 #slave連接master密碼,master可省略
requirepass 123456 #設(shè)置master連接密碼,slave可省略
appendonly yes #在/data/redis/目錄生成appendonly.aof文件,將每一次寫操作請求都追加到appendonly.aof 文件中
# echo 'vm.overcommit_memory=1' >> /etc/sysctl.conf
# sysctl -p
192.168.30.129
# mkdir -p /data/redis
# vim /usr/local/redis/redis.conf
bind 192.168.30.129
daemonize yes
logfile "/usr/local/redis/redis.log"
dir /data/redis
replicaof 192.168.30.128 6379
masterauth 123456
requirepass 123456
appendonly yes
# echo 'vm.overcommit_memory=1' >> /etc/sysctl.conf
# sysctl -p
192.168.30.130
# mkdir -p /data/redis
# vim /usr/local/redis/redis.conf
bind 192.168.30.130
daemonize yes
logfile "/usr/local/redis/redis.log"
dir /data/redis
replicaof 192.168.30.128 6379
masterauth 123456
requirepass 123456
appendonly yes
# echo 'vm.overcommit_memory=1' >> /etc/sysctl.conf
# sysctl -p
全部啟動(dòng)redis:
# systemctl start redis
查看集群狀態(tài):
# redis-cli -h 192.168.30.128 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.128:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.30.129,port=6379,state=online,offset=168,lag=1
slave1:ip=192.168.30.130,port=6379,state=online,offset=168,lag=1
master_replid:fb4941e02d5032ad74c6e2383211fc58963dbe90
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:168
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:168
# redis-cli -h 192.168.30.129 -a 123456 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:slave
master_host:192.168.30.128
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:196
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:fb4941e02d5032ad74c6e2383211fc58963dbe90
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:196
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:196
數(shù)據(jù)演示:
192.168.30.128:6379> keys *
(empty list or set)
192.168.30.128:6379> set key1 100
OK
192.168.30.128:6379> set key2 lzx
OK
192.168.30.128:6379> keys *
1) "key1"
2) "key2"
# redis-cli -h 192.168.30.129 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.129:6379> keys *
1) "key2"
2) "key1"
192.168.30.129:6379> CONFIG GET dir
1) "dir"
2) "/data/redis"
192.168.30.129:6379> CONFIG GET dbfilename
1) "dbfilename"
2) "dump.rdb"
192.168.30.129:6379> get key1
"100"
192.168.30.129:6379> get key2
"lzx"
192.168.30.129:6379> set key3 aaa
(error) READONLY You can't write against a read only replica.
# redis-cli -h 192.168.30.130 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.130:6379> keys *
1) "key2"
2) "key1"
192.168.30.130:6379> CONFIG GET dir
1) "dir"
2) "/data/redis"
192.168.30.130:6379> CONFIG GET dbfilename
1) "dbfilename"
2) "dump.rdb"
192.168.30.130:6379> get key1
"100"
192.168.30.130:6379> get key2
"lzx"
192.168.30.130:6379> set key3 aaa
(error) READONLY You can't write against a read only replica.
可以看到,在master節(jié)點(diǎn)寫入的數(shù)據(jù),很快就同步到slave節(jié)點(diǎn)上,而且在slave節(jié)點(diǎn)上無法寫入數(shù)據(jù)。
Sentinel模式
Sentinel模式介紹
主從模式的弊端就是不具備高可用性,當(dāng)master掛掉以后,Redis將不能再對外提供寫入操作,因此sentinel應(yīng)運(yùn)而生。
sentinel中文含義為哨兵,顧名思義,它的作用就是監(jiān)控redis集群的運(yùn)行狀況,特點(diǎn)如下:
* sentinel模式是建立在主從模式的基礎(chǔ)上,如果只有一個(gè)Redis節(jié)點(diǎn),sentinel就沒有任何意義
* 當(dāng)master掛了以后,sentinel會(huì)在slave中選擇一個(gè)做為master,并修改它們的配置文件,其他slave的配置文件也會(huì)被修改,比如slaveof屬性會(huì)指向新的master
* 當(dāng)master重新啟動(dòng)后,它將不再是master而是做為slave接收新的master的同步數(shù)據(jù)
* sentinel因?yàn)橐彩且粋€(gè)進(jìn)程有掛掉的可能,所以sentinel也會(huì)啟動(dòng)多個(gè)形成一個(gè)sentinel集群
* 多sentinel配置的時(shí)候,sentinel之間也會(huì)自動(dòng)監(jiān)控
* 當(dāng)主從模式配置密碼時(shí),sentinel也會(huì)同步將配置信息修改到配置文件中,不需要擔(dān)心
* 一個(gè)sentinel或sentinel集群可以管理多個(gè)主從Redis,多個(gè)sentinel也可以監(jiān)控同一個(gè)redis
* sentinel最好不要和Redis部署在同一臺機(jī)器,不然Redis的服務(wù)器掛了以后,sentinel也掛了
工作機(jī)制:
* 每個(gè)sentinel以每秒鐘一次的頻率向它所知的master,slave以及其他sentinel實(shí)例發(fā)送一個(gè) PING 命令
* 如果一個(gè)實(shí)例距離最后一次有效回復(fù) PING 命令的時(shí)間超過 down-after-milliseconds 選項(xiàng)所指定的值, 則這個(gè)實(shí)例會(huì)被sentinel標(biāo)記為主觀下線。
* 如果一個(gè)master被標(biāo)記為主觀下線,則正在監(jiān)視這個(gè)master的所有sentinel要以每秒一次的頻率確認(rèn)master的確進(jìn)入了主觀下線狀態(tài)
* 當(dāng)有足夠數(shù)量的sentinel(大于等于配置文件指定的值)在指定的時(shí)間范圍內(nèi)確認(rèn)master的確進(jìn)入了主觀下線狀態(tài), 則master會(huì)被標(biāo)記為客觀下線
* 在一般情況下, 每個(gè)sentinel會(huì)以每 10 秒一次的頻率向它已知的所有master,slave發(fā)送 INFO 命令
* 當(dāng)master被sentinel標(biāo)記為客觀下線時(shí),sentinel向下線的master的所有slave發(fā)送 INFO 命令的頻率會(huì)從 10 秒一次改為 1 秒一次
* 若沒有足夠數(shù)量的sentinel同意master已經(jīng)下線,master的客觀下線狀態(tài)就會(huì)被移除;
若master重新向sentinel的 PING 命令返回有效回復(fù),master的主觀下線狀態(tài)就會(huì)被移除
當(dāng)使用sentinel模式的時(shí)候,客戶端就不要直接連接Redis,而是連接sentinel的ip和port,由sentinel來提供具體的可提供服務(wù)的Redis實(shí)現(xiàn),這樣當(dāng)master節(jié)點(diǎn)掛掉以后,sentinel就會(huì)感知并將新的master節(jié)點(diǎn)提供給使用者。
Sentinel模式搭建
環(huán)境準(zhǔn)備:
master節(jié)點(diǎn) 192.168.30.128 sentinel端口:26379
slave節(jié)點(diǎn) 192.168.30.129 sentinel端口:26379
slave節(jié)點(diǎn) 192.168.30.130 sentinel端口:26379
修改配置:
前面已經(jīng)下載安裝了redis,這里省略,直接修改sentinel配置文件。
192.168.30.128
# vim /usr/local/redis/sentinel.conf
daemonize yes
logfile "/usr/local/redis/sentinel.log"
dir "/usr/local/redis/sentinel" #sentinel工作目錄
sentinel monitor mymaster 192.168.30.128 6379 2 #判斷master失效至少需要2個(gè)sentinel同意,建議設(shè)置為n/2+1,n為sentinel個(gè)數(shù)
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 30000 #判斷master主觀下線時(shí)間,默認(rèn)30s
這里需要注意,sentinel auth-pass mymaster 123456需要配置在sentinel monitor mymaster 192.168.30.128 6379 2下面,否則啟動(dòng)報(bào)錯(cuò):
# /usr/local/bin/redis-sentinel /usr/local/redis/sentinel.conf
*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 104
>>> 'sentinel auth-pass mymaster 123456'
No such master with specified name.
全部啟動(dòng)sentinel:
# mkdir /usr/local/redis/sentinel chown -R redis:redis /usr/local/redis
# /usr/local/bin/redis-sentinel /usr/local/redis/sentinel.conf
任一主機(jī)查看日志:
# tail -f /usr/local/redis/sentinel.log
21574:X 09 May 2019 15:32:04.298 # Sentinel ID is 30c417116a8edbab09708037366c4a7471beb770
21574:X 09 May 2019 15:32:04.298 # +monitor master mymaster 192.168.30.128 6379 quorum 2
21574:X 09 May 2019 15:32:04.299 * +slave slave 192.168.30.129:6379 192.168.30.129 6379 @ mymaster 192.168.30.128 6379
21574:X 09 May 2019 15:32:04.300 * +slave slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.128 6379
21574:X 09 May 2019 15:32:16.347 * +sentinel sentinel 79b8d61626afd4d059fb5a6a63393e9a1374e78f 192.168.30.129 26379 @ mymaster 192.168.30.128 6379
21574:X 09 May 2019 15:32:31.584 * +sentinel sentinel d7b429dcba792103ef0d80827dd0910bd9284d21 192.168.30.130 26379 @ mymaster 192.168.30.128 6379
Sentinel模式下的幾個(gè)事件:
· +reset-master :主服務(wù)器已被重置。
· +slave :一個(gè)新的從服務(wù)器已經(jīng)被 Sentinel 識別并關(guān)聯(lián)。
· +failover-state-reconf-slaves :故障轉(zhuǎn)移狀態(tài)切換到了 reconf-slaves 狀態(tài)。
· +failover-detected :另一個(gè) Sentinel 開始了一次故障轉(zhuǎn)移操作,或者一個(gè)從服務(wù)器轉(zhuǎn)換成了主服務(wù)器。
· +slave-reconf-sent :領(lǐng)頭(leader)的 Sentinel 向?qū)嵗l(fā)送了 [SLAVEOF](/commands/slaveof.html) 命令,為實(shí)例設(shè)置新的主服務(wù)器。
· +slave-reconf-inprog :實(shí)例正在將自己設(shè)置為指定主服務(wù)器的從服務(wù)器,但相應(yīng)的同步過程仍未完成。
· +slave-reconf-done :從服務(wù)器已經(jīng)成功完成對新主服務(wù)器的同步。
· -dup-sentinel :對給定主服務(wù)器進(jìn)行監(jiān)視的一個(gè)或多個(gè) Sentinel 已經(jīng)因?yàn)橹貜?fù)出現(xiàn)而被移除 —— 當(dāng) Sentinel 實(shí)例重啟的時(shí)候,就會(huì)出現(xiàn)這種情況。
· +sentinel :一個(gè)監(jiān)視給定主服務(wù)器的新 Sentinel 已經(jīng)被識別并添加。
· +sdown :給定的實(shí)例現(xiàn)在處于主觀下線狀態(tài)。
· -sdown :給定的實(shí)例已經(jīng)不再處于主觀下線狀態(tài)。
· +odown :給定的實(shí)例現(xiàn)在處于客觀下線狀態(tài)。
· -odown :給定的實(shí)例已經(jīng)不再處于客觀下線狀態(tài)。
· +new-epoch :當(dāng)前的紀(jì)元(epoch)已經(jīng)被更新。
· +try-failover :一個(gè)新的故障遷移操作正在執(zhí)行中,等待被大多數(shù) Sentinel 選中(waiting to be elected by the majority)。
· +elected-leader :贏得指定紀(jì)元的選舉,可以進(jìn)行故障遷移操作了。
· +failover-state-select-slave :故障轉(zhuǎn)移操作現(xiàn)在處于 select-slave 狀態(tài) —— Sentinel 正在尋找可以升級為主服務(wù)器的從服務(wù)器。
· no-good-slave :Sentinel 操作未能找到適合進(jìn)行升級的從服務(wù)器。Sentinel 會(huì)在一段時(shí)間之后再次嘗試尋找合適的從服務(wù)器來進(jìn)行升級,又或者直接放棄執(zhí)行故障轉(zhuǎn)移操作。
· selected-slave :Sentinel 順利找到適合進(jìn)行升級的從服務(wù)器。
· failover-state-send-slaveof-noone :Sentinel 正在將指定的從服務(wù)器升級為主服務(wù)器,等待升級功能完成。
· failover-end-for-timeout :故障轉(zhuǎn)移因?yàn)槌瑫r(shí)而中止,不過最終所有從服務(wù)器都會(huì)開始復(fù)制新的主服務(wù)器(slaves will eventually be configured to replicate with the new master anyway)。
· failover-end :故障轉(zhuǎn)移操作順利完成。所有從服務(wù)器都開始復(fù)制新的主服務(wù)器了。
· +switch-master :配置變更,主服務(wù)器的 IP 和地址已經(jīng)改變。 這是絕大多數(shù)外部用戶都關(guān)心的信息。
· +tilt :進(jìn)入 tilt 模式。
· -tilt :退出 tilt 模式。
master宕機(jī)演示:
192.168.30.128
# systemctl stop redis
# tail -f /usr/local/redis/sentinel.log
22428:X 09 May 2019 15:51:29.287 # +sdown master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.371 # +odown master mymaster 192.168.30.128 6379 #quorum 2/2
22428:X 09 May 2019 15:51:29.371 # +new-epoch 1
22428:X 09 May 2019 15:51:29.371 # +try-failover master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.385 # +vote-for-leader 30c417116a8edbab09708037366c4a7471beb770 1
22428:X 09 May 2019 15:51:29.403 # d7b429dcba792103ef0d80827dd0910bd9284d21 voted for 30c417116a8edbab09708037366c4a7471beb770 1
22428:X 09 May 2019 15:51:29.408 # 79b8d61626afd4d059fb5a6a63393e9a1374e78f voted for 30c417116a8edbab09708037366c4a7471beb770 1
22428:X 09 May 2019 15:51:29.451 # +elected-leader master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.451 # +failover-state-select-slave master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.528 # +selected-slave slave 192.168.30.129:6379 192.168.30.129 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.528 * +failover-state-send-slaveof-noone slave 192.168.30.129:6379 192.168.30.129 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:29.594 * +failover-state-wait-promotion slave 192.168.30.129:6379 192.168.30.129 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:30.190 # +promoted-slave slave 192.168.30.129:6379 192.168.30.129 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:30.190 # +failover-state-reconf-slaves master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:30.258 * +slave-reconf-sent slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:30.511 # -odown master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:31.233 * +slave-reconf-inprog slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:31.233 * +slave-reconf-done slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:31.297 # +failover-end master mymaster 192.168.30.128 6379
22428:X 09 May 2019 15:51:31.297 # +switch-master mymaster 192.168.30.128 6379 192.168.30.129 6379
22428:X 09 May 2019 15:51:31.298 * +slave slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.129 6379
22428:X 09 May 2019 15:51:31.298 * +slave slave 192.168.30.128:6379 192.168.30.128 6379 @ mymaster 192.168.30.129 6379
22428:X 09 May 2019 15:52:31.307 # +sdown slave 192.168.30.128:6379 192.168.30.128 6379 @ mymaster 192.168.30.129 6379
從日志中可以看到,master已經(jīng)從192.168.30.128轉(zhuǎn)移到192.168.30.129上
192.168.30.129上查看集群信息
# /usr/local/bin/redis-cli -h 192.168.30.129 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.129:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.30.130,port=6379,state=online,offset=291039,lag=1
master_replid:757aff269236ed2707ba584a86a40716c1c76d74
master_replid2:47a862fc0ff20362be29096ecdcca6d432070ee9
master_repl_offset:291182
second_repl_offset:248123
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:291182
192.168.30.129:6379> set key4 linux
OK
當(dāng)前集群中只有一個(gè)slave——192.168.30.130,master是192.168.30.129,且192.168.30.129具有寫權(quán)限。
192.168.30.130上查看redis的配置文件也可以看到replicaof 192.168.30.129 6379,這是sentinel在選舉master是做的修改。
重新把192.168.30.128上進(jìn)程啟動(dòng)
# systemctl start redis
# tail -f /usr/local/redis/sentinel.log
22428:X 09 May 2019 15:51:31.297 # +switch-master mymaster 192.168.30.128 6379 192.168.30.129 6379
22428:X 09 May 2019 15:51:31.298 * +slave slave 192.168.30.130:6379 192.168.30.130 6379 @ mymaster 192.168.30.129 6379
22428:X 09 May 2019 15:51:31.298 * +slave slave 192.168.30.128:6379 192.168.30.128 6379 @ mymaster 192.168.30.129 6379
22428:X 09 May 2019 15:52:31.307 # +sdown slave 192.168.30.128:6379 192.168.30.128 6379 @ mymaster 192.168.30.129 6379
22428:X 09 May 2019 16:01:24.872 # -sdown slave 192.168.30.128:6379 192.168.30.128 6379 @ mymaster 192.168.30.129 6379
查看集群信息
# /usr/local/bin/redis-cli -h 192.168.30.128 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.128:6379> info replication
# Replication
role:slave
master_host:192.168.30.129
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:514774
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:757aff269236ed2707ba584a86a40716c1c76d74
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:514774
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:376528
repl_backlog_histlen:138247
192.168.30.128:6379> get key4
"linux"
192.168.30.128:6379> set key5
(error) ERR wrong number of arguments for 'set' command
即使192.168.30.128重新啟動(dòng)redis服務(wù),也是作為slave加入redis集群,192.168.30.129仍然是master。
Cluster模式
Cluster模式介紹
sentinel模式基本可以滿足一般生產(chǎn)的需求,具備高可用性。但是當(dāng)數(shù)據(jù)量過大到一臺服務(wù)器存放不下的情況時(shí),主從模式或sentinel模式就不能滿足需求了,這個(gè)時(shí)候需要對存儲的數(shù)據(jù)進(jìn)行分片,將數(shù)據(jù)存儲到多個(gè)Redis實(shí)例中。cluster模式的出現(xiàn)就是為了解決單機(jī)Redis容量有限的問題,將Redis的數(shù)據(jù)根據(jù)一定的規(guī)則分配到多臺機(jī)器。
cluster可以說是sentinel和主從模式的結(jié)合體,通過cluster可以實(shí)現(xiàn)主從和master重選功能,所以如果配置兩個(gè)副本三個(gè)分片的話,就需要六個(gè)Redis實(shí)例。因?yàn)镽edis的數(shù)據(jù)是根據(jù)一定規(guī)則分配到cluster的不同機(jī)器的,當(dāng)數(shù)據(jù)量過大時(shí),可以新增機(jī)器進(jìn)行擴(kuò)容。
使用集群,只需要將redis配置文件中的cluster-enable配置打開即可。每個(gè)集群中至少需要三個(gè)主數(shù)據(jù)庫才能正常運(yùn)行,新增節(jié)點(diǎn)非常方便。
cluster集群特點(diǎn):
* 多個(gè)redis節(jié)點(diǎn)網(wǎng)絡(luò)互聯(lián),數(shù)據(jù)共享
* 所有的節(jié)點(diǎn)都是一主一從(也可以是一主多從),其中從不提供服務(wù),僅作為備用
* 不支持同時(shí)處理多個(gè)key(如MSET/MGET),因?yàn)閞edis需要把key均勻分布在各個(gè)節(jié)點(diǎn)上,
并發(fā)量很高的情況下同時(shí)創(chuàng)建key-value會(huì)降低性能并導(dǎo)致不可預(yù)測的行為
* 支持在線增加、刪除節(jié)點(diǎn)
* 客戶端可以連接任何一個(gè)主節(jié)點(diǎn)進(jìn)行讀寫
Cluster模式搭建
環(huán)境準(zhǔn)備:
三臺機(jī)器,分別開啟兩個(gè)redis服務(wù)(端口)
192.168.30.128 端口:7001,7002
192.168.30.129 端口:7003,7004
192.168.30.130 端口:7005,7006
修改配置文件:
192.168.30.128
# mkdir /usr/local/redis/cluster
# cp /usr/local/redis/redis.conf /usr/local/redis/cluster/redis_7001.conf
# cp /usr/local/redis/redis.conf /usr/local/redis/cluster/redis_7002.conf
# chown -R redis:redis /usr/local/redis
# mkdir -p /data/redis/cluster/{redis_7001,redis_7002} chown -R redis:redis /data/redis
# vim /usr/local/redis/cluster/redis_7001.conf
bind 192.168.30.128
port 7001
daemonize yes
pidfile "/var/run/redis_7001.pid"
logfile "/usr/local/redis/cluster/redis_7001.log"
dir "/data/redis/cluster/redis_7001"
#replicaof 192.168.30.129 6379
masterauth 123456
requirepass 123456
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7001.conf
cluster-node-timeout 15000
# vim /usr/local/redis/cluster/redis_7002.conf
bind 192.168.30.128
port 7002
daemonize yes
pidfile "/var/run/redis_7002.pid"
logfile "/usr/local/redis/cluster/redis_7002.log"
dir "/data/redis/cluster/redis_7002"
#replicaof 192.168.30.129 6379
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7002.conf
cluster-node-timeout 15000
其它兩臺機(jī)器配置與192.168.30.128一致,此處省略
啟動(dòng)redis服務(wù):
# redis-server /usr/local/redis/cluster/redis_7001.conf
# tail -f /usr/local/redis/cluster/redis_7001.log
# redis-server /usr/local/redis/cluster/redis_7002.conf
# tail -f /usr/local/redis/cluster/redis_7002.log
其它兩臺機(jī)器啟動(dòng)與192.168.30.128一致,此處省略
安裝ruby并創(chuàng)建集群(低版本):
如果redis版本比較低,則需要安裝ruby。任選一臺機(jī)器安裝ruby即可
# yum -y groupinstall "Development Tools"
# yum install -y gdbm-devel libdb4-devel libffi-devel libyaml libyaml-devel ncurses-devel openssl-devel readline-devel tcl-devel
# mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
# wget http://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.3.tar.gz -P ~/rpmbuild/SOURCES
# wget http://raw.githubusercontent.com/tjinjin/automate-ruby-rpm/master/ruby22x.spec -P ~/rpmbuild/SPECS
# rpmbuild -bb ~/rpmbuild/SPECS/ruby22x.spec
# rpm -ivh ~/rpmbuild/RPMS/x86_64/ruby-2.2.3-1.el7.x86_64.rpm
# gem install redis #目的是安裝這個(gè),用于配置集群
# cp /usr/local/redis/src/redis-trib.rb /usr/bin/
# redis-trib.rb create --replicas 1 192.168.30.128:7001 192.168.30.128:7002 192.168.30.129:7003 192.168.30.129:7004 192.168.30.130:7005 192.168.30.130:7006
創(chuàng)建集群:
我這里是redis6.0.5,所以不需要安裝ruby,直接創(chuàng)建集群即可
# redis-cli -a 123456 --cluster create 192.168.30.128:7001 192.168.30.128:7002 192.168.30.129:7003 192.168.30.129:7004 192.168.30.130:7005 192.168.30.130:7006 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.30.129:7004 to 192.168.30.128:7001
Adding replica 192.168.30.130:7006 to 192.168.30.129:7003
Adding replica 192.168.30.128:7002 to 192.168.30.130:7005
M: 80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001
slots:[0-5460] (5461 slots) master
S: b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002
replicates 6788453ee9a8d7f72b1d45a9093838efd0e501f1
M: 4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003
slots:[5461-10922] (5462 slots) master
S: b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004
replicates 80c80a3f3e33872c047a8328ad579b9bea001ad8
M: 6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005
slots:[10923-16383] (5461 slots) master
S: 277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006
replicates 4d74ec66e898bf09006dac86d4928f9fad81f373
Can I set the above configuration? (type 'yes' to accept): yes #輸入yes,接受上面配置
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
可以看到,
192.168.30.128:7001是master,它的slave是192.168.30.129:7004;
192.168.30.129:7003是master,它的slave是192.168.30.130:7006;
192.168.30.130:7005是master,它的slave是192.168.30.128:7002
自動(dòng)生成nodes.conf文件:
# ls /data/redis/cluster/redis_7001/
appendonly.aof dump.rdb nodes-7001.conf
# vim /data/redis/cluster/redis_7001/nodes-7001.conf
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557454406312 5 connected 10923-16383
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557454407000 6 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557454408371 5 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 myself,master - 0 1557454406000 1 connected 0-5460
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557454407366 4 connected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557454407000 3 connected 5461-10922
vars currentEpoch 6 lastVoteEpoch 0
集群操作
登錄集群:
# redis-cli -c -h 192.168.30.128 -p 7001 -a 123456 # -c,使用集群方式登錄
查看集群信息:
192.168.30.128:7001> CLUSTER INFO #集群狀態(tài)
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:580
cluster_stats_messages_pong_sent:551
cluster_stats_messages_sent:1131
cluster_stats_messages_ping_received:546
cluster_stats_messages_pong_received:580
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1131
列出節(jié)點(diǎn)信息:
192.168.30.128:7001> CLUSTER NODES #列出節(jié)點(diǎn)信息
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557455176000 5 connected 10923-16383
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557455174000 6 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557455175000 5 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 myself,master - 0 1557455175000 1 connected 0-5460
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557455174989 4 connected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557455175995 3 connected 5461-10922
這里與nodes.conf文件內(nèi)容相同
寫入數(shù)據(jù):
192.168.30.128:7001> set key111 aaa
-> Redirected to slot [13680] located at 192.168.30.130:7005 #說明數(shù)據(jù)到了192.168.30.130:7005上
OK
192.168.30.130:7005> set key222 bbb
-> Redirected to slot [2320] located at 192.168.30.128:7001 #說明數(shù)據(jù)到了192.168.30.128:7001上
OK
192.168.30.128:7001> set key333 ccc
-> Redirected to slot [7472] located at 192.168.30.129:7003 #說明數(shù)據(jù)到了192.168.30.129:7003上
OK
192.168.30.129:7003> get key111
-> Redirected to slot [13680] located at 192.168.30.130:7005
"aaa"
192.168.30.130:7005> get key333
-> Redirected to slot [7472] located at 192.168.30.129:7003
"ccc"
192.168.30.129:7003>
可以看出redis cluster集群是去中心化的,每個(gè)節(jié)點(diǎn)都是平等的,連接哪個(gè)節(jié)點(diǎn)都可以獲取和設(shè)置數(shù)據(jù)。
當(dāng)然,平等指的是master節(jié)點(diǎn),因?yàn)閟lave節(jié)點(diǎn)根本不提供服務(wù),只是作為對應(yīng)master節(jié)點(diǎn)的一個(gè)備份。
增加節(jié)點(diǎn):
192.168.30.129上增加一節(jié)點(diǎn):
# cp /usr/local/redis/cluster/redis_7003.conf /usr/local/redis/cluster/redis_7007.conf
# vim /usr/local/redis/cluster/redis_7007.conf
bind 192.168.30.129
port 7007
daemonize yes
pidfile "/var/run/redis_7007.pid"
logfile "/usr/local/redis/cluster/redis_7007.log"
dir "/data/redis/cluster/redis_7007"
#replicaof 192.168.30.129 6379
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7007.conf
cluster-node-timeout 15000
# mkdir /data/redis/cluster/redis_7007
# chown -R redis:redis /usr/local/redis chown -R redis:redis /data/redis
# redis-server /usr/local/redis/cluster/redis_7007.conf
192.168.30.130上增加一節(jié)點(diǎn):
# cp /usr/local/redis/cluster/redis_7005.conf /usr/local/redis/cluster/redis_7008.conf
# vim /usr/local/redis/cluster/redis_7007.conf
bind 192.168.30.130
port 7008
daemonize yes
pidfile "/var/run/redis_7008.pid"
logfile "/usr/local/redis/cluster/redis_7008.log"
dir "/data/redis/cluster/redis_7008"
#replicaof 192.168.30.130 6379
masterauth "123456"
requirepass "123456"
appendonly yes
cluster-enabled yes
cluster-config-file nodes_7008.conf
cluster-node-timeout 15000
# mkdir /data/redis/cluster/redis_7008
# chown -R redis:redis /usr/local/redis chown -R redis:redis /data/redis
# redis-server /usr/local/redis/cluster/redis_7008.conf
集群中增加節(jié)點(diǎn):
192.168.30.129:7003> CLUSTER MEET 192.168.30.129 7007
OK
192.168.30.129:7003> CLUSTER NODES
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 myself,master - 0 1557457361000 3 connected 5461-10922
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557457364746 1 connected 0-5460
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557457362000 6 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557457363000 4 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557457362000 5 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557457362729 0 connected
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557457363739 5 connected 10923-16383
192.168.30.129:7003> CLUSTER MEET 192.168.30.130 7008
OK
192.168.30.129:7003> CLUSTER NODES
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 myself,master - 0 1557457489000 3 connected 5461-10922
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557457489000 1 connected 0-5460
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557457489000 6 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557457488000 4 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557457489472 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 master - 0 1557457489259 0 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557457489000 0 connected
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557457490475 5 connected 10923-16383
可以看到,新增的節(jié)點(diǎn)都是以master身份加入集群的
更換節(jié)點(diǎn)身份:
將新增的192.168.30.130:7008節(jié)點(diǎn)身份改為192.168.30.129:7007的slave
# redis-cli -c -h 192.168.30.130 -p 7008 -a 123456 cluster replicate e51ab166bc0f33026887bcf8eba0dff3d5b0bf14
cluster replicate后面跟node_id,更改對應(yīng)節(jié)點(diǎn)身份。也可以登入集群更改
# redis-cli -c -h 192.168.30.130 -p 7008 -a 123456
192.168.30.130:7008> CLUSTER REPLICATE e51ab166bc0f33026887bcf8eba0dff3d5b0bf14
OK
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557458316881 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557458314864 1 connected 0-5460
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557458316000 3 connected 5461-10922
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557458315872 5 connected 10923-16383
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557458317890 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557458315000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557458315000 1 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557458314000 0 connected
查看相應(yīng)的nodes.conf文件,可以發(fā)現(xiàn)有更改,它記錄當(dāng)前集群的節(jié)點(diǎn)信息
# cat /data/redis/cluster/redis_7001/nodes-7001.conf
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557458236169 7 connected
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557458235000 5 connected 10923-16383
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557458234103 6 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557458235129 5 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 myself,master - 0 1557458234000 1 connected 0-5460
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557458236000 4 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557458236000 0 connected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557458233089 3 connected 5461-10922
vars currentEpoch 7 lastVoteEpoch 0
刪除節(jié)點(diǎn):
192.168.30.130:7008> CLUSTER FORGET 1a1c7f02fce87530bd5abdfc98df1cffce4f1767
(error) ERR I tried hard but I can't forget myself... #無法刪除登錄節(jié)點(diǎn)
192.168.30.130:7008> CLUSTER FORGET e51ab166bc0f33026887bcf8eba0dff3d5b0bf14
(error) ERR Can't forget my master! #不能刪除自己的master節(jié)點(diǎn)
192.168.30.130:7008> CLUSTER FORGET 6788453ee9a8d7f72b1d45a9093838efd0e501f1
OK #可以刪除其它的master節(jié)點(diǎn)
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557458887328 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557458887000 1 connected 0-5460
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557458886000 3 connected 5461-10922
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave - 0 1557458888351 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557458885000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557458883289 1 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557458885310 0 connected
192.168.30.130:7008> CLUSTER FORGET b4d3eb411a7355d4767c6c23b4df69fa183ef8bc
OK #可以刪除其它的slave節(jié)點(diǎn)
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557459031397 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557459032407 1 connected 0-5460
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557459035434 3 connected 5461-10922
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557459034000 5 connected 10923-16383
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557459032000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557459034000 1 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557459034427 0 connected
保存配置:
192.168.30.130:7008> CLUSTER SAVECONFIG #將節(jié)點(diǎn)配置信息保存到硬盤
OK
# cat /data/redis/cluster/redis_7001/nodes-7001.conf
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557458236169 7 connected
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557458235000 5 connected 10923-16383
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557458234103 6 connected
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557458235129 5 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 myself,master - 0 1557458234000 1 connected 0-5460
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557458236000 4 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557458236000 0 connected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557458233089 3 connected 5461-10922
vars currentEpoch 7 lastVoteEpoch 0
# redis-cli -c -h 192.168.30.130 -p 7008 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557459500741 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master - 0 1557459500000 1 connected 0-5460
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557459501000 3 connected 5461-10922
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557459500000 5 connected 10923-16383
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557459499737 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557459499000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 slave 80c80a3f3e33872c047a8328ad579b9bea001ad8 0 1557459501750 1 connected
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557459498000 0 connected
可以看到,之前刪除的節(jié)點(diǎn)又恢復(fù)了,這是因?yàn)閷?yīng)的配置文件沒有刪除,執(zhí)行CLUSTER SAVECONFIG恢復(fù)。
模擬master節(jié)點(diǎn)掛掉:
192.168.30.128
# netstat -lntp |grep 7001
tcp 0 0 192.168.30.128:17001 0.0.0.0:* LISTEN 6701/redis-server 1
tcp 0 0 192.168.30.128:7001 0.0.0.0:* LISTEN 6701/redis-server 1
# kill 6701
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557461178000 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 master,fail - 1557460950483 1557460947145 1 disconnected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557461174922 3 connected 5461-10922
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557461181003 5 connected 10923-16383
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557461179993 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557461176000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 master - 0 1557461178981 8 connected 0-5460
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557461179000 0 connected
對應(yīng)7001的一行可以看到,master fail,狀態(tài)為disconnected;而對應(yīng)7004的一行,slave已經(jīng)變成master。
重新啟動(dòng)7001節(jié)點(diǎn):
# redis-server /usr/local/redis/cluster/redis_7001.conf
192.168.30.130:7008> CLUSTER NODES
277daeb8660d5273b7c3e05c263f861ed5f17b92 192.168.30.130:7006@17006 slave 4d74ec66e898bf09006dac86d4928f9fad81f373 0 1557461307000 3 connected
80c80a3f3e33872c047a8328ad579b9bea001ad8 192.168.30.128:7001@17001 slave b6331cbc986794237c83ed2d5c30777c1551546e 0 1557461305441 8 connected
4d74ec66e898bf09006dac86d4928f9fad81f373 192.168.30.129:7003@17003 master - 0 1557461307962 3 connected 5461-10922
6788453ee9a8d7f72b1d45a9093838efd0e501f1 192.168.30.130:7005@17005 master - 0 1557461304935 5 connected 10923-16383
b4d3eb411a7355d4767c6c23b4df69fa183ef8bc 192.168.30.128:7002@17002 slave 6788453ee9a8d7f72b1d45a9093838efd0e501f1 0 1557461306000 5 connected
1a1c7f02fce87530bd5abdfc98df1cffce4f1767 192.168.30.130:7008@17008 myself,slave e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 0 1557461305000 7 connected
b6331cbc986794237c83ed2d5c30777c1551546e 192.168.30.129:7004@17004 master - 0 1557461308972 8 connected 0-5460
e51ab166bc0f33026887bcf8eba0dff3d5b0bf14 192.168.30.129:7007@17007 master - 0 1557461307000 0 connected
可以看到,7001節(jié)點(diǎn)啟動(dòng)后為slave節(jié)點(diǎn),并且是7004的slave節(jié)點(diǎn)。即master節(jié)點(diǎn)如果掛掉,它的slave節(jié)點(diǎn)變?yōu)樾耺aster節(jié)點(diǎn)繼續(xù)對外提供服務(wù),而原來的master節(jié)點(diǎn)如果重啟,則變?yōu)樾耺aster節(jié)點(diǎn)的slave節(jié)點(diǎn)。
另外,如果這里是拿7007節(jié)點(diǎn)做測試的話,會(huì)發(fā)現(xiàn)7008節(jié)點(diǎn)并不會(huì)切換,這是因?yàn)?007節(jié)點(diǎn)上根本沒數(shù)據(jù)。集群數(shù)據(jù)被分為三份,采用哈希槽 (hash slot)的方式來分配16384個(gè)slot的話,它們?nèi)齻€(gè)節(jié)點(diǎn)分別承擔(dān)的slot 區(qū)間是:
節(jié)點(diǎn)7004覆蓋0-5460
節(jié)點(diǎn)7003覆蓋5461-10922
節(jié)點(diǎn)7005覆蓋10923-16383
更多參考:
Redis集群常用命令