Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

This article briefly introduces redis data synchronization based on redis-shake. This tool synchronizes data based on each node, that is, each master node needs to be synchronized once to complete the data synchronization of the entire redis cluster.

1. Redis node operation

### 查看redis版本
./bin/redis-server --version

### 登录redis
./bin/redis-cli -c -h *.*.*.142 -p 17001 -a '******'

### 查看数据量
> info keyspace

### 批量删除keys
./bin/redis-cli  -c -h *.*.*.142 -p 17001 -a '******' keys  "*@_@*"  | xargs -r -t -n1 ./bin/redis-cli -c -h *.*.*.142 -p 17001 -a '******' del

./bin/redis-cli  -c -h *.*.*.144 -p 17001 -a '******' keys  "auth-dw*"  | xargs -r -t -n1 ./bin/redis-cli -c -h *.*.*.144 -p 17001 -a '******' del

### 删除所有keys
> flushall

### 查看set类型变量的内容
smembers "auth-ds:446fc...@24cfd7b0..."
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

2. Redis-shake configuration and startup

Edit the configuration file vim ./sync.toml

type = "sync"

[source]
version = 7.0 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ...
address = "*.*.*.93:17001"
username = "" # keep empty if not using ACL
password = "******" # keep empty if no authentication is required
tls = false
elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373

[target]
type = "standalone" # "standalone" or "cluster"  
version = 7.0 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ...
 # When the target is a cluster, write the address of one of the nodes.
 # redis-shake will obtain other nodes through the `cluster nodes` command.

address = "*.*.*.142:17001"
username = "" # keep empty if not using ACL
password = "******" # keep empty if no authentication is required
tls = false

[advanced]
dir = "data"

ncpu = 4 # runtime.GOMAXPROCS, 0 means use runtime.NumCPU() cpu cores

pprof_port = 0 # pprof port, 0 means disable

metrics_port = 0 # metric port, 0 means disable

log_file = "redis-shake.log"
log_level = "info" # debug, info or warn
log_interval = 5 # in seconds

rdb_restore_command_behavior = "skip" # panic, rewrite or skip

pipeline_count_limit = 1024 # pipeline

target_redis_client_max_querybuf_len = 1024_000_000

target_redis_proto_max_bulk_len = 512_000_000
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

Start redis-shake to synchronize data

nohup ./redis-shake sync.toml 1>> redis-shake.log 2>> redis-shake.log &
1