2024-07-12
한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina
Database level transactions
At the database level, a transaction is a set of operations that either all execute successfully or none execute at all.
Four characteristics of database transactions
Redis Transactions
A Redis transaction is a collection of commands. All commands in a transaction will be serialized, and a series of commands will be executed one-time, sequentially, and exclusively.
MULTIStart a transaction;EXECCommand triggers a transaction;
Multi、Exec、discard
Transaction from inputMultiAt the beginning of the command, the input commands will be pushed into the command buffer queue in sequence and will not be executed until the inputExecAfter that, Redis will execute the commands in the previous command buffer queue in sequence.discardTo give up teaming.
example
127.0.0.1:6379> set t1 1
OK
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379(TX)> set id 12
QUEUED
127.0.0.1:6379(TX)> get id
QUEUED
127.0.0.1:6379(TX)> incr t1
QUEUED
127.0.0.1:6379(TX)> incr t1
QUEUED
127.0.0.1:6379(TX)> get t1
QUEUED
127.0.0.1:6379(TX)> EXEC
1) OK
2) "12"
3) (integer) 2
4) (integer) 3
5) "3"
Abandoning a transaction
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379(TX)> set name z3
QUEUED
127.0.0.1:6379(TX)> set age 29
QUEUED
127.0.0.1:6379(TX)> incr t1
QUEUED
127.0.0.1:6379(TX)> DISCARD
OK
All sit together
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379(TX)> set name z3
QUEUED
127.0.0.1:6379(TX)> get name
QUEUED
127.0.0.1:6379(TX)> incr t1
QUEUED
127.0.0.1:6379(TX)> get t1
QUEUED
127.0.0.1:6379(TX)> set email
(error) ERR wrong number of arguments for 'set' command
127.0.0.1:6379(TX)> exec
(error) EXECABORT Transaction discarded because of previous errors.
Notice：
If there are incorrect instructions in the command set (note that they are syntax errors), all of them will fail.
Every wrong has its perpetrator, every debt has its creditor
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379(TX)> set age 11
QUEUED
127.0.0.1:6379(TX)> incr t1
QUEUED
127.0.0.1:6379(TX)> set email [email protected]
QUEUED
127.0.0.1:6379(TX)> incr email
QUEUED
127.0.0.1:6379(TX)> get age
QUEUED
127.0.0.1:6379(TX)> exec
1) OK
2) (integer) 5
3) OK
4) (error) ERR value is not an integer or out of range
5) "11"
Notice：
Runtime errors, that is, non-syntax errors, correct commands will be executed, and incorrect commands will return errors.
Overview
Among existing enterprises, 80% of companies use Redis stand-alone services. In actual scenarios, single-node Redis is prone to risks.
Problems:
- Machine failure. We deploy to a Redis server. When a machine failure occurs, we need to migrate to another server and ensure that the data is synchronized.
- Capacity bottleneck. When we need to expand the Redis memory from 16G to 64G, a single machine will definitely not be able to meet the needs. Of course, you can buy a new machine with 128G.
Solution
To achieve larger storage capacity of the distributed database and withstand high concurrent access, we will store the data of the original centralized database on multiple other network nodes.
Notice：
In order to solve this single node problem, Redis will also replicate multiple copies of the data and deploy them on other nodes for replication, achieving high availability of Redis and redundant backup of data to ensure high availability of data and services.
What is Master-Slave Replication
Master-slave replication refers to copying the data of a Redis server to other Redis servers. The former is called the master node, and the latter is called the slave node. Data replication is one-way, only from the master node to the slave node.
The role of master-slave replication
Writing configuration files
Create redis6379.conf
include /usr/local/redis-7.2.4/redis.config
pidfile /var/run/redis_6379.pid
port 6379
dbfilename dump6379.rdb
Create redis6380.conf
include /usr/local/redis-7.2.4/redis.config
pidfile /var/run/redis_6380.pid
port 6380
dbfilename dump6380.rdb
Create redis6381.conf
include /usr/local/redis-7.2.4/redis.config
pidfile /var/run/redis_6381.pid
port 6381
dbfilename dump6381.rdb
Start three redis servers
./redis-server ../redis6379.conf
./redis-server ../redis6380.conf
./redis-server ../redis6381.conf
View system processes
[root@localhost src]# ps -ef |grep redis
root 40737 1 0 22:05 ? 00:00:00 ./redis-server *:6379
root 40743 1 0 22:05 ? 00:00:00 ./redis-server *:6380
root 40750 1 0 22:05 ? 00:00:00 ./redis-server *:6381
root 40758 40631 0 22:05 pts/0 00:00:00 grep --color=auto redis
Check the operation status of the three hosts
#打印主从复制的相关信息
./redis-cli -p 6379
./redis-cli -p 6380
./redis-cli -p 6381
127.0.0.1:6379> info replication
127.0.0.1:6380> info replication
127.0.0.1:6381> info replication
The slave database is not equipped with the master database
Syntax format:
slaveof <ip> <port>
Example:
Executed on 6380 and 6381.
127.0.0.1:6380> SLAVEOF 127.0.0.1 6379
OK
Write on the host and read data on the slave
set k1 v1
Master-slave replication can be divided into 3 stages
The replication process is roughly divided into 6 steps
info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
4. Send ping command
After the connection is successfully established, the slave node sends a ping request for the first communication.
effect:
- Check whether the network socket between the master and the slave is available.
- Check whether the master node can currently accept commands
ASD.
If the requirepass parameter is set for the master node, password verification is required. The slave node must configure the masterauth parameter to ensure that the password is the same as that of the master node in order to pass the verification. If the verification fails, the replication will be terminated and the slave node will restart the replication process.
Synchronize the dataset.
After the master-slave replication connection is communicating normally, for the scenario of establishing replication for the first time, the master node will send all the data it holds to the slave node. This part of the operation is the longest step.
Master-slave synchronization strategy
When the master and slave are just connected, full synchronization is performed; after full synchronization, incremental synchronization is performed. Of course, if necessary, the slave can initiate full synchronization at any time. The redis strategy is that no matter what, incremental synchronization will be tried first. If it fails, the slave will be required to perform full synchronization.
For example
Save a cache
set name jjy
The recorded command is
$3 r n
set r n
$4 r n
name r n
$5 r n
jjy r n
| Offset | 1000 | 1001 | 1002 | 1003 | 1004 | 1005 | 1006 | 1007 | 1008 |
|---|---|---|---|---|---|---|---|---|---|
| Byte value | $ | 3 | r | n | $ | 4 | n | a | m |
7. Command continuous replication.
When the master node synchronizes the current data to the slave node, the replication establishment process is completed. Next, the master node will continue to send write commands to the slave node to ensure the consistency of the master and slave data.
Disadvantages of Redis master-slave replication
When the host Master goes down, we need to manually solve the switch.
Leakage issues:
Once the master node goes down and the write service is unavailable, you need to manually switch, reselect the master node, and manually set the master-slave relationship.
Master-slave switching technology
When the master server goes down, you need to manually switch a slave server to the master server, which requires manual intervention, is time-consuming and labor-intensive, and will cause service unavailability for a period of time. This is not a recommended method. More often, we give priority toSentry Mode。
Sentinel Overview
Sentinel mode is a special mode. First, Redis provides the sentinel command. Sentinel is an independent process. As a process, it will run independently. The principle is that Sentinel monitors multiple running Redis instances by sending commands and waiting for the Redis server to respond.
Sentinel role
Create a new sentinel-26379.conf file
#端口
port 26379
#守护进程运行
daemonize yes
#日志文件
logfile "26379.log"
sentinel monitor mymaster localhost 6379 2
parameter:
sentinel monitor mymaster 192.168.92.128 6379 2 The configuration means that the sentinel node monitors the master node 192.168.92.128:6379, which is named mymaster. The last 2 is related to the failure determination of the master node: at least two sentinel nodes must agree to determine that the master node has failed and perform a failover.
Create a new sentinel-26380.conf file
#端口
port 26380
#守护进程运行
daemonize yes
#日志文件
logfile "26380.log"
sentinel monitor mymaster localhost 6379 2
Create a new sentinel-26381.conf file
#端口
port 26381
#守护进程运行
daemonize yes
#日志文件
logfile "26381.log"
sentinel monitor mymaster localhost 6379 2
Sentinel node startup
redis-sentinel sentinel-26379.conf
View the sentinel node status
[root@localhost src]# ./redis-cli -p 26379
127.0.0.1:26379>
127.0.0.1:26379>
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.66.100:6379,slaves=2,sentinels=3
Monitoring Phase
Notice:
- Sentinel 1 sends an info message to the master and slave to get the full information.
- Sentinel (Sentinel 2) -----> Initiate info to the master (master), then you will know the information of the existing sentinel (Sentinel 1) and connect to the slave (slave).
- Sentinel (Sentinel 2)----->Initiate a subscribe to sentinel (Sentinel 1).
Notification Phase
Sentinel continuously sends notifications to the master and slave to collect information.
Failover phase
In the notification phase, if the notification sent by sentinel does not receive a response from the master, the master will be marked as SRI_S_DOWN, and the master status will be sent to each sentinel. When other sentinels hear that the master has died, they will say, "I don't believe it, let me go and check it out too," and share the results with each sentinel. When half of the sentinels believe that the master has died, the master will be marked as SRI_0_DOWN.
Here comes the question:
At this time, the master needs to be replaced. Who will be the master?
Voting method
Way:
I will vote for the sentinel whose election notification I receive first.
Excluding some cases:
Overview
Demonstrates Sentinel's monitoring and automatic failover capabilities when the primary node fails.
Demonstrating Failover
Use the kill command to kill the master node
ps aux |grep redis
kill -9 pid
View sentinel node information
If you use the info Sentinel command in the sentinel node immediately, you can check it.
[root@localhost src]# ./redis-cli -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6381,slaves=5,sentinels=3
Notice：
You will find that the master node has not yet switched over, because it takes some time for the Sentinel to detect the failure of the master node and transfer.
Restart 6379 nodes
[root@localhost src]# ./redis-cli info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:down
Configuration files will be rewritten
During the failover phase, the configuration files of the sentinel and the master and slave nodes will be rewritten.
include /usr/local/redis/redis.conf
pidfile "/var/run/redis_6379.pid"
port 6379
dbfilename "dump6379.rdb"
# Generated by CONFIG REWRITE
daemonize yes
protected-mode no
appendonly yes
slowlog-max-len 1200
slowlog-log-slower-than 1000
save 5 1
user default on nopass ~* &* +@all
dir "/usr/local/redis"
replicaof 127.0.0.1 6381
in conclusion
Redis has three cluster modes
Disadvantages of Sentry Mode
shortcoming：
Cluster Mode Overview
Redis cluster is a distributed service cluster consisting of multiple master-slave node groups, which has replication, high availability and sharding features.
Advantages of Redis Cluster
A Redis cluster requires at least three master nodes. We build three masters here, each with a slave node, for a total of six Redis nodes.
Cluster construction
Create 6 different redis nodes with port numbers 6379, 6380, 6381, 6382, 6383, and 6384 respectively.
Notice: The dump.rdb and appendonly.aof files must be deleted before copying
1. Create a new configuration file
Create redis6379.config, redis6380.config, redis6381.config, redis6382.config, redis6383.config, and redis6384.config files and modify the port number in the configuration file to correspond to the file port number.
daemonize yes
dir /usr/local/redis-7.2.4/redis-cluster/6382/
bind 192.168.47.100
port 6382
dbfilename dump6382.rdb
cluster-enabled yes
cluster-config-file nodes-6382.conf
cluster-node-timeout 5000
appendonly yes
protected-mode no
parameter：
- cluster-config-file: Cluster persistent configuration file, which contains the status of other nodes, persistent variables, etc., and is automatically generated in the dir directory configured above. Each node will maintain a cluster configuration file during operation; whenever the cluster information changes (such as adding or removing nodes), all nodes in the cluster will update the latest information to the configuration file; when the node restarts, it will re-read the configuration file to obtain the cluster information, and can easily rejoin the cluster. The cluster configuration file is maintained by Redis and does not need to be modified manually.
- clouster-enabled: Enable clustering
Create a folder
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6379/
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6380/
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6381/
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6382/
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6383/
mkdir -p /usr/local/redis-7.2.4/redis-cluster/6384/
Start six nodes
[root@bogon src]# ./redis-server ../redis6379.config
[root@bogon src]# ./redis-server ../redis6380.config
[root@bogon src]# ./redis-server ../redis6381.config
[root@bogon src]# ./redis-server ../redis6382.config
[root@bogon src]# ./redis-server ../redis6383.config
[root@bogon src]# ./redis-server ../redis6384.config
Check whether each node is started successfully
[root@bogon src]# ps -ef | grep redis
root 3889 1 0 09:56 ? 00:00:03 ./redis-server 0.0.0.0:6379 [cluster]
root 3895 1 0 09:56 ? 00:00:03 ./redis-server 0.0.0.0:6380 [cluster]
root 3901 1 0 09:57 ? 00:00:03 ./redis-server 0.0.0.0:6381 [cluster]
root 3907 1 0 09:57 ? 00:00:02 ./redis-server *:6382 [cluster]
root 3913 1 0 09:57 ? 00:00:02 ./redis-server 0.0.0.0:6383 [cluster]
root 3919 1 0 09:57 ? 00:00:02 ./redis-server 0.0.0.0:6384 [cluster]
root 4247 2418 0 10:22 pts/0 00:00:00 grep --color=auto redis
Configuring the cluster
Command format: --cluster-replicas 1 means creating a slave node for each master
Notice：The IP here is the real IP of the machine where each node is located
[root@localhost src]# ./redis-cli --cluster create 192.168.47.100:6379 192.168.47.100:6380 192.168.47.100:6381 192.168.47.100:6382 192.168.47.100:6383 192.168.47.100:6384 --cluster-replicas 1
Verify the cluster
Connect to any client
./redis-cli -h 192.168.47.100 -p 6379 -c
parameter:
-h : host address
-p : port number
-c: indicates cluster mode
Data write test
[root@bogon src]# ./redis-cli -p 6379 -c
127.0.0.1:6379> set name zhangsan
-> Redirected to slot [5798] located at 192.168.47.100:6380
OK
192.168.47.100:6380> get name
"zhangsan"
192.168.47.100:6380>
[root@bogon src]# ./redis-cli -p 6383 -c
127.0.0.1:6383> get name
-> Redirected to slot [5798] located at 192.168.47.100:6380
"zhangsan"
192.168.47.100:6380>
[root@bogon src]# ./redis-cli -p 6383 -c
127.0.0.1:6383> readonly
OK
127.0.0.1:6383> get name
"zhangsan"
127.0.0.1:6383>
Redis ClusterAll data is divided into 16384 slots, and each node is responsible for a part of the slots. The slot information is stored in each node. Only the master node will be assigned slots, and the slave node will not be assigned slots.
Slot location algorithm： k1 = 127001
By default, the Cluster uses the crc16 algorithm to hash the key value to get an integer value, and then uses this integer value to perform a modulo operation on 16384 to get the specific slot.
HASH_SLOT = CRC16(key) % 16384
Recovery
View Node
192.168.66.103:8001> cluster nodes
Kill the Master Node
lsof -i:8001
kill -9 pid
Observe node information
Modify the configuration file
spring.data.redis.cluster.nodes=192.168.47.100:6381,192.168.47.100:6383,192.168.47.100:6380
Notice
1. A Redis cluster requires at least 3 nodes to ensure high availability.
2. You should try to avoid adding or deleting nodes during the operation of the Redis cluster, as this may cause data migration and affect the overall performance of the Redis cluster.
Code written in Java
@SpringBootTest
public class CluseterTest {
@Autowired
private RedisTemplate<String,Object> redisTemplate;
@Test
void string() {
// 保存字符串
redisTemplate.opsForValue().set("itbaizhan","itbaizhan123");
System.out.println(redisTemplate.opsForValue().get("itbaizhan"));
}
}
If my content is helpful to you, pleaseLike, comment, favoriteCreation is not easy, your support is my motivation to keep going.
I have devoted myself to the research of technology for more than 30 years. I am proficient in various languages such as Java, Linux, JavaScript, PHP, CSS, etc. I have made many contributions in the field of open source. I have established a developer documentation site to share some problems in technology development for everyone to read.
