Technology Sharing

03-NoSQL Redis configuration and optimization

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

1. Overall comparison between redis and memcache

1. Performance

Redis: Only a single core is used. On average, Redis has higher performance than Memcached when storing small data on each core.

Memcached: It can use multiple cores, and for data larger than 100k, Memcached has better performance than Redis.

2. Memory usage efficiency

MemCached: Using simple key-value storage, Memcached has higher memory utilization.

Redis: If a hash structure is used for key-value storage, its memory utilization rate will be higher than Memcached due to its combined compression.

3. Memory space and data size

MemCached: The maximum memory can be modified and the LRU algorithm is used. The size of a single key-value in Memcached is limited, and a value only supports a maximum of 1MB.

Redis: Added VM features to break through the limitations of physical memory. Redis supports a maximum of 512MB for a single key-value.

4. Data structure support

MemCached: The data structure is simple and is only used to cache data.

Redis: supports richer data types. Redis not only supports simple k/v type data, but also provides storage for list, set, zset, hash and other data structures.

Rich operations can be performed directly on the data on the server side, which can reduce network IO times and data volume.

5. Reliability

Memcached: It is just a memory cache with low reliability requirements. MemCached does not support data persistence, and the data disappears after power failure or restart, but stability is guaranteed.

Redis: It has high reliability requirements, supports data persistence and data recovery, allows single point failure, and will also affect some performance. It supports data backup, that is, data backup in master-slave mode. Redis supports data persistence, and can save data in memory to disk, which can be loaded again for use when restarting.

6. Application scenarios

Memcached: Reduces database load and improves performance in dynamic systems; it is used for caching and is suitable for scenarios with more reads and less writes and large data volumes.

Redis: Suitable for systems with high requirements for read and write efficiency, complex data processing operations, and high security requirements.

2. Experimental Case

1. Installation of redis

Turn off the firewall first

  1. 先把防火墙关掉
  2. systemctl stop firewalld
  3. set setenforce 0
  4. [root@localhost ~]# yum -y install gcc* zlib-devel
  5. 解压 redis包
  6. [root@localhost ~]# tar xvzf redis-4.0.9.tar.gz
  7. [root@localhost ~]# cd redis-4.0.9/
  8.  make

  1. 2:查看进程
  2. [root@localhost utils]# netstat -anpt | grep redis

  1. 3:redis服务控制
  2. [root@localhost ~]#/etc/init.d/redis_6379 stop
  3. [root@localhost ~]#/etc/init.d/redis_6379 start
  4. [root@localhost ~]#/etc/init.d/redis_6379 restart
  5. [root@localhost ~]#/etc/init.d/redis_6379 status

  1. 4.配置参数的修改
  2. [root@localhost ~]#vim /etc/redis/6379.conf
  3. bind 127.0.0.1 192.168.10.101 //监听的主机地址
  4. port 6379 //端口
  5. daemonize yes //启用守护进程
  6. pidfile /var/run/redis_6379.pid //指定 PID 文件
  7. loglevel notice //日志级别
  8. logfile /var/log/redis_6379.log //指定日志文件
  9. [root@localhost~]#/etc/init.d/redis_6379 restart
  10. [root@localhost utils]# netstat -anpt | grep redis

  1. 二:Redis 命令工具
  2.  redis-server:用于启动 Redis 的工具;
  3.  redis-benchmark:用于检测 Redis 在本机的运行效率;
  4.  redis-check-aof:修复 AOF 持久化文件;
  5.  redis-check-rdb:修复 RDB 持久化文件;
  6.  redis-cli:Redis 命令行工具。

Common redis commands:

Set creation
get View
keys * View all
rename (will overwrite)
renamenx (checks whether there is a duplicate name, and then decides whether to execute the rename command)
del (command can delete the specified key of the current database)
exists (command can determine whether a key value exists)
type (Use the type command to get the value type corresponding to the key)
select (Switch Database)
move (Move Data)
flushdb (clear the current database data)
flushall (clear all database data)

  1.  -p:指定服务器端口;
  2.  -s:指定服务器 socket;
  3.  -c:指定并发连接数;
  4.  -n:指定请求数;
  5.  -d:以字节的形式指定 SET/GET 值的数据大小;
  6.  -k:1=keep alive 0=reconnect;
  7.  -r:SET/GET/INCR 使用随机 key, SADD 使用随机值;
  8.  -P:通过管道传输<numreq>请求;
  9.  -q:强制退出 redis。仅显示 query/sec 值;
  10.  --csv:以 CSV 格式输出;
  11.  -l:生成循环,永久执行测试;
  12.  -t:仅运行以逗号分隔的测试命令列表;
  13.  -I:Idle 模式。仅打开 N 个 idle 连接并等待。

Five: Redis persistence

Redis All data is stored in memory and then saved to disk asynchronously from time to time.(This is calledSemi-persistent mode”); You can also write every data change to a append only file(aof)in(This is calledFull persistence mode”)

because Redis The data is stored in memory. If no persistence is configured,Redis After restarting, all data will be lost. Therefore, you need to turn on Redis The persistence function saves the data to disk. Redis After reboot, the data can be restored from the disk.Redis Provides two ways to persist, one is RDBRedis DataBase Persistence (the principle is to ReidsIn-memory database logging timing dump to disk RDB persistence), the other is AOFappend only filePersistence (the principle is to Reids The operation log is written to the file in append mode).

1: The difference between RDB and AOF

(1)What is RDB?

The default method

RDB persistence is the process of generating a snapshot of the current process data and saving it to the hard disk. The RDB persistence process can be triggered manually or automatically.

Trigger mechanism: Manual trigger corresponds to the save and bgsave commands respectively

save command: blocks the current Redis server until the RDB process is completed. This may cause time blocking for instances with large amounts of memory. It is not recommended for online environments.

bgsave command: The Redis process executes the fork (function used to create a process) operation to create a child process. The RDB persistence process is the responsibility of the child process and ends automatically after completion. Blocking only occurs in the fork phase.

(2) Advantages and disadvantages of RDB:

Advantages of RDB:

RDB is a compact and compressed binary file that represents a snapshot of Redis data at a certain point in time. It is very suitable for backup, full replication and other scenarios. For example, perform bgsave backup every 6 hours and copy the RDB file to a remote machine or file system for disaster recovery.

Redis loads RDB to restore data much faster than the AOF method.

Disadvantages of RDB:

RDB data cannot be persisted in real time or seconds. This is because bgsave creates a child process by forking each time it runs, which is a heavyweight operation and the cost of frequent execution is too high.

RDB files are saved in a specific binary format. There are multiple RDB versions in different formats during the evolution of Redis versions. There is a problem that the old version of Redis service is not compatible with the new version of RDB format.

2. What is AOF

AOF (append only file) persistence: Each write command is recorded in an independent log, and the command in the AOF file is re-executed when restarting to achieve the purpose of data recovery. The main function of AOF is to solve the real-time persistence of data, and it is currently the mainstream of Redis persistence.

The criteria for choosing between the two:

Sacrifice some performance in exchange for higher cache consistency (AOF),

When write operations are frequent, do not enable backup to obtain higher performance. save When you have time, make a backup (RDB

Remark:

If a persistent file needs to be loaded after redies is restarted, the AOF file will be selected.

If RDB is enabled first and then AOF is enabled, and RDB is persisted first, the content in the RDB file will be overwritten by AOF.

3: Redis persistence configuration

(1) RDB persistence configuration

[root@localhost ~]# vim /etc/redis/6379.conf

Open 6379.conf After the file, search save, you can see the configuration information as shown below.

  •  save 900 1: After 900 seconds (15 minutes), if at least 1 key has changed, dump the memory snapshot.
  •  save 300 10: After 300 seconds (5 minutes), if at least 10 keys have changed, dump the memory snapshot.
  •  save 60 10000: After 60 seconds (1 minute), if at least 10,000 keys have changed, dump the memory snapshot.
  • dbfilename dump.rdb :RDB file name ##254 lines
  • dir /var/lib/redis/6379: RDB file path ##264 lines
  • rdbcompression yes: whether to perform compression ##242 lines

(2) AOF persistence configuration

exist Redis There are three synchronization modes in the configuration file, they are:

  • appendonly yes: Enable AOF persistence (default is no) ##673 lines
  • appendfilename "appendonly.aof": AOF file name ##677 lines
  • # appendfsync always
  • appendfsync everysec
  • # appendfsync no

always: Synchronous persistence, every data change will be written to disk immediately

everysec: Default recommendation, asynchronous recording once per second (default value)

no: no synchronization, let the operating system decide how to synchronize

  • aof-load-truncated yes          ##769行

Ignore the last potentially problematic instruction

[root@localhost ~]#/etc/init.d/redis_6379 restart

(2) AOF rewrite

To solve AOF To solve the problem of increasing file size, users can Redis send BGREWRITEAOFOrder.BGREWRITEAOF The command will be removed by AOF Redundant commands in the file are rewritten (rewriteAOFFile, AOF The file size is as small as possible.

127.0.0.1:6379> bgrewriteaof

Background append only file rewriting started

# When the log is BGREWRITEAOF,ifno-appendfsync-on-rewriteIf set to yes, new write operations will not be synchronized with fsync, but will be temporarily stored in the buffer., to avoid disk IO operation conflicts, wait until the rewrite is complete before writing. The default value in Redis is no

no-appendfsync-on-rewrite no 

# When the current AOF file size is twice the size of the AOF file at the last log rewrite, a BGREWRITEAOF operation occurs

auto-aof-rewrite-percentage 100 

Remark:

100RefersaofFile growth ratio refers to the growth ratio of the current aof file compared to the last rewrite100Twice

#The minimum value for executing the BGREWRITEAOF command on the current AOF file to avoid frequent BGREWRITEAOF due to the small file size when Reids is just started

auto-aof-rewrite-min-size 64mb

6. Performance Management

1: View memory information

192.168.9.236:7001> info memory

used_memory:1210776 #The size of the memory used.In bytes
used_memory_human:1.15M #With unit display,In M
used_memory_rss:7802880 # How much memory is used by redis from the operating system's perspective
used_memory_rss_human:7.44M # Display with unit
maxmemory:1073741824 # Maximum memory size
maxmemory_human:1.00G # Display with units

2: Recycling strategy

maxmemory-policy: Recycling strategy

Ø volatile-lru: It allows Redis Pick the least recently used key To delete

Ø volatile-ttlaccording tokeyExpiration time to eliminate

Ø volatile-random: Randomly select data from the data set with set expiration time and eliminate it;

Ø allkeys-lru:use LRU The algorithm eliminates data from all data sets;

Ø allkeys-random: Randomly select data from the data set and eliminate them;

Ø noeviction: Disable data elimination (default value)

Remark:

Set the expiration time of the key

expire v1 10

The expiration time of v1 is 10 seconds

Remark:

when Redis Due to memory pressure, a key hour,Redis The first consideration is not to recycle the oldest data, but the least recently used data. key or about to expire key Randomly select one key, removed from the dataset

redis set password

1. Setting method

Method 1: Set the password through the configuration file redis.conf

Find the requirepass keyword, followed by the password. It is commented out by default, which means that no password is required by default, as follows:

  

 Open the comment, set it as your own password, and restart.

Method 2: Set password by naming

Use redis-cli to connect to redis and execute the following command

config set requirepass 123456

After the execution is completed, there is no need to restart. Exit the client and enter the password when you log in again.

2. Connection method

1. Enter the password when connecting

[root@localhost bin]# ./redis-cli -a 123456

2. Connect first and then enter the password

[root@localhost bin]# ./redis-cli127.0.0.1:6379> auth 123456

3. Closing method

[root@localhost bin]# ./redis-cli -a 123456 shutdown

4. Differences

1) The password set by modifying the configuration file will take effect permanently; the password set by using the command will take effect temporarily and will become invalid after the restart.

2) To modify the password set in the configuration file, you need to restart the computer to take effect; the password set using the command will take effect after logging out and then logging in, and will become invalid after restarting the computer.

3) The command has a higher priority than the configuration file