Technology Sharing

Redis configuration and optimization

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

1. Relational Database and Non-relational Database

1. Relational Database

A relational database is a structured database, built on a relational model database, and is record-oriented.

Common relational databases: Oracle, MySQL, SQL Server, Microsoft Access, DB2.

2. Non-relational databases

NOSQL=Not Only SQL is a general term for non-relational databases. Depending on the storage method, storage structure and usage scenario, it is called a non-relational database. Databases other than mainstream relational databases can be called non-relational databases.

Common non-relational databases: Redis, MongoDB, Hbase, CouhDB.

2. Redis Basics

1. Introduction to Redis

Redis is an open source NoSQL database written in C language. It runs based on memory and supports persistence. It uses key-value storage. Its port number is 6379.

2. Advantages of Redis

  1. It has a higher data reading and writing speed.
  2. Supports a wide variety of data types: key-value, strings, lists, hashes, sets, and ordered sets.
  3. Supports data persistence. Data is saved on disk and can be loaded and used after restart.
  4. Atomicity.
  5. Support data backup.

3. Redis installation

1. Install Redis

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# setenforce 0

[root@localhost ~]# yum -y install gcc* zlib-devel

[root@localhost ~]# tar xvzf redis-4.0.9.tar.gz

[root@localhost ~]# cd redis-4.0.9/

[root@localhost redis-4.0.9]# make

Notice:

After make, an error message will be generated.

solution:
Solution 1: Use make MALLOC=libc to specify the memory allocator as libc for compilation
Solution 2: make clean && make distclean

[root@localhost redis-4.0.9]# make PREFIX=/usr/local/redis install
[root@localhost utils]# ln -s /usr/local/redis/bin/* /usr/local/bin/

 [root@localhost redis-4.0.9]# cd utils/
Among them: install_server.sh is the initialization script

[root@localhost utils]# ./install_server.sh
Please select the redis executable path [] /usr/local/redis/bin/redis-server (Give the executable path)

View process and service control
[root@localhost utils]# netstat -anpt | grep redis
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      5360/redis-server 1 
[root@localhost utils]# /etc/init.d/redis_6379 stop     (redis is closed)
[root@localhost utils]# /etc/init.d/redis_6379 start        (Open)
[root@localhost utils]# /etc/init.d/redis_6379 status      (state)

Modification of configuration parameters
[root@localhost ~]#vim /etc/redis/6379.conf
bind 127.0.0.1 192.168.10.101                    //Host address to listen to
port 6379                                                          //port
daemonize yes                                              // Enable the daemon process
pidfile /var/run/redis_6379.pid                    //Specify PID file
loglevel notice                                                //Log level
logfile /var/log/redis_6379.log                        //Specify the log file

[root@localhost~]#/etc/init.d/redis_6379 restart

2. redis command tool

[root@localhost utils]# redis-cli                 (Local login)
[root@localhost utils]# redis-cli -h 192.168.10.101 -p 6379     (Remote login)
192.168.10.101:6379> ping (Test service is running normally)
PONG

3: redis-benchmark test tool

redis-benchmark is the official Redis performance testing tool that can effectively test the performance of the Redis service.

  • -h: specifies the server host name;
  • -p: specifies the server port;
  • -s: specifies the server socket;
  • -c: specifies the number of concurrent connections;
  • -n: specifies the number of requests;
  • -d: specifies the data size of the SET/GET value in bytes;
  •  -k:1=keep alive 0=reconnect;
  • -r: SET/GET/INCR uses a random key, SADD uses a random value;
  • -P: transfer through pipe<numreq> ask;
  • -q: Force exit redis. Only query/sec value is displayed;
  • --csv: Output in CSV format;
  • -l: Generate a loop and execute the test forever;
  • -t: only run the comma-separated list of test commands;
  • -I: Idle mode. Only open N idle connections and wait.
  • (1) Test request performance

    [root@localhost ~]#redis-benchmark -h 192.168.10.101 -p 6379 -c 100 -n 100000

    Remark:

    -h: specifies the server host name;

  • -p: specifies the server port;

    -c: specifies the number of concurrent connections;

    -n: specifies the number of requests;

(2) Test access performance

[root@localhost ~]#redis-benchmark -h 192.168.10.101 -p 6379 -q -d 100

Remark:

-h: specifies the server host name;

-p: specifies the server port;

-d: specifies the data size of the SET/GET value in bytes;

-q: Force exit redis. Only query/sec value is displayed;

  1. 4. Common commands for Redis database

set stores data Command format set key value

get Get data Command format get key

(I) Key related commands

1. Add key-value pairs

127.0.0.1:6379> set 1 1
OK
127.0.0.1:6379> set 2 2
OK
127.0.0.1:6379> set 3 3
OK

2. View all values ​​in the database

127.0.0.1:6379> keys *
1) "3"
2) "1"
3) "2"

3. View the data starting with v in the database
127.0.0.1:6379>KEYS v*

4. View any data starting with v in the database

127.0.0.1:6379>KEYS v?

5.View the current database v At the beginningContains any two digits of data

127.0.0.1:6379>KEYS v??

2. exists

Determine whether the value exists.

127.0.0.1:6379>exists f5        Determine whether f5 exists

(integer) 1          The result is 1, indicating that the F5 key exists

like

(integer) 0          The result is 0, indicating that the F5 key does not exist.

(III) del

del The command can delete the specified key

127.0.0.1:6379> del v5      Delete v5 in the database

(integer) 1

127.0.0.1:6379>get v5

(nil)

(IV) type

use type The command can be obtained key corresponding value Value Types

127.0.0.1:6379>type k1

string

Remark:

Data types supported by redis

  • String: String is the simplest type, which is a common set and get, used as key value cache.
  • Hash: Hash algorithm, a structure similar to map, generally can cache structured data, such as an object, in redis
  • List: List is an ordered list. You can use it to store some list-like data structures, such as fan lists, article comment lists, etc.
  • Set: Set is an unordered collection with automatic deduplication.
  • Sorted Set: Sorted Set is a sorted set that removes duplicates but can be sorted. When writing, a score is given and the data is automatically sorted based on the score.

5. rename

rename The command is to key Rename

In actual use, it is recommended to use exists Command to view target key Whether it exists, and then decide whether to execute rename command to avoid overwriting important data.

127.0.0.1:6379>rename v22 v2        Rename v22 to v2

OK

(VI) renamenx

renamenx The command is used to key Rename and check if the new name exists.

userenamenx When the command is rename, if the target key If it exists, it will not be renamed.

dbsize

dbsize The command is used to view the current database key Number of.

127.0.0.1:6379> dbsize

(integer) 5

5. Common commands for multiple databases

1. Switching between multiple databases

Redis Included by default without any changes 16 databases, the database names are numbers 0-15 To be named in sequence

(1)Switch to the sequence number 10 Database

127.0.0.1:6379>select 10

OK

(2)Switch to the sequence number 15 Database

127.0.0.1:6379[10]>select 15

OK

(3)Switch to the sequence number 0 Database

127.0.0.1:6379[15]>select 0

2. Moving Data Between Multiple Databases

127.0.0.1:6379>set k1 100          Create k1 in database 0

OK

127.0.0.1:6379>get k1

"100"

127.0.0.1:6379>move k1 1               // Move k1 from database 0 to database 1

(integer) 1

127.0.0.1:6379>select 1                    //Switch to target database 1

OK

127.0.0.1:6379[1]>get k1                    //View the moved data

"100"

127.0.0.1:6379[1]> select 0

OK

127.0.0.1:6379> get k1                        //The value of k1 cannot be found in database 0

(nil)

(III) Clearing data in the database

Clear the current database data, use FLUSHDB

Command implementation; clear all database data, use FLUSHALL Command implementation.

6. Redis persistence

       Redis All data is stored in memory and then saved to disk asynchronously from time to time.(This is calledSemi-persistent mode”); You can also write every data change to a append only file(aof)in(This is calledFull persistence mode”)

because Redis The data is stored in memory. If no persistence is configured,Redis After restarting, all data will be lost. Therefore, you need to turn on Redis The persistence function saves the data to disk. Redis After reboot, the data can be restored from the disk.Redis Provides two ways to persist, one is RDBRedis DataBase Persistence (the principle is to ReidsIn-memory database records are dumped regularly (dump) to disk RDB persistence), the other is AOFappend only filePersistence (the principle is to Reids The operation log is written to the file in append mode).

The difference between RDB and AOF

ROB writes data snapshots to disk within a specified time interval. It is a child process of fork. It first writes data to a temporary folder. After success, it replaces the previous file and stores it in binary compression.

AOF records every write and delete operation of the server in the form of logs. Query operations are not recorded but recorded in text form.

 

Advantages and disadvantages of RDB and AOF

RDB advantages:

RDB is a compact and compressed binary file that represents a snapshot of Redis data at a certain point in time. It is very suitable for backup, full replication and other scenarios. For example, perform bgsave backup every 6 hours and copy the RDB file to a remote machine or file system for disaster recovery.

Fast data recovery speed.

Maximize performance

High startup efficiency

RDBshortcoming:

Any data not saved before the crash will be lost.

       RDB is completed through the fork child process, consuming resources

AOF advantages:

High data durability

In append mode, a crash will not destroy the log file contents.

The rewrite mechanism can be enabled to protect data security.

AOF Disadvantages:

AOF data recovery speed is slow

AOF operation efficiency is low

The criteria for choosing between the two:

Sacrifice some performance in exchange for higher cache consistency (AOF),

When write operations are frequent, do not enable backup to obtain higher performance. save When you have time, make a backup (RDB

Remark:

If redies restarts, a persistent file needs to be loaded.priorityThe AOF file will be selected.

If RDB is enabled first and then AOF is enabled, and RDB is persisted first, the content in the RDB file will be overwritten by AOF.

7. Redis persistence configuration

1. RDB persistence configuration

[root@localhost ~]# vim /etc/redis/6379.conf

Open 6379.conf After the file, search save, you can see the configuration information as shown below.

  • save 900 1: After 900 seconds (15 minutes), if at least one key has changed, dump the memory snapshot.
  • save 300 10: After 300 seconds (5 minutes), if at least 10 keys have changed, dump the memory snapshot.
  • save 60 10000: After 60 seconds (1 minute), if at least 10,000 keys have changed, dump the memory snapshot.
  • dbfilename dump.rdb :RDB file name##254 lines
  • dir /var/lib/redis/6379 : RDB file path##264 lines
  • rdbcompression yes: whether to perform compression##242 lines

(II) AOF persistence configuration

exist Redis There are three synchronization modes in the configuration file, they are:

  • appendonly yes                             Enable AOF persistence (default is no) ##673 lines
  • appendfilename "appendonly.aof "                    AOF file name##Line 677
  • # appendfsync always
  • appendfsync everysec
  • # appendfsync no

always: Synchronous persistence, every data change will be written to disk immediately ##702 lines

everysec: Default recommendation, asynchronous recording once per second (default value)

no: no synchronization, let the operating system decide how to synchronize

  • aof-load-truncated yes   ##769行

Ignore the last potentially problematic instruction

[root@localhost ~]#/etc/init.d/redis_6379 restart

8. Performance Issues

(I) View memory information

192.168.9.236:7001> info memory

used_memory:1210776                          The size of the memory used, in bytes
used_memory_human:1.15M                    Displayed with units, in M
used_memory_rss:7802880                    How much memory does redis take up from the operating system perspective
used_memory_rss_human:7.44M                        With unit display
maxmemory:1073741824                                   Maximum memory size
maxmemory_human:1.00G                                With unit display

2. Recycling strategy

maxmemory-policy: Recycling strategy

 volatile-lru: It allows Redis Pick the least recently used key To delete

 volatile-ttlEliminate according to the expiration time of the key 

 volatile-random: Randomly select data from the data set with set expiration time and eliminate it;

 allkeys-lru:use LRU The algorithm eliminates data from all data sets;

 allkeys-random: Randomly select data from the data set and eliminate them;

 noeviction: Disable data elimination (default value)

Set the expiration time of the key

127.0.0.1:6379> expire v1 10          The expiration time of v1 is 10 seconds

Remark:

when Redis Due to memory pressure, a key hour,Redis The first consideration is not to recycle the oldest data, but the least recently used data. key or about to expire key Randomly select one key, removed from the dataset