Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Introduction

This article will build a [pseudo] stand-alone redis instance in a k8s environment. Since the io performance of shared storage is relatively low, shared storage is used for data backup, and hostpath is used to store redis data, which helps to improve the io performance of redis. This article will explain how to use two containers as the master-slave form of redis in a pod to implement a stand-alone, highly available redis.

1. Conditions and environment description:

The k8s version is k8s-1.29.4. The environment is built in the telecommunications room. There are six nodes in total. Each node has a non-system SSD disk mounted to the /data/ path. There is NAS shared storage, which uses Alibaba Cloud NAS and uses a dedicated line with a dedicated line bandwidth of 500Mb.

2. Requirements:

• Build a redis instance: stand-alone or master-slave.
• High availability: If a k8s node dies, the service will not be affected for a long time.
• High performance: requires high IO, using shared storage or ceph distributed storage, the reading and writing IO will be greatly discounted.
• Data security: After the switch occurs, the data should be kept as complete as possible.
• Simple installation, easy management and maintenance

3. Implementation principle and description

1. Set up redis in master-slave mode. The data read and write of the master redis falls on the local SSD disk, and the data of the slave redis falls on the external shared NAS storage.
2. Then create a statefulset configuration and place two containers. One container is used as the master redis, mounted to the hostpath, creates a directory with the host name, and stores the rdb file. The other container is used as the slave redis, mounts the nas path, creates a directory with the host name, and stores the rdb file.
3. Adjust the data persistence parameter save from redis and adjust the data storage time to a smaller parameter to ensure that the updated data is stored in the RDB file as soon as possible.
4. Every time the pod is scheduled or restarted, copy the RDB file stored in the NAS to the hostpath path and then start redis.

Note: This is done to allow the master Redis to process business data with high performance, and the slave Redis is responsible for data persistence. The following problems may exist: 1. When the master's read and write data IO is too high, the slave Redis will take longer to write data to the disk because it is mounted on NAS.

4. Detailed steps

4.1. Create a configmap configuration file

The configuration contains three files: the master redis configuration file, the slave redis configuration file, and a simple script that performs different operations based on the execution role. The specific configuration is as follows:
redis-master:conf main redis configuration file, memory is configured to 256M, port is configured to 6379, password is configured: redis#123, storage path is /data/redis.
redis.conf: From the redis configuration file, set the memory to 256M, the port to 6380, the password to redis#123, the storage path to /data/redis-2, and configure data synchronization from port 6379. Adjust the storage time to store data to disk if there are 10 data changes within 60 seconds.
run.sh: Its main function is to determine the role and create a directory according to the host name, and then soft-link to the storage directory. This step is mainly to store the redis data of each service in its own directory, so that when a new redis is created, it will not cause directory conflicts on the node.

apiVersion: v1
kind: ConfigMap
metadata:
  name: defaultapp-redis-standalone-config
  namespace: default
  labels:
    appname: default-app
    app: defaultapp-redis-standalone-config
data:
  redis-master.conf: |
    port 6379
    maxmemory 256mb
    requirepass redis#123
    dir /data/redis
  redis.conf: |
    port 6380
    maxmemory 256mb
    requirepass redis#123
    save 3600 1 300 5 60 10
    dir /data/redis-2
    replicaof 127.0.0.1 6379
    masterauth redis#123
  run.sh: |
    #!/bin/sh
    role=$1
    hname=$(hostname)
    if [ $role == "master" ];then
        echo "run redis master"
        ls /etc/redis/ -l
        [ -d /home/redis/$hname ] || mkdir -p /home/redis/$hname
        [ -d /data ] || mkdir /data
        ln -s /home/redis/$hname /data/redis
        if [ -f /home/redis-2/$hname/dump.rdb ];then
                cp -vf /home/redis-2/$hname/dump.rdb /data/redis/
        else
                echo "[info] no bak dump.rdb"
        fi
        redis-server /etc/redis/redis-master.conf
    else
        echo "run redis slave"
        ls /etc/redis/ -l
        [ -d /home/redis-2/$hname ] || mkdir -p /home/redis-2/$hname
        [ -d /data ] || mkdir /data
        ln -s /home/redis-2/$hname /data/redis-2
        redis-server /etc/redis/redis.conf
    fi

                    上一篇
                    下一篇