Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

DR mode:

Three modes of LVS NAT address translation

DR Direct Routing Mode

tun tunnel mode

The scheduler is the most important in the entire LVS cluster. In NAT mode, it is responsible for accepting requests, forwarding traffic according to the load balancing algorithm, and sending responses to clients.

DR mode: The scheduler is still responsible for accepting requests and also forwards traffic to the RS according to the load balancing algorithm. The response is directly sent by the RS to the client.

Direct Routing is a Layer 2 forwarding mode. Layer 2 forwarding is data frames. It is forwarded based on the source MAC address and the destination MAC address.

The source and destination IP addresses of the data packets will not be modified, and they will be forwarded based on the MAC addresses of the data packets.

In DR mode, LVS also maintains a virtual IP address, and all requests are sent to this VIP. Since it is a Layer 2 forwarding, when the client's request reaches the scheduler, an RS is selected according to the load balancing algorithm, and the MAC of the VIP server is modified to the MAC address of the RS. After the RS processes the request, it directly sends the response to the client based on the source MAC address of the client in the message, without the need for a scheduler.

1. The scheduler is configured with VIP, and the VIP address is also configured on the RS.

VIP address conflict, the scheduler and RS are in the same network segment, ARP communication is disordered, because it is broadcast to the entire LAN, all devices receive it.

How to block the loopback response of lo so that only the local physical IP address responds.

Modify kernel parameters:

arp_igrone=1

The physical IP address of the system will respond to the request. lo will not respond to ARP requests.

2. When returning the message, the VIP address is still there. How can the client receive the response?

arp_announce=2

The system does not use the source address of the IP data packet to respond to the ARP request, but directly sends the IP address of the physical interface.

Node server configuration:

Implementation of DR mode:

nginx1 RS1 192.168.233.100

nginx2 RS2 192.168.233.110

vip 192.168.233.200

test1 scheduler 192.168.233.10

test2 client 192.168.233.20

route add -host 192.168.233.100 dev lo:0

Set the IP address to 192.168.233.100

Disable the firewall and defense mechanisms of the scheduler (test1), client (test2), rs1 (nginx1), and rs2 (nginx2)

[root@test1 ~]# systemctl stop firewalld
[root@test1 ~]# setenforce 0

The scheduler loads the kernel and installs the ipvsadm tool

[root@test1 ~]# modprobe ip_vs
[root@test1 ~]# yum -y install ipvsadm*

Create VIP address

[root@test1 ~]# cd /etc/sysconfig/network-scripts/
[root@test1 network-scripts]# vim ifcfg-ens33:0
DEVICE=ens33:0
ONBOOT=YES
IPADDR=192.168.233.200
NETMASK=255.255.255.255
[root@test1 network-scripts]# ifup ens33:0

Modify the response parameters of the scheduler

[root@test1 network-scripts]# vim /etc/sysctl.conf
net.ipv4.ip_forward=0
#关闭数据包转发功能
net.ipv4.conf.all.send_redirects=0
#禁止系统发送icmp重定向的消息
net.ipv4.conf.default.send_redirects=0
#禁止默认网络接口发送ICMP重定向的消息
net.ipv4.conf.ens33.send_redirects=0
#禁止ens33设备，禁止发送ICMP重定向消息

Specify VIP and load balancing algorithm

[root@test1 opt]# ipvsadm -C
[root@test1 opt]# ipvsadm -A -t 192.168.233.200:80 -s rr
#指定vip和负载均衡的算法

Add real server specified mode and backup

[root@test1 opt]# ipvsadm -a -t 192.168.233.200:80 -r 192.168.233.100:80 -g
[root@test1 opt]# ipvsadm -a -t 192.168.233.200:80 -r 192.168.233.110:80 -g
[root@test1 opt]# ipvsadm-save /etc/sysconfig/ipvsadm
[root@test1 opt]# systemctl restart ipvsadm
 

Configure 2 real servers (nginx1 nginx2)   

Set the access pages of the two servers to nginx1 and nginx2

[root@nginx1 ~]# vim /usr/local/nginx/html/index.html
                 nginx1
[root@nginx1 ~]# systemctl restart nginx
[root@nginx2 ~]# vim /usr/local/nginx/html/index.html
                 nginx2
[root@nginx2 ~]# systemctl restart nginx

Visit and test it yourselfCreate a loopback interface for the loopback addresses of the two servers

[root@nginx1 ~]# cd /etc/sysconfig/network-scripts/
[root@nginx1 network-scripts]# cp ifcfg-lo ifcfg-lo:0
[root@nginx1 network-scripts]# vim ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.233.200
NETMASK=255.255.255.255
ONBOOT=yes
[root@nginx1 network-scripts]# ifup lo:0

Set the IP address to 192.168.233.100 and add it to the loopback interface. As a VIP of LVS, forward it to RS through the routing mode, so that VIP can identify the real server.

[root@nginx1 network-scripts]# route add -host 192.168.233.200 dev lo:0

Adjusting the kernel response of real servers

[root@nginx1 ~]# vim /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
#设置回环接口忽略来自任何接口的ARP请求
net.ipv4.conf.lo.arp_announce = 2
#设置回环地址仅仅公告本地的ip地址，但是不响应ARP请求
net.ipv4.conf.all.arp_ignore = 1
#设置所有接口忽略来自任何接口的ARP请求
net.ipv4.conf.all.arp_announce = 2
#设置所有接口仅仅公告本地的ip地址，但是不响应ARP请求

Test Results

Three working modes of LVS:

NAT DR TUN

Advantages: The address translation performance is the best WAN, and data packets can be transmitted over longer distances.

Configuring a simple dedicated channel

Disadvantages: Performance bottleneck, does not support cross-segment, requires VPN (costs money)

RS requirements: No restrictions ARP responses from non-physical interfaces must be prohibited Tunnel mode must be supported

Number of RS 10-20 100 100

Interview questions:

Briefly describe the three modes of lvs and their differences

Interview questions:

How to solve keepalive split-brain?

The high availability architecture in the LVS cluster is only for the high availability of the scheduler.

The master and backup schedulers are implemented based on VRRP.

Highly available HA architecture

Main scheduler and backup scheduler (multiple)

When the main scheduler is working normally, the backup is completely in a redundant state (pending). It does not participate in the operation of the cluster. Only when the main scheduler fails, the backup will take over the work of the main scheduler. After the main scheduler recovers, the main scheduler continues to be the entrance of the cluster, and the backup continues to be in a redundant state (depending on the priority).

KeepLive implements the LVS high availability solution based on the VRRP protocol.

1. Multicast address:

224.0.0.18 communicates based on the multicast address, and the primary and backup nodes send messages to each other to determine whether the other node is alive.

2. Determine the position of the primary and backup nodes based on their priority

3. Failure switching: if the primary server fails, the backup server will continue to work; if the primary server recovers, the backup server will continue to wait.

4. The switch between the primary and backup is the switch of the VIP address

Keepalive is specifically for LVS, but it is not exclusive to LVS.