Technology Sharing

Python Network Programming - TCP/IP

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Link Layer

Frame composition (in order):

Target MAC: 6B

Source MAC: 6B

Type: 2B

Data: 46B-1500B

CRC:4B

The source MAC is the host network card address, the type is the data type of the source network layer, ipv4 is 0800, arp is 0806, pppoe is 8864, 1qtag is 8100, and ipv6 is 86dd.

Python can obtain the local MAC address through the uuid module

Network layer

Datagram Structure

Header: 20-60B, the first 20B are mandatory, the last 40B are optional

Data: 0-65516B

Header structure

Version: 4 bits, indicating ipv4 or ipv6

ihl: 4 bits, indicating the header length, the unit is 4B, so the maximum is 4x15=60B, which is exactly the maximum length of the header

ds: differentiated service, 8 bits, the first three bits indicate priority, the next four bits indicate minimum delay, maximum throughput, highest reliability and minimum cost, the last bit is not used

Total length: 16 bits, indicating the total length of the header and data, in bytes. Note that the link layer data length is 46-1500. When the network layer datagram is less than 46 bytes, it will be padded with data, and when it exceeds 1500 bytes, it will be sliced.

Identifier: 16 bits, representing the unique ID of the shard data

Flag: 3 bits, the highest bit is reserved, the second bit indicates whether it can be fragmented, 0 indicates it can be fragmented, 1 indicates it cannot be fragmented, and the last bit indicates whether this datagram is fragmented, 0 indicates it is not fragmented and it is the last fragment, 1 indicates it is fragmented and there are more fragments to come

Slice displacement: 13 bits, 8 bytes

Lifetime, 8 bits, maximum 255, indicating the maximum number of hops in the route

Protocol: 8 bits, indicating the type of IP datagram, such as ICMP, IGMP, TCP, UDP, etc.

Header checksum: 16 bits

Source IP address and destination IP address, 32 bits each

Python can obtain network information through the psutil and netifaces modules. The netifaces version is deprecated. You can install netifacce2

Transport Layer

udp

Data availability is not guaranteed

Header 8B: Source port 2B, destination port 2B, length 2B, checksum 2B

Data 0-65527B

TCP

Create a connection with 3 handshakes and close the connection with 4 handshakes

Datagram Structure

First 20-60B

Data 0-65535B

Header structure

Source port 16 bits

Destination port 16 bits

Sequence number: 32 bits: Each byte of data has a sequence number.After pow(2,32) starts from 0

Confirmation number: 32 bits, sent by the receiver to the sender, the sequence number of the first byte of the next data message that the receiver hopes to receive. For example, if the sender sends bytes 0-999, the confirmation number sent by the receiver is 1000.

Data offset: 4 bits, unit 4B, similar to the IHL of the IPv4 header

Reserved: 6 bits, usually 0

URG: 1 bit, 1 indicates urgent data, the urgent pointer is valid

ACK: 1 bit, 1 means the confirmation number field is valid

PSH: 1 bit, 1 means the message is created and sent out immediately. The receiving end will immediately hand it over to the process after receiving it, instead of putting it in the cache and waiting for the cache to be full before handing it over.

RST: 1 bit. If it is 1, it means that there is a serious problem with the current connection. Xu Ao releases the connection and re-establishes the connection. RST is 1 and can also be used to refuse to accept illegal messages or refuse to open illegal connections.

SYN: 1 bit, 1 means the message is a connection request or connection acceptance message

FIN: 1 bit, 1 means the data has been sent and the connection needs to be released

Window: 16 bits, sent by the receiver to the sender, indicating the expected length of the next message to be received

Checksum: 16 bits

Urgent pointer: 16 bits, when URG is 1, it indicates the end position of the urgent data

Python gets network information

psutil can obtain some network statistics. The net_io_counters method collects network statistics for the entire machine by default. If pernic=True is entered, statistics for each network card will be collected separately.

Application Layer

http

HTTP is stateless communication by default, and stateful services can be provided through cookies

The http port is usually 80

Message structure

Request message: method, URL, version, other header fields, message entity (optional)

Response message: version, status code, phrase, other header fields, message entity (optional)

Methods: There are 8 methods, get, post, head, put, delete, option, connect, trace

Version: http version number

Other header fields: such as Cache-Control, Accept-Encoding, etc.

Message entity: The length is unlimited and can be 0

Status code: a three-digit number such as 404.1xx means the server has received the request, 2xx means the server has accepted the client request, 3xx means the server has redirected the client request, 4xx means the client request is incorrect, and 5xx means there is a problem on the server.

Phrase: An annotation for the status code. For example, the phrase for the 202 return code is Accept, and the phrase for the 404 return code is Not Fount.

https

HTTP has no security guarantee and can be captured. HTTPS uses SSL or TLS to encrypt data messages and then transmit them via TCP. The default port443

Symmetric encryption and asymmetric encryption are generally used. Symmetric encryption is generally used for message encryption, and asymmetric encryption is generally used for password encryption.

Request Process

When the client requests, it sends a set of encryption rules it supports to the server.

The server combines a set of encryption and hash algorithms according to the client's encryption rules, and returns it to the client in the form of a digital certificate along with its own identity information.

The client verifies the legitimacy of the server certificate. If it passes, it generates a random password and encrypts the random password with the server's public key. The client calculates the hash of the message to be sent, encrypts the message to be sent and the message hash value with the random password, and finally returns the random password encrypted with the server's public key, the message encrypted with the random password, and the message hash to the server.

After receiving the client data, the server uses its own private key to decrypt the random number password, and then uses the random number password to decrypt the message and the message hash value, and compares the decrypted hash value with the hash value of the received message to see if they are consistent. If they are consistent, the connection between the client and the server is successfully established. After that, both parties use the random number password to encrypt and decrypt the data. During the subsequent data transmission process, if an error occurs or the hash is inconsistent, the connection is automatically terminated

ftp

dns

snmp

SNMP is a simple mail transfer protocol used to transfer mail from a source address to a destination address. The port is usually 25.

It is a commonly used email sending protocol. After sending an email, it will be sent to the snmp server

pop3

The recipient uses it to receive mails from other users on their own servers. The port is usually 110.

POP3 is an offline protocol. When a user logs in, the emails on the POP3 server will be downloaded to the local computer at one time. At the same time, the emails on the server will be automatically deleted.

imap is an online mail receiving protocol, which makes up for the offline problem of pop3. It is the most commonly used mail receiving protocol at present. The port is generally 143

dhcp

When a DHCP client connects to the network, it broadcasts a DHCP disocover message. If no response is received after a limited number of broadcasts according to certain rules, DHCP fails.

After receiving the message, the server will select an IP from the available address pool and send a DHCP offer message in broadcast form to return the IP to the client.

After the client receives the IP assigned by the DHCP server, it will send an ARP request to see if the IP is occupied. If it is occupied, the client will send a DHCP decline report to the DHCP server. If it is not occupied, it will send a DHCP request to the server. After receiving it, the server will send a DHCP ACK response to the client, and the DHCP service process ends here.

The DHCP server port is usually 67, and the DHCP client port is usually 68. DHCP uses UDP for communication