Technology Sharing

Record of the problem of bandwidth surge in the intranet streaming of SRS live broadcast

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Background

srsDeployed on a cloud server, 32-core CPU, 64G memory, 300M bandwidth.
The client pulls the stream from SRS and finds that the external client pulls the stream.cpuThe bandwidth is normal. However, when the intranet client pulls the stream, if the number of people pulling the stream exceeds 5, the bandwidth will soar rapidly.
insert image description here

Troubleshooting

Use srs-bench to perform srs stress testing, vlc player srs streaming, and client srs streaming

Streaming

Select ffmpeg for streaming

ffmpeg -re -i C:UserswDesktoptest.mp4 -vcodec copy -acodec copy -f flv -y rtmp://27.128.236.38/live/livestream
  • 1

A.srs-bench pull flow

./objs/srs_bench -sr webrtc://27.128.236.38/live/livestream -nn 10
  • 1

Reference articles for srs-bench compilation and deployment:SRS stress test-SRS-Bench

B. VLC streaming

媒体->打开网络串流
输入url:https://ip:8088/live/livestream.flv
  • 1
  • 2

Conduct srs-bench, client and vlc stress tests in Xi'an, Nanjing and Beijing
The test log is as follows:

environment1 person5-people6 people10 people30 people
Xi'an server stress test A network segmentnormalnormalabnormalabnormalabnormal
Xi'an server stress test B network segmentnormalnormalnormalUnstableUnstable
Xi'an real clientnormalnormalnormalabnormalabnormal
Xi'an client stress testnormalnormalnormalabnormalabnormal
Nanjing Servernormalnormalnormalnormalnormal
Nanjing Real Clientnormalnormalnormalnormal/
Nanjing Client Stress Testnormalnormalnormalnormal/
Beijing Servernormalnormalabnormalabnormalabnormal
Beijing Real Clientnormalnormalnormalnormal/
External network stress testnormalnormalnormalnormalnormal
vlc stress testnormalnormalnormalnormal/

Validation results

External network environment stress test, bandwidth is normal, CPU is normal
Intranet environment stress test, bandwidth will soar to 10 times when there are more than 5 people

Packet capture comparison

insert image description here

analyze

The delay rate in abnormal environment is higher than that in normal environment, and there is packet loss and retransmission.

Check the SRS official websitesrs official website
Core protocol – config in webrtc for the configuration of webrtc part

The first part, rtc_server, is the global RTC server configuration. Some key configurations include:

enabled:是否开启RTC服务器,默认是off。
listen:侦听的RTC端口,注意是UDP协议。
candidate:服务器提供服务的IP地址,由于RTC的特殊性,必须配置这个地址。详细参考Config: Candidate
tcp.listen: 使用TCP传输WebRTC媒体数据,侦听的TCP端口。详细参考WebRTC over TCP
  • 1
  • 2
  • 3
  • 4

The second part is the RTC configuration in each vhost. Some key configurations include:

rtc.enabled:是否开启RTC能力,默认是off。
rtc.rtmp_to_rtc:是否开启RTMP转RTC。
rtc.rtc_to_rtmp:是否开启RTC转RTMP。
rtc.stun_timeout:会话超时时间,单位秒。
rtc.nack:是否开启NACK的支持,即丢包重传,默认on。
rtc.twcc:是否开启TWCC的支持,即拥塞控制的反馈机制,默认on。
rtc.dtls_role:DTLS角色,active就是DTLS Client(主动发起),passive是DTLS Server(被动接受)
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

It is found that the rtc.nack configuration is on by default, which means that if SRS finds that there is packet loss, it will continue to retransmit data

in conclusion

After checking the company's intranet environment, it was found that the intranet environment had bandwidth restrictions. When the client's streaming bandwidth exceeded a certain size, the streaming would be restricted.
At this time, SRS regards the network as abnormal, and retransmits the packet loss, so the bandwidth continues to soar.

solve

Solution 1: Remove bandwidth restrictions in the intranet environment

Advantages: Ensure the stability of live streaming
Disadvantage: The company cannot monitor client bandwidth, which increases costs

Scenario 2:

Advantages: When the intranet and extranet are working properly, live streaming is normal and bandwidth consumption is low.
Defect: Network abnormality, SRS does not retransmit lost packets, mosaic, freeze and other problems may occur