Detailed Description
When viewing the live stream of a device on a mini program or App, the picture lags, is unsmooth, or has a mosaic screen effect.
Cause Analysis
There are many reasons for lagging. They need to be excluded one by one. The troubleshooting methods are as follows:
Save the audio and video stream received on the App or mini program locally, usually in flv format; use a third-party player (PotPlayer or VLC is recommended) to view the locally saved audio and video data. If the playback is still choppy, start analysis from Reason 1, otherwise start from Reason 2.
Reason 1: This kind of stutter is generally due to missing video data. Extract H264/H265 raw data (recommended to use ffmpeg) from the locally saved audio and video streams. Use elecard to analyze the H264/H265 raw data and find the time point of stutter. Confirm if there is any frame loss (judged by the frame_num value in the silce header). Usually, there is frame loss, which is generally caused at the device end. Search for the audio and video data that occurred at this time point on the device end. Check if there are any errors when calling
iv_avt_send_stream, or if there is frame loss in the data generated by the encoder (print out the real-time GOP value of the encoder to judge).Reason 2: This kind of stutter is generally caused by the network bandwidth being lower than the data bitrate or issues with the time sequence. First, judge if there is a water level alarm at the stuttering device end. Then, use flv analysis tools to analyze if there are issues with the data time sequence. If there are no issues with the time sequence, bitrate control needs to be done at the device end.
Solution
If the stutter is caused by the bandwidth at the device end, bitrate adaptation is required.
If it is caused by frame loss at the device end, users are advised to check if there are any problems with the frame loss logic in the code.
If there is a problem reading encoder data, users are advised to check whether the CPU occupancy is too high or the thread priority of fetching data is too low in business logic.