Detailed Description
When viewing the live stream of a device on a mini program or App, there is significant image delay or a black screen.
Cause Analysis
The image delay is large, which is generally caused by exceptions in the PTS of audio and video frames. The IoT Video SDK requires that the PTS unit of the input audio and video frames must be milliseconds. If the configured unit is not milliseconds, exceptions will occur during viewing.
The image delay is large. It might also be that there is too much cached audio and video data on the device end. Users need to check whether there is too much cached data in the currently connected business.
The large image delay may also be due to too much cached audio and video data in the player. It is mostly accompanied by synchronization issues of audio and video PTS. It is required to check whether the difference in the PTS of audio and video frames is too large on the App side or mini program side.
A black screen is generally also due to problems with the PTS. The most common issue is that the PTS has a loopback and is not monotonically increasing, causing playback problems.
Solution
If it is a PTS issue, the device-side user needs to check whether there is an issue with the PTS configuration of the input audio and video frames. The device-side IoT Video SDK will not synchronize and cache the audio and video frames or modify their PTS when sending.
If the issue is caused by excessive cached data on the device end, users need to check their own business logic issues. The IoT Video SDK on the device end will only cache data due to high network latency; it will not cache otherwise.
Another business scenario is that some camera equipment has a PTZ function. When turning, in order to filter out the motor sound, it does not send audio but waits until the equipment is stationary before sending. This way, it can also destroy the continuity of the PTS, resulting in a large delay. It is recommended to solve this problem by sending silent frames.