多路RTSP转RTMP推送方案的两个选择

原创

音视频牛哥

发布于 2024-08-09 11:10:46

950

发布于 2024-08-09 11:10:46

技术选型

RTSP转RTMP推送到流媒体服务器，说起来技术实现不难，简单来说，获取RTSP流后，拿到未经解码的H.264/H.265和audio数据，重新打包RTMP发送出去即可。需要注意的是，大多RTSP转RTMP模块，需要长时间运行，所以，需要有好多错误处理和自动重连机制，确保转发模块的稳定性。以下是两个可选的技术方案：

方案1：FFMPEG命令转发

ffmpeg -i rtsp://[摄像头地址]/[流媒体地址] -c:v libx264 -preset veryfast -maxrate 3000k -bufsize 6000k -pix_fmt yuv420p -g 50 -c:a aac -b:a 160k -ac 2 -ar 44100 -f flv rtmp://[服务器地址]/[应用名称]/[流密钥]

rtsp://[摄像头地址]/[流媒体地址] 是摄像头的RTSP流地址。
-c:v libx264 指定视频编码器为libx264。
-preset veryfast 设置编码速度为非常快，以牺牲一些压缩效率换取更快的编码速度。
-maxrate 和 -bufsize 设置最大码率和缓冲区大小。
-pix_fmt yuv420p 设置像素格式为YUV420P，这是RTMP兼容的格式。
-g 50 设置关键帧间隔为50帧。
-c:a aac 指定音频编码器为AAC。
-b:a、-ac 和 -ar 分别设置音频比特率、声道数和采样率。
-f flv 指定输出格式为FLV，RTMP流通常以FLV格式封装。
rtmp://[服务器地址]/[应用名称]/[流密钥] 是目标RTMP服务器的推送地址。

上述命令中的参数可能需要根据实际情况进行调整。

方案2：SmartRelaySDK

大牛直播SDK发布的RTSP转RTMP推送模块（SmartRelaySDK）C#的界面如下：

技术设计：

1. 拉流：通过RTSP直播播放SDK的数据回调接口，拿到音视频数据；

2. 转推：通过RTMP直播推送SDK的编码后数据输入接口，把回调上来的数据，传给RTMP直播推送模块，实现RTSP数据流到RTMP服务器的转发；

3. 录像：如果需要录像，借助RTSP直播播放SDK，拉到音视频数据后，直接存储MP4文件即可；

4. 快照：如果需要实时快照，拉流后，解码调用播放端快照接口，生成快照，因为快照涉及到video数据解码，如无必要，可不必开启，不然会额外消耗性能。

5. 拉流预览：如需预览拉流数据，只要调用播放端的播放接口，即可实现拉流数据预览；

6. 数据转AAC后转发：考虑到好多监控设备出来的音频可能是PCMA/PCMU的，如需要更通用的音频格式，可以转AAC后，在通过RTMP推送；

7. 转推RTMP实时静音：只需要在传audio数据的地方，加个判断即可；

8. 拉流速度反馈：通过RTSP播放端的实时码率反馈event，拿到实时带宽占用即可；

9. 整体网络状态反馈：考虑到有些摄像头可能会临时或异常关闭，RTMP服务器亦是，可以通过推拉流的event回调状态，查看那整体网络情况，如此界定：是拉不到流，还是推不到RTMP服务器。

上述是C#的基础demo，如果对C++比较熟悉，也可以直接用C++的，大牛直播SDK的RTSP转RTMP推送模块，通过配置xml的形式，程序启动后，从configure.xml读取相关的参数，实现一键拉流转发。

常规的参数配置，比如推拉流的rtsp rtmp url，如果需要自采集audio，设置采集的audio类型，比如rtsp自带audio、麦克风、扬声器或麦克风扬声器混音。

<?xml version="1.0" encoding="utf-8" ?>
<StreamRelays>
  <Relay>
    <id>0</id>
	<AudioOption>4</AudioOption>
    <PullUrl>rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream00</PushUrl>
  </Relay>
  <Relay>
    <id>1</id>
	<AudioOption>1</AudioOption>
    <PullUrl>rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1<![CDATA[&]]>subtype=0</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream01</PushUrl>
  </Relay>
  <Relay>
    <id>2</id>
	<AudioOption>3</AudioOption>
    <PullUrl>rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream02</PushUrl>
  </Relay>
  <Relay>
    <id>3</id>
	<AudioOption>3</AudioOption>
    <PullUrl>rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1<![CDATA[&]]>subtype=0</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream03</PushUrl>
  </Relay>
    <Relay>
    <id>4</id>
	<AudioOption>4</AudioOption>
    <PullUrl>rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream04</PushUrl>
  </Relay>
  <Relay>
    <id>5</id>
	<AudioOption>1</AudioOption>
    <PullUrl>rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1<![CDATA[&]]>subtype=0</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream05</PushUrl>
  </Relay>
  <Relay>
    <id>6</id>
	<AudioOption>4</AudioOption>
    <PullUrl>rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream06</PushUrl>
  </Relay>
  <Relay>
    <id>7</id>
	<AudioOption>2</AudioOption>
    <PullUrl>rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1<![CDATA[&]]>subtype=0</PullUrl>
    <PushUrl>rtmp://192.168.0.103:1935/hls/stream07</PushUrl>
  </Relay>
</StreamRelays>

封装代码如下：

/*
 * nt_relay_wrapper.cs.cs
 * nt_relay_wrapper.cs
 * 
 * WebSite: https://daniusdk.com
 * WeChat: xinsheng120
 * 
 * Created by DaniuLive on 2017/11/14.
 * Copyright © 2014~2024 DaniuLive. All rights reserved.
 */
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
 
namespace SmartRelayDemo
{
    class nt_relay_wrapper
    {
        int relay_index_;
        nt_player_wrapper player_wrapper_;
        nt_publisher_wrapper publisher_wrapper_;
 
        UInt32 video_option_ = (UInt32)NT.NTSmartPublisherDefine.NT_PB_E_VIDEO_OPTION.NT_PB_E_VIDEO_OPTION_ENCODED_DATA;
        UInt32 audio_option_ = (UInt32)NT.NTSmartPublisherDefine.NT_PB_E_AUDIO_OPTION.NT_PB_E_AUDIO_OPTION_ENCODED_DATA;
 
        public nt_player_wrapper GetPlayerWrapper() { return player_wrapper_; }
        public nt_publisher_wrapper GetPublisherWrapper() { return publisher_wrapper_; }
 
        public nt_relay_wrapper(int index, System.Windows.Forms.Control render_wnd, System.ComponentModel.ISynchronizeInvoke sync_invoke)
        {
            relay_index_ = index;
            player_wrapper_ = new nt_player_wrapper(index, render_wnd, sync_invoke);
            publisher_wrapper_ = new nt_publisher_wrapper(index, render_wnd, sync_invoke);
        }
 
        ~nt_relay_wrapper() { }
 
        private void OnVideoDataHandle(IntPtr handle, IntPtr user_data,
            UInt32 video_codec_id, IntPtr data, UInt32 size,
            IntPtr info, IntPtr reserve)
        {
            if (publisher_wrapper_.is_rtmp_publishing())
            {
                publisher_wrapper_.OnVideoDataHandle(handle, user_data, video_codec_id, data, size, info, reserve);
            }
        }
 
        private void OnAudioDataHandle(IntPtr handle, IntPtr user_data,
            UInt32 audio_codec_id, IntPtr data, UInt32 size,
            IntPtr info, IntPtr reserve)
        {
            if (publisher_wrapper_.is_rtmp_publishing())
            {
                publisher_wrapper_.OnAudioDataHandle(handle, user_data, audio_codec_id, data, size, info, reserve);
            }
        }
 
        public void StartPull(String url)
        {
            if (!player_wrapper_.is_pulling())
            {
                player_wrapper_.SetBuffer(0);
 
                if (!player_wrapper_.StartPull(url, false))
                    return;
 
                player_wrapper_.EventOnVideoDataHandle += new nt_player_wrapper.DelOnVideoDataHandle(OnVideoDataHandle);
 
                if (audio_option_ == (UInt32)NT.NTSmartPublisherDefine.NT_PB_E_AUDIO_OPTION.NT_PB_E_AUDIO_OPTION_ENCODED_DATA)
                {
                    player_wrapper_.EventOnAudioDataHandle += new nt_player_wrapper.DelOnAudioDataHandle(OnAudioDataHandle);
                }
            }
        }
 
        public void StopPull()
        {
            player_wrapper_.StopPull();
        }
 
        public void StartPlayer(String url, bool is_rtsp_tcp_mode, bool is_mute)
        {
            player_wrapper_.SetBuffer(0);
 
            if (!player_wrapper_.StartPlay(url, is_rtsp_tcp_mode, is_mute))
                return;
        }
 
        public void StopPlayer()
        {
            player_wrapper_.StopPlay();
        }
 
        public void PlayerDispose()
        {
            player_wrapper_.Dispose();
        }
 
        public void SetPusherOption(UInt32 video_option, UInt32 audio_option)
        {
            video_option_ = video_option;
            audio_option_ = audio_option;
        }
 
        public void StartPublisher(String url)
        {
            if (!publisher_wrapper_.OpenPublisherHandle(video_option_, audio_option_))
                return;
 
            if (url.Length < 8)
            {
                publisher_wrapper_.try_close_handle();
                return;
            }
 
            if (!publisher_wrapper_.StartPublisher(url))
            {
                return;
            }
        }
 
        public void StopPublisher()
        {
            publisher_wrapper_.StopPublisher();
        }
 
        public void PublisherDispose()
        {
            publisher_wrapper_.Dispose();
        }
    }
}

如需播放，播放端封装逻辑实现如下：

/*
 * nt_player_wrapper.cs
 * nt_player_wrapper
 * 
 * WebSite: https://daniusdk.com
 * 
 * Created by DaniuLive on 2017/11/14.
 * Copyright © 2014~2024 DaniuLive. All rights reserved.
 */
 
public bool is_playing() {  return is_playing_; }
 
public bool is_pulling() {  return is_pulling_; }
	
public bool is_recording() { return is_recording_; }
 
public static bool is_zero_ptr(IntPtr ptr) { return IntPtr.Zero == ptr; }
 
public bool is_empty_handle() { return is_zero_ptr(player_handle_); }
 
private bool is_running()
{
	if (is_empty_handle())
		return false;
 
	return is_playing_ || is_recording_ || is_pulling_;
}
 
public bool OpenPullHandle(String url, bool is_rtsp_tcp_mode, bool is_mute)
{
	if ( player_handle_ != IntPtr.Zero )
		return true;
 
	if ( String.IsNullOrEmpty(url) )
		return false;
 
	IntPtr pull_handle = IntPtr.Zero;
 
	if (NTBaseCodeDefine.NT_ERC_OK != NTSmartPlayerSDK.NT_SP_Open(out pull_handle, IntPtr.Zero, 0, IntPtr.Zero))
	{
		return false;
	}
 
	if (pull_handle == IntPtr.Zero)
	{
		return false;
	}
 
	pull_event_call_back_ = new SP_SDKEventCallBack(SDKPullEventCallBack);
	NTSmartPlayerSDK.NT_SP_SetEventCallBack(pull_handle, IntPtr.Zero, pull_event_call_back_);
 
	resolution_notify_callback_ = new ResolutionNotifyCallback(PlaybackWindowResized);
 
	set_video_frame_call_back_ = new VideoFrameCallBack(SDKVideoFrameCallBack);
 
	NTSmartPlayerSDK.NT_SP_SetBuffer(pull_handle, play_buffer_);
	NTSmartPlayerSDK.NT_SP_SetFastStartup(pull_handle, 1);
	NTSmartPlayerSDK.NT_SP_SetRtspAutoSwitchTcpUdp(pull_handle, 1);
	NTSmartPlayerSDK.NT_SP_SetRTSPTcpMode(pull_handle, is_rtsp_tcp_mode ? 1 : 0);
 
	NTSmartPlayerSDK.NT_SP_SetMute(pull_handle, is_mute_ ? 1 : 0);
 
	NTSmartPlayerSDK.NT_SP_SetAudioVolume(pull_handle, cur_audio_volume_);
 
	Int32 is_report = 1;
	Int32 report_interval = 3;
	NTSmartPlayerSDK.NT_SP_SetReportDownloadSpeed(pull_handle, is_report, report_interval);
 
	//RTSP timeout设置
	Int32 rtsp_timeout = 10;
	NTSmartPlayerSDK.NT_SP_SetRtspTimeout(pull_handle, rtsp_timeout);
 
	//RTSP TCP/UDP自动切换设置
	Int32 is_auto_switch_tcp_udp = 1;
	NTSmartPlayerSDK.NT_SP_SetRtspAutoSwitchTcpUdp(pull_handle, is_auto_switch_tcp_udp);
 
	if (NTBaseCodeDefine.NT_ERC_OK != NTSmartPlayerSDK.NT_SP_SetURL(pull_handle, url))
	{
		NTSmartPlayerSDK.NT_SP_Close(pull_handle);
		pull_handle = IntPtr.Zero;
		return false;
	}
 
	player_handle_ = pull_handle;
 
	return true;
}
 
private void PlaybackWindowResized(Int32 width, Int32 height)
{
	String resolution = width + "*" + height;
	EventGetVideoSize(player_index_, resolution);
}
 
public void SP_SDKVideoSizeHandle(IntPtr handle, IntPtr userData, Int32 width, Int32 height)
{
	if (null == sync_invoke_)
		return;
 
	System.ComponentModel.ISynchronizeInvoke sync_invoke_target = sync_invoke_.Target as System.ComponentModel.ISynchronizeInvoke;
 
	if (sync_invoke_target != null)
	{
		if (sync_invoke_target.InvokeRequired)
		{
			sync_invoke_target.BeginInvoke(resolution_notify_callback_, new object[] { width, height });
		}
		else
		{
			resolution_notify_callback_(width, height);
		}
	}
 
}
 
public bool StartPlay(String url, bool is_rtsp_tcp_mode, bool is_mute)
{
	if ( is_playing_ )
		return false;
 
	if (!is_pulling() && !is_recording())
	{
		if (!OpenPullHandle(url, is_rtsp_tcp_mode, is_mute))
			return false;
	}
 
	//video resolution callback
	video_size_call_back_ = new SP_SDKVideoSizeCallBack(SP_SDKVideoSizeHandle);
	NTSmartPlayerSDK.NT_SP_SetVideoSizeCallBack(player_handle_, IntPtr.Zero, video_size_call_back_);
 
	bool is_support_d3d_render = false;
	Int32 in_support_d3d_render = 0;
 
	if (NT.NTBaseCodeDefine.NT_ERC_OK == NTSmartPlayerSDK.NT_SP_IsSupportD3DRender(player_handle_, render_wnd_.Handle, ref in_support_d3d_render))
	{
		if (1 == in_support_d3d_render)
		{
			is_support_d3d_render = true;
		}
	}
 
	// is_support_d3d_render = false;
 
	if (is_support_d3d_render)
	{
		// 支持d3d绘制的话，就用D3D绘制
		NTSmartPlayerSDK.NT_SP_SetRenderWindow(player_handle_, render_wnd_.Handle);
		NTSmartPlayerSDK.NT_SP_SetRenderScaleMode(player_handle_, 1);
	}
	else
	{
		// 不支持D3D就让播放器吐出数据来，用GDI绘制，本demo仅用来展示一对一互动使用，具体可参考播放端的demo
 
		//video frame callback (YUV/RGB)
		//format请参见 NT_SP_E_VIDEO_FRAME_FORMAT，如需回调YUV，请设置为 NT_SP_E_VIDEO_FRAME_FROMAT_I420
		video_frame_call_back_ = new SP_SDKVideoFrameCallBack(SetVideoFrameCallBack);
		NTSmartPlayerSDK.NT_SP_SetVideoFrameCallBack(player_handle_, (Int32)NT.NTSmartPlayerDefine.NT_SP_E_VIDEO_FRAME_FORMAT.NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, IntPtr.Zero, video_frame_call_back_);
	}
 
	uint ret = NTSmartPlayerSDK.NT_SP_StartPlay(player_handle_);
	if ( NTBaseCodeDefine.NT_ERC_OK != ret )
	{
		NTSmartPlayerSDK.NT_SP_Close(player_handle_);
		player_handle_ = IntPtr.Zero;
 
		return false;
	}
 
	is_playing_ = true;
 
	return true;
}
 
public void StopPlay(bool is_update_ui =true)
{
	if ( !is_playing_ )
		return;
 
	NTSmartPlayerSDK.NT_SP_StopPlay(player_handle_);
 
	if (!is_pulling() && !is_recording())
	{
		NTSmartPlayerSDK.NT_SP_Close(player_handle_);
		player_handle_ = IntPtr.Zero;
	}
 
	is_playing_ = false;
 
	if (is_update_ui && render_wnd_ != null)
	{
		render_wnd_.Invalidate();
	}
}
 
public bool StartPull(String url, bool is_rtsp_tcp_mode)
{
	if (is_pulling())
		return false;
 
	if (!is_playing() && !is_recording())
	{
		if (!OpenPullHandle(url, is_rtsp_tcp_mode, is_mute_))
			return false;
	}
 
	pull_stream_video_data_call_back_ = new SP_SDKPullStreamVideoDataCallBack(OnVideoDataHandle);
	pull_stream_audio_data_call_back_ = new SP_SDKPullStreamAudioDataCallBack(OnAudioDataHandle);
 
	NTSmartPlayerSDK.NT_SP_SetPullStreamVideoDataCallBack(player_handle_, IntPtr.Zero, pull_stream_video_data_call_back_);
	NTSmartPlayerSDK.NT_SP_SetPullStreamAudioDataCallBack(player_handle_, IntPtr.Zero, pull_stream_audio_data_call_back_);
 
	int is_transcode_aac = 1;   //PCMA/PCMU/Speex格式转AAC后 再转发
	NTSmartPlayerSDK.NT_SP_SetPullStreamAudioTranscodeAAC(player_handle_, is_transcode_aac);
 
	UInt32 ret = NTSmartPlayerSDK.NT_SP_StartPullStream(player_handle_);
 
	if (NTBaseCodeDefine.NT_ERC_OK != ret)
	{
		if (!is_playing_)
		{
			NTSmartPlayerSDK.NT_SP_Close(player_handle_);
			player_handle_ = IntPtr.Zero;
		}
 
		return false;
	}
 
	is_pulling_ = true;
 
	return true;
}
 
public void StopPull()
{
	if (!is_pulling_)
		return;
 
	NTSmartPlayerSDK.NT_SP_StopPullStream(player_handle_);
 
	if (!is_playing() && !is_recording())
	{
		NTSmartPlayerSDK.NT_SP_Close(player_handle_);
		player_handle_ = IntPtr.Zero;
	}
 
	is_pulling_ = false;
}
 
private void OnVideoDataHandle(IntPtr handle, IntPtr user_data,
	UInt32 video_codec_id, IntPtr data, UInt32 size,
	IntPtr info, IntPtr reserve)
{
	EventOnVideoDataHandle(handle, user_data, video_codec_id, data, size, info, reserve);
}
 
private void OnAudioDataHandle(IntPtr handle, IntPtr user_data,
					UInt32 audio_codec_id, IntPtr data, UInt32 size,
					IntPtr info, IntPtr reserve)
{
	EventOnAudioDataHandle(handle, user_data, audio_codec_id, data, size, info, reserve);
}