前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >音视频协议-RTP协议

音视频协议-RTP协议

作者头像
全栈程序员站长
发布2022-09-13 11:53:23
7950
发布2022-09-13 11:53:23
举报
文章被收录于专栏:全栈程序员必看

大家好,又见面了,我是你们的朋友全栈君。

1 协议简介

视频传输的基石:RTP和RTCP。对于协议的讲解主要是是对于RFC文档的阅读和理解。不同的使用场景用到的字段也有所侧重,RTP和RTCP定义在RFC3550中。其中RTP用于数据流的传输;RTCP用于数据流的控制。可以说rtp/rtcp协议是即时通讯不可或缺的组成。RTCP协议介绍见:音视频协议-RTCP协议介绍

2 协议格式介绍

rtp协议定义在rfc3550第5.1章RTP头定义

版本号(2bit):默认为2; 填充标志(1bit):当设置为1时,最后一个字节表示填充字节数包括该字节本身,这些填充不属于荷载,解析时需要被忽略; 扩展标志(1bit):当设置为1时,rtp头后面会接一个扩展头需要解析,需要注意的是length长度是32bit为单位计算的,也就是4字节加1;

CSRC计数(4bit):CSRC 个数最多就是15个; 标志位M(1bit):视频编码表示一帧的结束标志; 荷载类型(7bit):具体见RFC3551,0-95已经被定义,动态协商采用96-127;

序列号(16bit):序列号为2字节,只能在0-65535之间不断循环; 时间戳(32bit):初始值为随机值,根据采样步长递增,主要用于音视频同步; 同步源(32bit):随机值,同一个会话源相同; 贡献源(32bit):贡献源主要用于混合器产生数据。

3 协议解析

这里选用目前业界比较认可的JRTPLIB库进行讲解。

3.1 协议头定义

结构体设计需要考虑设备的大小端问题,大端和小的差别主要是单个字节的内部顺序,大端与协议顺序一致,小端则是相反的。定义结构体与协议一致有一个好处就是可以接收到数据后直接进行强转得到对应的rtp字段。

代码语言:javascript
复制
struct RTPHeader
{ 
   
#ifdef RTP_BIG_ENDIAN
	uint8_t version:2;	//版本
	uint8_t padding:1;	//填充
	uint8_t extension:1;//扩展
	uint8_t csrccount:4;//csrc count
	
	uint8_t marker:1; 	//标志
	uint8_t payloadtype:7;//荷载类型
#else // little endian
	uint8_t csrccount:4;
	uint8_t extension:1;
	uint8_t padding:1;
	uint8_t version:2;
	
	uint8_t payloadtype:7;
	uint8_t marker:1;
#endif // RTP_BIG_ENDIAN
	
	uint16_t sequencenumber;//序列号
	uint32_t timestamp; 	//时间戳
	uint32_t ssrc;			//同步源
};

扩展头包含两个字段:扩展id和长度

代码语言:javascript
复制
struct RTPExtensionHeader
{ 
   
	uint16_t extid;
	uint16_t length;
};

3.2 RTP协议解析

协议解析核心包含几个步骤:

  1. 利用rtp定义的头进行数据的强转,得到rtp头部信息;
  2. 跳过rtp协议头,这里需要注意没有定义cssrc所以需要利用cc计算csrc个数
  3. 填充处理,获取填充字节数
  4. 扩展头处理,这里需要注意的是extlen是32位长度的个数
  5. 计算荷载数据长度
  6. 赋值到RTPPacket中,需要主要网络字节序转换
代码语言:javascript
复制
int RTPPacket::ParseRawPacket(RTPRawPacket &rawpack)
{ 
   
	uint8_t *packetbytes;
	size_t packetlen;
	uint8_t payloadtype;
	RTPHeader *rtpheader;
	bool marker;
	int csrccount;
	bool hasextension;
	int payloadoffset,payloadlength;
	int numpadbytes;
	RTPExtensionHeader *rtpextheader;
	
	if (!rawpack.IsRTP()) // If we didn't receive it on the RTP port, we'll ignore it
		return ERR_RTP_PACKET_INVALIDPACKET;
	
	// The length should be at least the size of the RTP header
	packetlen = rawpack.GetDataLength();
	if (packetlen < sizeof(RTPHeader))
		return ERR_RTP_PACKET_INVALIDPACKET;
	
	packetbytes = (uint8_t *)rawpack.GetData();
	//1 利用rtp定义的头进行数据的强转,得到rtp头部信息
	rtpheader = (RTPHeader *)packetbytes;
	
	// The version number should be correct
	if (rtpheader->version != RTP_VERSION)
		return ERR_RTP_PACKET_INVALIDPACKET;
	
	// We'll check if this is possibly a RTCP packet. For this to be possible
	// the marker bit and payload type combined should be either an SR or RR
	// identifier
	marker = (rtpheader->marker == 0)?false:true;
	payloadtype = rtpheader->payloadtype;
	if (marker)
	{ 
   
		if (payloadtype == (RTP_RTCPTYPE_SR & 127)) // don't check high bit (this was the marker!!)
			return ERR_RTP_PACKET_INVALIDPACKET;
		if (payloadtype == (RTP_RTCPTYPE_RR & 127))
			return ERR_RTP_PACKET_INVALIDPACKET;
	}

	csrccount = rtpheader->csrccount;
	//2 跳过rtp协议头,这里需要注意没有定义cssrc所以需要利用cc计算csrc个数
	payloadoffset = sizeof(RTPHeader)+(int)(csrccount*sizeof(uint32_t));
	//3 填充处理,获取填充字节数
	if (rtpheader->padding) // adjust payload length to take padding into account
	{ 
   
		numpadbytes = (int)packetbytes[packetlen-1]; // last byte contains number of padding bytes
		if (numpadbytes <= 0)
			return ERR_RTP_PACKET_INVALIDPACKET;
	}
	else
		numpadbytes = 0;
	//4 扩展头处理,这里需要注意的是extlen是32位长度的个数
	hasextension = (rtpheader->extension == 0)?false:true;
	if (hasextension) // got header extension
	{ 
   
		rtpextheader = (RTPExtensionHeader *)(packetbytes+payloadoffset);
		payloadoffset += sizeof(RTPExtensionHeader);
		
		uint16_t exthdrlen = ntohs(rtpextheader->length);
		payloadoffset += ((int)exthdrlen)*sizeof(uint32_t);
	}
	else
	{ 
   
		rtpextheader = 0;
	}	
	//5 计算荷载数据长度
	payloadlength = packetlen-numpadbytes-payloadoffset;
	if (payloadlength < 0)
		return ERR_RTP_PACKET_INVALIDPACKET;

	// Now, we've got a valid packet, so we can create a new instance of RTPPacket
	// and fill in the members
	//6 赋值到RTPPacket中
	RTPPacket::hasextension = hasextension;
	if (hasextension)
	{ 
   
		RTPPacket::extid = ntohs(rtpextheader->extid);
		RTPPacket::extensionlength = ((int)ntohs(rtpextheader->length))*sizeof(uint32_t);
		RTPPacket::extension = ((uint8_t *)rtpextheader)+sizeof(RTPExtensionHeader);
	}

	RTPPacket::hasmarker = marker;
	RTPPacket::numcsrcs = csrccount;
	RTPPacket::payloadtype = payloadtype;
	
	// Note: we don't fill in the EXTENDED sequence number here, since we
	// don't have information about the source here. We just fill in the low
	// 16 bits
	RTPPacket::extseqnr = (uint32_t)ntohs(rtpheader->sequencenumber);

	RTPPacket::timestamp = ntohl(rtpheader->timestamp);
	RTPPacket::ssrc = ntohl(rtpheader->ssrc);
	RTPPacket::packet = packetbytes;
	RTPPacket::payload = packetbytes+payloadoffset;
	RTPPacket::packetlength = packetlen;
	RTPPacket::payloadlength = payloadlength;

	// We'll zero the data of the raw packet, since we're using it here now!
	rawpack.ZeroData();

	return 0;
}

3.3 RTP包构建

RTP包构建比较简单,就是一个简单的填空题,先计算出包的总长度,然后分配好包的大小,最后将内存强转成rtp头,然后填空即可。需要注意主机字节序转网络字节序问题。

代码语言:javascript
复制
int RTPPacket::BuildPacket(uint8_t payloadtype,const void *payloaddata,size_t payloadlen,uint16_t seqnr,
		  uint32_t timestamp,uint32_t ssrc,bool gotmarker,uint8_t numcsrcs,const uint32_t *csrcs,
		  bool gotextension,uint16_t extensionid,uint16_t extensionlen_numwords,const void *extensiondata,
		  void *buffer,size_t maxsize)
{ 
   
	//cc大小校验
	if (numcsrcs > RTP_MAXCSRCS)
		return ERR_RTP_PACKET_TOOMANYCSRCS;
	//payloadtype校验
	if (payloadtype > 127) // high bit should not be used
		return ERR_RTP_PACKET_BADPAYLOADTYPE;
	if (payloadtype == 72 || payloadtype == 73) // could cause confusion with rtcp types
		return ERR_RTP_PACKET_BADPAYLOADTYPE;
	//rtp包长度计算
	packetlength = sizeof(RTPHeader);
	packetlength += sizeof(uint32_t)*((size_t)numcsrcs);
	if (gotextension)
	{ 
   
		packetlength += sizeof(RTPExtensionHeader);
		packetlength += sizeof(uint32_t)*((size_t)extensionlen_numwords);
	}
	packetlength += payloadlen;

	if (maxsize > 0 && packetlength > maxsize)
	{ 
   
		packetlength = 0;
		return ERR_RTP_PACKET_DATAEXCEEDSMAXSIZE;
	}

	// Ok, now we'll just fill in...
	//RTP包内存分配
	RTPHeader *rtphdr;
	
	if (buffer == 0)
	{ 
   
		packet = RTPNew(GetMemoryManager(),RTPMEM_TYPE_BUFFER_RTPPACKET) uint8_t [packetlength];
		if (packet == 0)
		{ 
   
			packetlength = 0;
			return ERR_RTP_OUTOFMEM;
		}
		externalbuffer = false;
	}
	else
	{ 
   
		packet = (uint8_t *)buffer;
		externalbuffer = true;
	}
	//rtp包赋值
	RTPPacket::hasmarker = gotmarker;
	RTPPacket::hasextension = gotextension;
	RTPPacket::numcsrcs = numcsrcs;
	RTPPacket::payloadtype = payloadtype;
	RTPPacket::extseqnr = (uint32_t)seqnr;
	RTPPacket::timestamp = timestamp;
	RTPPacket::ssrc = ssrc;
	RTPPacket::payloadlength = payloadlen;
	RTPPacket::extid = extensionid;
	RTPPacket::extensionlength = ((size_t)extensionlen_numwords)*sizeof(uint32_t);
	
	rtphdr = (RTPHeader *)packet;
	rtphdr->version = RTP_VERSION;
	rtphdr->padding = 0;
	if (gotmarker)
		rtphdr->marker = 1;
	else
		rtphdr->marker = 0;
	if (gotextension)
		rtphdr->extension = 1;
	else
		rtphdr->extension = 0;
	rtphdr->csrccount = numcsrcs;
	rtphdr->payloadtype = payloadtype&127; // make sure high bit isn't set
	rtphdr->sequencenumber = htons(seqnr);
	rtphdr->timestamp = htonl(timestamp);
	rtphdr->ssrc = htonl(ssrc);
	
	uint32_t *curcsrc;
	int i;

	curcsrc = (uint32_t *)(packet+sizeof(RTPHeader));
	for (i = 0 ; i < numcsrcs ; i++,curcsrc++)
		*curcsrc = htonl(csrcs[i]);

	payload = packet+sizeof(RTPHeader)+((size_t)numcsrcs)*sizeof(uint32_t); 
	if (gotextension)
	{ 
   
		RTPExtensionHeader *rtpexthdr = (RTPExtensionHeader *)payload;

		rtpexthdr->extid = htons(extensionid);
		rtpexthdr->length = htons((uint16_t)extensionlen_numwords);
		
		payload += sizeof(RTPExtensionHeader);
		memcpy(payload,extensiondata,RTPPacket::extensionlength);
		
		payload += RTPPacket::extensionlength;
	}
	memcpy(payload,payloaddata,payloadlen);
	return 0;
}

3.4 RTP发送流程

下面是整个发送过程的调用栈,整体比较简单,就是将需要发送的数据,加入到rtp构建器中构建一个rtp包,然后调用网络管理器发送数据,具体调用时序图图如下:

3.5 RTP接收流程

rtp接收流程在rtppoll线程内完成,主要是接收rtp包加入到rtppcket类利用rtp解析函数进行数据解析得到rtp数据,然后为应用层所用。

发布者:全栈程序员栈长,转载请注明出处:https://javaforall.cn/153987.html原文链接:https://javaforall.cn

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 1 协议简介
  • 2 协议格式介绍
  • 3 协议解析
    • 3.1 协议头定义
      • 3.2 RTP协议解析
        • 3.3 RTP包构建
          • 3.4 RTP发送流程
            • 3.5 RTP接收流程
            相关产品与服务
            即时通信 IM
            即时通信 IM(Instant Messaging)基于腾讯二十余年的 IM 技术积累,支持 Android、iOS、Mac、Windows、Web、H5、小程序平台且跨终端互通,低代码 UI 组件助您30分钟集成单聊、群聊、好友与资料、消息漫游、群组管理、会话管理、直播弹幕、内容审核和推送等能力。适用于直播互动、电商带货、客服咨询、社交沟通、企业办公、互动游戏、医疗健康等场景。
            领券
            问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档