腾讯云FLV AV1扩展以及FFmpeg修改说明

腾讯云音视频

修改于 2020-01-22 11:34:22

2.4K0

修改于 2020-01-22 11:34:22

文章被收录于专栏：音视频咖

腾讯云对FLV协议部分做了扩展，用于支持FLV AV1的直播场景，并在FFmpeg 4.1.3/4.2中做了对应的实现，具体细节如下：

针对最新的FLV标准，VIDEODATA部分扩展如下：

参考文档：

https://www.adobe.com/content/dam/acom/en/devnet/flv/video_file_format_spec_v10_1.pdf

VIDEODATA

The VideoTagHeader and VideoTagBody contains video-specific metadata/the video frame payload.

VideoTagHeader & VideoTagBody

Field

Type

Comment

Frame Type

UB [4]

Type of video frame. The following values are defined:

1 = key frame (for AVC and HEVC, a seekable frame)

2 = inter frame (for AVC and HEVC, a non-seekable frame)

3 = disposable inter frame (H.263 only)

4 = generated key frame (reserved for server use only)

5 = video info/command frame

CodecID

UB [4]

The following values are defined:

2 = Sorenson H.263

3 = Screen video

4 = On2 VP6

5 = On2 VP6 with alpha channel

6 = Screen video version 2

7 = AVC

13 = AV1

AV1PacketType

IF CodecID == 13

UI8

The following values are defined:

0 = Sequence Header

1 = OBU Frame(Full frames are required)

CompositionTime

IF CodecID==7 OR CodecID == 13

SI24

IF AVCPacketType == 1 OR AV1PacketType == 1

Composition time offset

ELSE

See ISO 14496-12, 8.15.3 for an explanation of composition

times. The offset in an FLV file is always in milliseconds.

VideoTagBody

IF FrameType == 5

UI8

ELSE (

IF CodecID == 2

H263VIDEOPACKET

IF CodecID == 3

SCREENVIDEOPACKET

IF CodecID == 4

VP6FLVVIDEOPACKET

IF CodecID == 5

VP6FLVALPHAVIDEOPACKET

IF CodecID == 6

SCREENV2VIDEOPACKET

IF CodecID == 7

AVCVIDEOPACKET

IF CodecID == 13 AV1VIDEOPACKET )

Video frame payload or frame info

If FrameType == 5, instead of a video payload, the

Video Data Body contains a UI8 with the following

meaning:

0 = Start of client-side seeking video frame

sequence

1 = End of client-side seeking video frame

sequence

For all but AVCVIDEOPACKET or AV1VIDEOPACKET,

see the SWF File Format Specification for details

See AV1 Codec ISO Media File Format Binding v1.2.0, 12 December 2019 for the description of AV1CodecConfigurationBox. This contains the same information that would be stored in av1C box in an MP4/FLV file.

AV1CodecConfigurationBox defined in AV1 Codec ISO Media File Format Binding v1.2.0, 12 December 2019 as follow:

参考文档：https://aomediacodec.github.io/av1-isobmff/

class AV1CodecConfigurationBox extends Box('av1C'){

AV1CodecConfigurationRecord av1Config; } aligned (8) class AV1CodecConfigurationRecord { unsigned int (1) marker = 1; unsigned int (7) version = 1; unsigned int (3) seq_profile; unsigned int (5) seq_level_idx_0; unsigned int (1) seq_tier_0; unsigned int (1) high_bitdepth; unsigned int (1) twelve_bit; unsigned int (1) monochrome; unsigned int (1) chroma_subsampling_x; unsigned int (1) chroma_subsampling_y; unsigned int (2) chroma_sample_position; unsigned int (3) reserved = 0; unsigned int (1) initial_presentation_delay_present; if (initial_presentation_delay_present) { unsigned int (4) initial_presentation_delay_minus_one; } else { unsigned int (4) reserved = 0; } unsigned int (8)[] configOBUs; }

关于FLV/AV1扩展patch的使用说明

获取patch并应用

· 获取FFmpeg 代码:

git clone https://github.com/FFmpeg.git FFmpeg

· 按照如下应用patch:

cd FFmpeg

git reset --hard origin/release/4.2 git am 0001-The-FLV-extensions-for-AV1.patch

以上即应用腾讯云关于AV1 FLV的扩展patch到FFmpeg 4.2

patch说明

涉及改动以下文件：

libavcodec/av1_parse.c
libavcodec/av1_parse.h
libavcodec/cbs_av1.c
libavcodec/cbs_av1.h
libavform/flv.h
libavform/flvenc.c
libavform/flvdec.c

在关于H.264的分支判断处，加入了AV1判断，并支持了av1C的解析。具体协议细节说明请见上面的文档部分。下图是一个FLV AV1扩展示例：

由于目前FFmpeg 并无原生的AV1 decoder（需要依赖dav1d 或者libaom），在解析extradata部分有所缺失，为了支持FLV AV1， IVF AV1 等格式的转换，我们也增加就AV1 extradata部分的支持。在此基础上，推拉流的基本流程与原FLV h264/h265并未特别差异，如下所示：