专栏首页hank【分享】以H.265 1080p 运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display

【分享】以H.265 1080p 运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display

2020-07-172020-07-17 09:39:18阅读 1940

运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display，可以测试HDMI输入输出，和VCU的低延时编码。Xilinx wiki的文章MPSoC VCU TRD 2019.2 - Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display以H.264和4K分辨率为例。下面记录H.265和1080p分辨率的运行命令。

1. 设置HDMI输入分辨率

首先设置HDMI输入，保证输入分辨率是1080p。我使用电脑Dell 5580作为ZCU106单板的HDMI输入源。电脑Dell 5580上看到两个显示器，一个是真实显示器，一个是ZCU106单板的HDMI输入。虽然我在电脑Dell 5580设置ZCU106单板的HDMI输入为1080p，但是在ZCU106单板使用命令“xmedia-ctl -p -d /dev/media1”查看，HDMI输入的分辨率一直是4K。后来把电脑Dell 5580的输出模式从扩展(Extend)改为复制(Duplicate)后，在ZCU106单板才看到HDMI输入的分辨率是1080p。这一点很莫名其妙。

错误的分辨率信息

root@vcu_llp2_xv20:~#  xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:RBG888_1X24/3840x2160 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:    progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 3840x2160p30 (4400x2250) stds:CEA-861 flags:CE-video]
                -> "a0040000.v_proc_ss":0 [ENABLED]

正确的分辨率信息

root@vcu_llp2_xv20:~#  xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:    progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0040000.v_proc_ss":0 [ENABLED]

2. 设置VPSS分辨率和格式

接下来设置ZCU106的VPSS的分辨率和格式。设置后，再使用命令“xmedia-ctl -p -d /dev/media1”查看，所有pad的分辨率信息都是1080p了。

root@vcu_llp2_xv20:~#  xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":0 [fmt:RBG888_1X24/1920x1080 field:none]"

root@vcu_llp2_xv20:~#  xmedia-ctl -d /dev/media1 -V "\"a0040000.v_proc_ss\":1 [fmt:UYVY10_1X20/1920x1080 field:none]"

root@vcu_llp2_xv20:~#  xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:RBG888_1X24/1920x1080 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:UYVY10_1X20/1920x1080 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0040000.v_proc_ss":0 [ENABLED]

3. 设置HDMI输出分辨率

使用命令“modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24”，设置设置HDMI输出分辨率。设置时，会产生异常，可以暂时忽略。设置后，HDMI显示器会显示蓝屏。

root@vcu_llp2_xv20:~# modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24
setting mode 1920x1080-30Hz@BG24 on connectors 35, crtc 33
[  254.364908] ------------[ cut here ]------------
[  254.369527] driver forgot to call drm_crtc_vblank_off()
[  254.374783] WARNING: CPU: 2 PID: 2878 at drivers/gpu/drm/drm_atomic_helper.c:1000 drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[  254.386941] Modules linked in: dmaproxy(O) al5d(O) al5e(O) allegro(O) xlnx_vcu xlnx_vcu_clk xilinx_hdmi_tx(O) xilinx_hdmi_rx(O) xlnx_vcu_core xilinx_vphy(O) dp159(O) mali(O) uio_pdrv_genirq
[  254.403812] CPU: 2 PID: 2878 Comm: modetest Tainted: G           O      4.19.0-xilinx-v2019.2 #1
[  254.412584] Hardware name: ZynqMP ZCU106 RevA (DT)
[  254.417361] pstate: 60000005 (nZCv daif -PAN -UAO)
[  254.422135] pc : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[  254.428646] lr : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[  254.435154] sp : ffffff8014feba00
[  254.438453] x29: ffffff8014feba00 x28: ffffffc87b2d2400
[  254.443757] x27: ffffffc85fc14800 x26: 0000000000000000
[  254.449061] x25: 0000000000000038 x24: ffffff80091c725b
[  254.454365] x23: ffffff8008d74e98 x22: 0000000000000000
[  254.459669] x21: ffffffc85f29c800 x20: ffffffc85fc53890
[  254.464973] x19: ffffffc87ae49980 x18: 0000000000000010
[  254.470277] x17: 0000000000000000 x16: 0000000000000000
[  254.475581] x15: ffffffffffffffff x14: ffffff8009148648
[  254.480885] x13: ffffff80891e7f0f x12: ffffff80091e7f17
[  254.486189] x11: ffffff800915a000 x10: ffffff8014feb6e0
[  254.491493] x9 : 00000000ffffffd0 x8 : ffffff8008543da8
[  254.496797] x7 : 6c6163206f742074 x6 : 00000000000001d7
[  254.502101] x5 : 0000000000000003 x4 : 0000000000000000
[  254.507396] x3 : 0000000000000000 x2 : ffffffffffffffff
[  254.512700] x1 : 6b059e08b5ce2000 x0 : 0000000000000000
[  254.518004] Call trace:
[  254.520438]  drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[  254.526609]  drm_atomic_helper_commit_tail+0x20/0x78
[  254.531556]  commit_tail+0x74/0x78
[  254.534942]  drm_atomic_helper_commit+0xc8/0x140
[  254.539544]  drm_atomic_commit+0x48/0x58
[  254.543458]  drm_atomic_helper_set_config+0xa0/0xb0
[  254.548321]  drm_mode_setcrtc+0x144/0x5d8
[  254.552322]  drm_ioctl_kernel+0xb4/0x100
[  254.556228]  drm_ioctl+0x204/0x3a8
[  254.559616]  do_vfs_ioctl+0xb8/0x8a0
[  254.563182]  ksys_ioctl+0x44/0x90
[  254.566481]  __arm64_sys_ioctl+0x1c/0x28
[  254.570388]  el0_svc_common+0x84/0xd8
[  254.574041]  el0_svc_handler+0x68/0x80
[  254.577774]  el0_svc+0x8/0xc
[  254.580637] ---[ end trace 44a2c3f75cacf5a7 ]---

4. 本板运行

复制文件/media/card/config/input.cfg为/media/card/config/input-1080p.cfg，修改其中的分辨率为1080p。再以命令"vcu_gst_app /media/card/config/input-1080p.cfg"运行，可以看到本板的HDMI输入显示到了HDMI输出上。

root@vcu_llp2_xv20:/media/card/config# cat input-1080p.cfg
Common Configuration    : START
Num Of Input            : 1
Output                  : HDMI
Out Type                : Display
Frame Rate              : 60
Exit


Input Configuration     : START
Input Num               : 1
Input Type              : hdmi_1
Raw                     : FALSE
Width                   : 1920
Height                  : 1080
Format                  : XV20
Enable LLP2             : FALSE
Exit


Encoder Configuration   : START
Encoder Num             : 1
Encoder Name            : AVC
Profile                 : High
Rate Control            : Low_Latency
Filler Data             : False
QP                      : Auto
L2 Cache                : TRUE
Latency Mode            : Sub_Frame
Low Bandwidth           : FALSE
Gop Mode                : Basic
Bitrate                 : 25000
B Frames                : 0
Slice                   : 8
GoP Length              : 60
Preset                  : Custom
Exit


Streaming Configuration : START
Streaming Num           : 1
Host IP                 : 192.168.25.89
Port                    : 5004
Exit


Trace Configuration     : START
FPS Info                : TRUE
APM Info                : TRUE
Pipeline Info           : TRUE
Exit

root@vcu_llp2_xv20:/media/card/config# vcu_gst_app /media/card/config/input-1080p.cfg
/*************************Pipeline Information Start*************************/
Pipeline Info : On
Fps Info : On
APM Info : On
Output goes on : HDMI
Frame rate : 60
Number of Source is : 1
Use case is to : Display
B Frames : 0
Bitrate : 25000
Enable L2Cache : True
Enc Name : AVC
Gop Len : 60
Profile : High
Qp Mode : Auto
Rc Mode : Low_Latency
Num Slice : 8
GoP Mode : Basic
Filler Data : False
Low Bandwidth : False
Latency Mode : Sub_Frame
Device Type : HDMI
Format : XV20
Width : 1920
Height : 1080
Raw : False
Accelerator flag : False
Enable_scd flag : False
Enable_roi flag : False
Enable_llp2 flag : False
Src Type : Live Src
/*************************Pipeline Information End*************************/
!! The specified Level is too low and will be adjusted !!
Encoder Memory Bandwidth (1.04 Gbps)
Decoder Memory Bandwidth (0.00 Gbps)
Pipeline [1] Fps[0]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.99 Gbps)
Pipeline [1] Fps[20]
Encoder Memory Bandwidth (8.78 Gbps)
Decoder Memory Bandwidth (7.12 Gbps)
Pipeline [1] Fps[22]

Encoder Memory Bandwidth (8.76 Gbps)
Decoder Memory Bandwidth (7.18 Gbps)
Pipeline [1] Fps[22]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.17 Gbps)
Pipeline [1] Fps[22]
^CHit Ctrl-C
Quitting the app now
Quitting the loop
playback count 1
pipeline stopped successfully
root@vcu_llp2_xv20:/media/card/config# ^C

5. 网络运行

5.1. 设置网络

在HDMI输入单板上执行命令“ifconfig eth0 192.168.25.10 up”，配置HDMI输入单板的IP为192.168.25.10。在HDMI输出单板上执行命令“ifconfig eth0 192.168.25.89 up”，配置HDMI输出单板的IP为192.168.25.89。再执行命令，“ping 192.168.25.10”，检查网络是否通畅。

5.2. 发送视频流

root@vcu_llp2_xv20:~#  ifconfig eth0 192.168.25.10 up

root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60  b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:21.804704005
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw\(memory:XLNXLL\), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60  b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!

5.3. 接收视频流

root@vcu_llp2_xv20:~#  ifconfig eth0 192.168.25.89 up
root@vcu_llp2_xv20:~#
root@vcu_llp2_xv20:~# ping 192.168.25.10
PING 192.168.25.10 (192.168.25.10): 56 data bytes
64 bytes from 192.168.25.10: seq=0 ttl=64 time=0.229 ms
64 bytes from 192.168.25.10: seq=1 ttl=64 time=0.071 ms
^C
--- 192.168.25.10 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.071/0.150/0.229 ms

root@vcu_llp2_xv20:~#  gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw\(memory:XLNXLL\) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true -v
Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-height = 1080
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:1:7:1, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 14, dropped: 15, fps: 27.85, drop rate: 29.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 29, dropped: 23, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 45, dropped: 31, fps: 30.00, drop rate: 15.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 60, dropped: 39, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 75, dropped: 44, fps: 29.99, drop rate: 10.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 90, dropped: 49, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 105, dropped: 54, fps: 29.98, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 120, dropped: 59, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 136, dropped: 64, fps: 30.09, drop rate: 9.40
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 150, dropped: 70, fps: 26.25, drop rate: 11.25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 162, dropped: 76, fps: 23.85, drop rate: 11.92

6. 技巧

Xilinx wiki的文章的命令中，大约每秒发送一个I帧，造成视频闪烁。如果去掉“periodicity-idr=60”，就不会有视频闪烁了。

以网络运行demo时，先执行接收端Gstreamer命令，再执行发送端Gstreamer命令。如果顺序反了，接收端收不到包，不能显示图像。

7. 已知问题

使用Yavta不能成功捕获视频帧。

root@vcu_llp2_xv20:~#  xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:RBG888_1X24/1920x1080 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:UYVY10_1X20/1920x1080 field:none]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0040000.v_proc_ss":0 [ENABLED]

root@vcu_llp2_xv20:~#  yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63898.159475] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
 device.
Video format set: UYVY (59565955) 1920x1080 field none[63898.173868] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
, 1 planes:
 * Stride 3840, buffer size 4147200
Video format:[63898.188189] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
 UYVY (59565955) 1920x1080 field none, 1 planes:
 * Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fa84b0000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fa80bb000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fa7cc6000.
Unable to start streaming: Invalid argument (22).
3 buffers released.

root@vcu_llp2_xv20:~#  yavta -n 3 -c10 -f YUYV -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
Video format set: YUYV (56595559) 1920x1080 field none, 1 planes:
 * Stride 3840, buffer size 4147200
Video format: YUYV (56595559) 1920x1080 field none, 1 planes:
 * Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7f8ad60000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7f8a96b000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7f8a576000.
Unable to start streaming: Invalid argument (22).
3 buffers released.

root@vcu_llp2_xv20:~#  yavta -n 3 -c10 -f Y10 -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63965.043523] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
 device.
Video format set: Y10 (20303159) 1920x1080 field none,[63965.057886] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
 1 planes:
 * Stride 3840, buffer size 4147200
Video format: [63965.072212] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
Y10 (20303159) 1920x1080 field none, 1 planes:
 * Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fbdb14000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fbd71f000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fbd32a000.
Unable to start streaming: Invalid argument (22).
3 buffers released.

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/174784624/Zynq+UltraScale+MPSoC+VCU+TRD+-+Debugging+-+HDMI+Rx+Capture+Pipeline

root@vcu_llp2_xv20:~# v4l2-ctl --list-devices
vcap_hdmi output 0 (platform:vcap_hdmi:0):
        /dev/video0

vcap_hdmi_2 output 0 (platform:vcap_hdmi_2:0):
        /dev/video1

root@vcu_llp2_xv20:~# ls /sys/class/video4linux/video0/device/ | grep -i media
media1
root@vcu_llp2_xv20:~#  xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0

Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  4.19.0

Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "a0040000.v_proc_ss":1 [ENABLED]

- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:RBG888_1X24/1920x1080 field:none]
                <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
        pad1: Source
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
                -> "vcap_hdmi output 0":0 [ENABLED]

- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
                [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                -> "a0040000.v_proc_ss":0 [ENABLED]

root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1  [fmt:VYYUYY8_1X24/1920x10e]'field:none
Unable to setup formats: Invalid argument (22)

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver

root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1  [fmt:UYVY/1920x1080 colorspace:srgb field:none]'
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video1
Device /dev/video1 opened.
Device `vcap_hdmi_2 output 0' on `platform:vcap_hdmi_2:0' is a video output (without mplan[64761.083514] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
es) device.
Video format set: UYVY (59565955) 1920x1080 field n[64761.097896] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
one, 1 planes:
 * Stride 3840, buffer size 4147200
Video form[64761.112561] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
at: UYVY (59565955) 1920x1080 field none, 1 planes:
 * Stride [64761.125320] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
3840, buffer size 4147200
3 buffers requested.
length: 1 offse[64761.139647] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
t: 4059898224 timestamp type/source: mono/EoF
Buffer 0/0 mapped[64761.154311] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
 at address 0x7fb4223000.
length: 1 offset: 4059898224 timestam[64761.167077] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
p type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fb3e2e0[64761.181400] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
00.
length: 1 offset: 4059898224 timestamp type/source: mono/Eo[64761.196071] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
F
Buffer 2/0 mapped at address 0x7fb3a39000.
[64761.265216] WARNING: CPU: 2 PID: 18896 at drivers/media/common/videobuf2/videobuf2-core.c:1371 vb2_start_streaming+0xe4/0x140
[64761.276502] Modules linked in: dmaproxy(O) al5e(O) al5d(O) allegro(O) xlnx_vcu_clk xlnx_vcu xilinx_hdmi_rx(O) xilinx_hdmi_tx(O) dp159(O) xlnx_vcu_core xilinx_vphy(O) mali(O) uio_pdrv_genirq
[64761.293373] CPU: 2 PID: 18896 Comm: yavta Tainted: G        W  O      4.19.0-xilinx-v2019.2 #1
[64761.301972] Hardware name: ZynqMP ZCU106 RevA (DT)
[64761.306748] pstate: 20000005 (nzCv daif -PAN -UAO)
[64761.311523] pc : vb2_start_streaming+0xe4/0x140
[64761.316036] lr : vb2_start_streaming+0x64/0x140
[64761.320549] sp : ffffff8009663b20
[64761.323847] x29: ffffff8009663b20 x28: ffffffc87924bc00
[64761.329151] x27: ffffff800919e5c8 x26: ffffff8008c02d60
[64761.334455] x25: 0000000000000000 x24: ffffffc87bb757a8
[64761.339759] x23: 0000000000000000 x22: ffffff8009148648
[64761.345063] x21: ffffffc87bb75a90 x20: ffffffc87bb75aa8
[64761.350367] x19: 00000000ffffffea x18: 0000000000000400
[64761.355671] x17: 0000000000000000 x16: 0000000000000000
[64761.360975] x15: 0000000000000400 x14: 000000000000011a
[64761.366279] x13: 0000000000000000 x12: 0000000000000001
[64761.371583] x11: 0000000000000000 x10: 00000000000007f0
[64761.376879] x9 : ffffff80096638e0 x8 : ffffffc8600aeb50
[64761.382183] x7 : ffffffc87ff86a00 x6 : ffffffc8600ae3c0
[64761.387487] x5 : 0000000000000005 x4 : 0000000000004000
[64761.392782] x3 : ffffffc87bb75b08 x2 : 6b059e08b5ce2000
[64761.398086] x1 : 0000000000000000 x0 : ffffffc85fb611b8
[64761.403390] Call trace:
[64761.405823]  vb2_start_streaming+0xe4/0x140
[64761.409990]  vb2_core_streamon+0x88/0x180
[64761.413983]  vb2_streamon+0x18/0x60
[64761.417455]  vb2_ioctl_streamon+0x48/0x58
[64761.421449]  v4l_streamon+0x20/0x28
[64761.424929]  __video_do_ioctl+0x23c/0x498
[64761.428922]  video_usercopy+0x144/0x518
[64761.432741]  video_ioctl2+0x14/0x1c
[64761.436214]  v4l2_ioctl+0x3c/0x58
[64761.439515]  do_vfs_ioctl+0xb8/0x8a0
[64761.443081]  ksys_ioctl+0x44/0x90
[64761.446380]  __arm64_sys_ioctl+0x1c/0x28
[64761.450288]  el0_svc_common+0x84/0xd8
[64761.453941]  el0_svc_handler+0x68/0x80
[64761.457674]  el0_svc+0x8/0xc
[64761.460537] ---[ end trace 091bf6f1324177c3 ]---
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~#

7.1. 参考文章

Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/174784624/Zynq+UltraScale+MPSoC+VCU+TRD+-+Debugging+-+HDMI+Rx+Capture+Pipeline

Cannot capture image by yavta tool https://forums.xilinx.com/t5/Embedded-Linux/Cannot-capture-image-by-yavta-tool/m-p/1064135#M39452

Xilinx V4L2 MIPI CSI driver https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver

本文参与腾讯云自媒体分享计划，欢迎正在阅读的你也加入，一起分享。

展开阅读全文

我来说两句

0 条评论

登录后参与评论

【分享】OpenAMP的RPMSG_ADDR_ANY含义
在OpenAMP的应用程序中，经常看到地址被设置成RPMSG_ADDR_ANY。在通信过程中，为什么可以把源地址、目的地址设置成任意值？
hankfu
【分享】解决gst-launch-1.0错误“ERROR: Failed to allocate required memory.”
执行gst-launch-1.0时，得到错误“ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l...
hankfu
【分享】Xilinx最新7nm器件Versal处理器列表，及Vitis能创建的模板软件工程
Xilinx的28nm、16nm SoC在业界应用非常广泛。最近也发布了7nm的SoC器件Versal VC1902和对应的开发板VCK190。Versal是新...
hankfu
深入剖析 linux GCC 4.4 的 STL String
gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC)
程序员小王
PbootCMS-XSS(stored)漏洞分析
bootCMS是全新内核且永久开源免费的cms,在其V2.0.2版本中存在存储型XSS 漏洞名称：PbootCMS存储型XSS 产品首页：https://ww...
徐焱
String ends with?
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。 ...
喜欢ctrl的cxk
发短信API
谭广健
.net字符串数组查找方式效率比较
下面是代码: static void Main(string[] args) { string[] arr = new...
用户1177503
go语言strings库总结
songleo
Unity打iOS包之xcodeapi的使用
https://bitbucket.org/Unity-Technologies/xcodeapi
bering

腾讯云·社区

hankfu