RTSP H264 流 MSE 播放

elsampsa/websocket-mse-demo[1] 项目介绍：

It this demo we're streaming live video from an RTSP camera to your HTML5 browser. Video is streamed as H264 encapsulated in MP4. No transcoding takes place in the stream-to-browser pipeline. No browser-side flash or pure-javascript decoders required. Video is decoded and presented in the browser using the W3 Media Source Extensions that is well supported by all major browsers by > now. Only H264 is supported - browser MSE's do not support H246 yet (and neither does libValkka).

实现了浏览器 MSE (Media Source Extensions) 播放相机 RTSP (Real Time Streaming Protocol) 流。动手体验一下咯~

准备

# 系统 Ubuntu 18

# 如有 conda 虚拟环境，请停用
# conda deactivate

# 获取代码
git clone --depth 1 https://github.com/elsampsa/websocket-mse-demo.git

# 安装 libValkka
sudo apt-add-repository ppa:sampsa-riikonen/valkka
sudo apt-get update
sudo apt-get install valkka

# 安装 nginx
sudo apt-get install nginx

# 安装 python websockets
pip3 install websockets

# 准备 nginx 用户权限
sudo addgroup nginx
sudo adduser $USER nginx
sudo chgrp -R nginx /var/lib/nginx /var/log/nginx
sudo chmod -R g+r+w+x /var/lib/nginx /var/log/nginx
# 切换用户组
newgrp nginx
id

# 测试 nginx 工作是否正常（无 Permission denied）
cd websocket-mse-demo
nginx -p $PWD -c ./nginx.conf -g 'error_log error.log warn;'

# 设置相机发送 H264（该项目只支持 H264）

使用

发送 RTSP 流：

sudo snap install vlc
# allow camera permission
snap connect vlc:camera :camera

# 发送 ~/Videos/test.mp4 文件（循环）
#  https://archive.org/download/archive-video-files/test.mp4
vlc file:///$HOME/Videos/test.mp4 --loop \
--sout '#gather:transcode{vcodec=h264}:rtp{sdp=rtsp://:8554/test}' \
--network-caching=1500 --sout-all --sout-keep

# 或，发送 /dev/video0 webcam（实时）
vlc v4l2:///dev/video0:chroma=yuyv:width=640:height=480:fps=30 \
--sout '#transcode{vcodec=h264,width=640,height=480}:rtp{sdp=rtsp://:8554/test}'

# 测试接收 RTSP 流
vlc rtsp://127.0.0.1:8554/test

启动程序：

killall -9 nginx
python3 ws_serve_new.py rtsp://127.0.0.1:8554/test

打开浏览器：

xdg-open http://localhost:8089

更多

技术：

• WebSocket[2]
• MediaSource[3]: 构建媒体流，供 video/audio 元素播放
• WebRTC[4]: 可直接捕获客户端视频流，并点对点传输、不依赖服务器中转
• WebCodecs[5]: 编解码音视频
• WebAssembly[6]: C/C++/Rust 等编译进前端使用
  • ffmpegwasm/ffmpeg.wasm[7]
  • GoogleChromeLabs/webm-wasm[8]

项目：

• MSE: 需构建流，通常容器格式是 ISO BMFF (Fragmented MP4) = fMP4
  • elsampsa/websocket-mse-demo[9]: RTSP > fMP4
  • ChihChengYang/wfs.js[10]: H.264 NAL > fMP4
  • bilibili/flv.js[11]: FLV > fMP4
  • Garefield/RTSP-WS-Proxy[12]: RTSP > fMP4
  • Streamedian/html5_rtsp_player[13]: RTSP > fMP4, Proxy 收费
• JS 解码
  • 131/h264-live-player[21]
  • kyriesent/node-rtsp-stream[15]
  • wanghaoxi3000/gin-rtsp[16]
  • YE-Fan/JAVA-RTSP-JSMpeg[17]
  • Aleckgt/rtsp-ws-proxy[18]
  • k-yle/rtsp-relay[19]
  • phoboslab/jsmpeg[14]: MPEG1 Video & MP2 Audio Decoder
  • mbebenita/Broadway[20]: H.264 decoder
• Wasm 解码
  • sonysuqin/WasmVideoPlayer[22]: H265 > YUV, WebGL
  • wupeng-engineer/decoder_wasm[23]: H264,H265 > YUV, WebGL
  • gliese1337/h264decoder[24]: H264 > YUV
  • mattdesl/mp4-h264[25]: H264 > YUV/RGB
  • mattdesl/mp4-wasm[26]: H264
• 其他
  • HTML5 RTSP player[27]: 收费

脚注

[1]

elsampsa/websocket-mse-demo: https://github.com/elsampsa/websocket-mse-demo

[2]

WebSocket: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket

[3]

MediaSource: https://developer.mozilla.org/en-US/docs/Web/API/MediaSource

[4]

WebRTC: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API

[5]

WebCodecs: https://github.com/w3c/webcodecs

[6]

WebAssembly: https://webassembly.org/

[7]

ffmpegwasm/ffmpeg.wasm: https://github.com/ffmpegwasm/ffmpeg.wasm

[8]

GoogleChromeLabs/webm-wasm: https://github.com/GoogleChromeLabs/webm-wasm

[9]

elsampsa/websocket-mse-demo: https://github.com/elsampsa/websocket-mse-demo

[10]

ChihChengYang/wfs.js: https://github.com/ChihChengYang/wfs.js

[11]

bilibili/flv.js: https://github.com/Bilibili/flv.js

[12]

Garefield/RTSP-WS-Proxy: https://github.com/Garefield/RTSP-WS-Proxy

[13]

Streamedian/html5_rtsp_player: https://github.com/Streamedian/html5_rtsp_player

[14]

phoboslab/jsmpeg: https://github.com/phoboslab/jsmpeg

[15]

kyriesent/node-rtsp-stream: https://github.com/kyriesent/node-rtsp-stream

[16]

wanghaoxi3000/gin-rtsp: https://github.com/wanghaoxi3000/gin-rtsp

[17]

YE-Fan/JAVA-RTSP-JSMpeg: https://github.com/YE-Fan/JAVA-RTSP-JSMpeg

[18]

Aleckgt/rtsp-ws-proxy: https://github.com/Aleckgt/rtsp-ws-proxy

[19]

k-yle/rtsp-relay: https://github.com/k-yle/rtsp-relay

[20]

mbebenita/Broadway: https://github.com/mbebenita/Broadway

[21]

131/h264-live-player: https://github.com/131/h264-live-player

[22]

sonysuqin/WasmVideoPlayer: https://github.com/sonysuqin/WasmVideoPlayer

[23]

wupeng-engineer/decoder_wasm: https://gitee.com/wupeng-engineer/decoder_wasm

[24]

gliese1337/h264decoder: https://github.com/gliese1337/h264decoder

[25]

mattdesl/mp4-h264: https://github.com/mattdesl/mp4-h264

[26]

mattdesl/mp4-wasm: https://github.com/mattdesl/mp4-wasm

[27]

HTML5 RTSP player: https://flashphoner.com/ip-camera-streaming-via-rtsp-for-webrtc-and-websocket-browsers/