获取每个视频的帧作为QImage
获取每个视频的帧作为QImage
提问于 2021-09-16 17:12:00
我开始了一个项目，其中包括接收来自RTSP服务器的视频，并使用QT c++和GSTreamer在面板中显示它。我想以QImage对象的形式接收每个帧，以运行一些我需要的功能。我知道我们可以使用QTMultimedia来链接Gstreamer并查看视频，但在这种情况下，我希望使用QImages。我找到了一种方法(或者我想我找到了)通过调用函数"gst_app_sink_pull_sample“从视频中获取样本。我们从中得到一个GSTSample。然而，我没有找到任何方法来将其转换为，例如，JPG原始数据，这很容易转换为QImage。
我还找到了一种从这里访问GSTSample数据的方法："How to get video stream frame-by-frame from Gstreamer pipeline? (without OpenCV)“，但是，我还是不知道如何将这些数据转换为QImage。
/* Initialize GStreamer */
gst_init (NULL, NULL);
/* Create the elements */
GstElement * source = gst_element_factory_make ("videotestsrc", "source"); // HERE I'M USING THE GSTREAMER SOURCE TEST
GstElement * sink = gst_element_factory_make ("appsink", "sink");
GstAppSink *appsink = GST_APP_SINK(sink);
/* Create the empty pipeline */
GstElement * pipeline = gst_pipeline_new ("test-pipeline");
if (!pipeline || !source || !sink) {
g_printerr ("Not all elements could be created.\n");
return ;
}
/* Build the pipeline */
gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
if (gst_element_link (source, sink) != TRUE) {
g_printerr ("Elements could not be linked.\n");
gst_object_unref (pipeline);
return ;
}
/* Modify the source's properties */
g_object_set (source, "pattern", 0, NULL);
/* Start playing */
ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (pipeline);
return;
}
// My question starts HERE
GstSample *sample = nullptr;
do{
sample = gst_app_sink_pull_sample(appsink); // Get the frame
if (!sample) {
printf("sample is NULL\n");
}else{
// Get the raw data (or trying to...)
GstBuffer * buffer = gst_sample_get_buffer(sample);
GstMapInfo info;
gst_buffer_map(buffer, &info, GST_MAP_READ);
// Dumb way to check if the frame is being received properly
QImage imageAux((const unsigned char*)info.data, 100, 100, QImage::Format_RGB16);
imageAux.save("out.jpg"); //returns garbage
}
}while(sample);Stack Overflow用户
发布于 2021-09-24 14:32:42
明白了!我找到了这个例子：Qt+GStreamer: How to take a snapshot while playing live video stream，它使用了QT-Gstreamer库，不幸的是，它不能在我的系统中工作。我错过了帧的格式。因此，我需要将其转换为已知格式的(gst_caps_new_simple(...) )，并最终将其转换为QImage。
/* Initialize GStreamer */
gst_init (NULL, NULL);
/* Create the elements */
source = gst_element_factory_make ("videotestsrc", "source");
//sink = gst_element_factory_make ("autovideosink", "sink");
sink = gst_element_factory_make ("appsink", "sink");
GstAppSink *appsink = GST_APP_SINK(sink);
/* Create the empty pipeline */
pipeline = gst_pipeline_new ("test-pipeline");
if (!pipeline || !source || !sink) {
g_printerr ("Not all elements could be created.\n");
return ;
}
/* Build the pipeline */
gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
if (gst_element_link (source, sink) != TRUE) {
g_printerr ("Elements could not be linked.\n");
gst_object_unref (pipeline);
return ;
}
/* Modify the source's properties */
g_object_set (source, "pattern", 0, NULL);
// Start the ThreadVideoProcessor
/* Start playing */
ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
g_printerr ("Unable to set the pipeline to the playing state.\n");
gst_object_unref (pipeline);
return;
}
GstSample *sample = nullptr;
do{
sample = gst_app_sink_pull_sample(appsink);
if (!sample) {
printf("sample is NULL\n");
}else{
GstMapInfo map;
GstCaps *caps = gst_sample_get_caps(sample);
GError *err = NULL;
GstStructure *structure = gst_caps_get_structure(caps, 0);
const int width = g_value_get_int(gst_structure_get_value(structure, "width"));
const int height = g_value_get_int(gst_structure_get_value(structure, "height"));
GstCaps * capsTo = gst_caps_new_simple("video/x-raw",
"format", G_TYPE_STRING, "RGB",
"width", G_TYPE_INT, width,
"height", G_TYPE_INT, height,
NULL);
GstSample *convertedSample = gst_video_convert_sample(sample,capsTo,GST_SECOND,&err);
if (convertedSample == nullptr) {
//qWarning() << "gst_video_convert_sample Failed:" << err->message;
}
else {
//qDebug() << "Converted sample caps:" << convertedSample->caps()->toString();
GstBuffer * buffer = gst_sample_get_buffer(convertedSample);
gst_buffer_map(buffer, &map, GST_MAP_READ);
QImage snapShot = QImage((const uchar *)map.data,
width,
height,
QImage::Format_RGB888);
//qDebug() << "Saving snap to" << "out.jpg";
snapShot.save("out.jpg");
}
}执行后的：
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/69212572
复制相关文章
相似问题
腾讯云开发者
