如何从VideoMediaFrame.Direct3DSurface中获取像素数据？
如何从VideoMediaFrame.Direct3DSurface中获取像素数据？
提问于 2021-08-19 02:05:24
我为Micorsoft HoloLens 2创建了统一应用程序。
该应用程序使用Windows.Media.Capture.MediaCapture和Microsoft.MixedReality.WebRTC捕捉并共享摄像机视频帧。
我在VideoMediaFrame.Direct3DSurface(Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface)事件上得到了一个由"NV12“格式化的MediaFrameReader.FrameArrived。
但我不知道如何从IDirect3DSurface对象中获取像素数据。
我尝试了如下：
C#代码
[DllImport("Direct3DSurfaceAccess")]
private static extern void Direct3DSurfaceAccess([MarshalAs(UnmanagedType.Interface)]Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface d3dSurface);
// MediaFrameReader.FrameArrived event.
private void Nv12FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args) {
using var mediaFrameReference = sender.TryAcquireLatestFrame();
// I could't find how to get pixel data from IDirect3DSurface using C# only.
// Use C++/WinRT for access IDirect3DSurface as below.
Direct3DSurfaceAccess(mediaFrameReference.VideoMediaFrame.Direct3DSurface);
}C++代码
void __stdcall Direct3DSurfaceAccess(const winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DSurface* d3dSurface) {
// Here, confirmed that d3dSurface is not null.
try {
// Issue is here.
winrt::com_ptr<::Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess> dxgiInterfaceAccess;
d3dSurface->try_as(dxgiInterfaceAccess);
// App crashes on d3dSurface->try_as.
// Call other d3dSurface method as a test.
winrt::Windows::Graphics::DirectX::Direct3D11::Direct3DSurfaceDescription desc = d3dSurface->Description();
// Same above. App crashes on d3dSurface->Description.
// Unreachable to here...
// When succeeded to get the IDirect3DDxgiInterfaceAccess, read pixel data by below code.
winrt::com_ptr<::IDXGISurface> nativeSurface;
dxgiInterfaceAccess->GetInterface(IID_PPV_ARGS(nativeSurface.put()));
::DXGI_MAPPED_RECT rect;
nativeSurface->Map(&rect, DXGI_MAP_READ);
const BYTE* pixels = rect.pBits; // Read pixel data from the pointer.
nativeSurface->Unmap();
} catch (...) {
// Can't catch any exceptions.
// App freezes and downs on HoloLens 2.
}
}我认为如何封送IDirect3DSurface是错误的。
如何从IDirect3DSurface中获取像素数据
回答 1
Stack Overflow用户
回答已采纳
发布于 2021-08-20 19:06:06
不能在dll边界上定义C++/WinRT对象指针，因为在二进制级别上，它不是COM对象(因此C#传递的是映射到C++/WinRT对象的COM对象，这很糟糕)。相反，你可以这样做：
void __stdcall Direct3DSurfaceAccess(
IUnknown* surface, // pass a raw COM object (here I use IUnknown)
char* message,
size_t messageLength
) {
try {
// map to a C++/WinRT object here
auto d3dSurface = convert_from_abi<winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DSurface>(surface);
winrt::com_ptr<::Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess> dxgiInterfaceAccess;
if (!d3dSurface.try_as(dxgiInterfaceAccess)) {
::sprintf_s(message, messageLength, "d3dSurface->try_as failed.");
return;
}
// Call other d3dSurface method as a test.
auto desc = d3dSurface.Description();convert_from_abi来自这里，abi函数
在这种情况下，您可以像这样定义interop函数：
[DllImport(@"Direct3DSurfaceAccess.dll")]
private static extern void Direct3DSurfaceAccess([MarshalAs(UnmanagedType.IUnknown)] object d3dSurface);页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/68841164
复制相关文章
相似问题
腾讯云开发者
