blocks|key|2954848|text|所以在你的问题中有几个东西是没有定义的，但我会尽我最大的努力来概述一下答案。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2954849|你想做的事情的基本思想是取图像的梯度，然后对梯度应用变换以获得法向量。在matlab中获取梯度很容易：|2954850|[m,+g]+=+imgradient(d);|code-block|syntax|javascript|2954851|为我们提供图像在每个点的梯度(相对于水平线和度数)的大小(m)和方向(g)。例如，如果我们为你的图像显示梯度的大小，它看起来像这样：|offset|length|style|CODE|2954852|​|2954853|📷|atomic|2954854|2954855|现在，更难的部分是获取我们所拥有的关于梯度的信息，并将其转换为法向量。为了正确地做到这一点，我们需要知道如何从图像坐标转换到世界坐标。对于像您这样的CAD生成的图像，此信息包含在用于生成图像的投影变换中。对于从Kinect获得的真实图像，您必须查找图像捕获设备的规格。|2954856|我们需要的关键信息是:在现实世界的坐标中，每个像素到底有多宽？对于非正交投影(如真实世界图像捕获设备使用的投影)，我们可以通过假设每个像素表示真实世界固定角度内的光来近似这一点。如果我们知道这个角度(称为p并以弧度测量它)，那么像素覆盖的实际距离就是sin(p)+.*+d，或者大约是p+.*+d，其中d是图像在每个像素上的深度。|2954857|现在，如果我们有了这个信息，我们就可以构造法向量的3个分量：|2954858|width+=+p+.*+d;
gradx+=+m+.*+cos(g)+*+width;
grady+=+m+.*+sin(g)+*+width;

normx+=+-+gradx;
normy+=+-+grady;
normz+=+1;

len+=+sqrt(normx+.%5E+2+%2B+normy+.%5E+2+%2B+normz+.%5E+2);
x+=+normx+./+len;
y+=+normy+./+len;
z+=+normz+./+len;|2954859|entityMap|0|IMAGE|mutability|IMMUTABLE|imageUrl|https://ask.qcloudimg.com/http-save/yehe-900000/cd9a8c6d300a0cb194f944797ce93f67.png|imageAlt^0|0|0|0|T|1|Z|1|0|0|0|1|0|0|0|0|2U|1|3H|B|3Y|6|47|1|0|0|0^^$0|@$1|2|3|4|5|6|7|1B|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1C|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|1D|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|1E|8|@$K|1F|L|1G|M|N]|$K|1H|L|1I|M|N]]|9|@]|A|$]]|$1|O|3|P|5|6|7|1J|8|@]|9|@]|A|$]]|$1|Q|3|R|5|S|7|1K|8|@]|9|@$K|1L|L|1M|1|1N]]|A|$]]|$1|T|3|P|5|6|7|1O|8|@]|9|@]|A|$]]|$1|U|3|V|5|6|7|1P|8|@]|9|@]|A|$]]|$1|W|3|X|5|6|7|1Q|8|@$K|1R|L|1S|M|N]|$K|1T|L|1U|M|N]|$K|1V|L|1W|M|N]|$K|1X|L|1Y|M|N]]|9|@]|A|$]]|$1|Y|3|Z|5|6|7|1Z|8|@]|9|@]|A|$]]|$1|10|3|11|5|F|7|20|8|@]|9|@]|A|$G|H]]|$1|12|3|-4|5|6|7|21|8|@]|9|@]|A|$]]]|13|$14|$5|15|16|17|A|$18|19|1A|-4]]]]

So there are a couple of things that are undefined in your question, but I'll do my best to outline an answer.

The basic idea for what you want to do is to take the gradient of the image, and then apply a transformation to the gradient to get the normal vectors. Taking the gradient in matlab is easy:

<pre><code>[m, g] = imgradient(d);
</code></pre>

gives us the magnitude (<code>m</code>) and the direction (<code>g</code>) of the gradient (relative to the horizontal and measured in degrees) of the image at every point. For instance, if we display the magnitude of the gradient for your image it looks like this:

<img src="https://i.stack.imgur.com/1Lcrq.png" alt="enter image description here">

Now, the harder part is to take this information we have about the gradient and turn it into a normal vector. In order to do this properly we need to know how to transform from image coordinates to world coordinates. For a CAD-generated image like yours, this information is contained in the projection transformation used to make the image. For a real-world image like one you'd get from a Kinect, you would have to look up the spec for the image-capture device. 

The key piece of information we need is this: just how wide is each pixel in real-world coordinates? For non-orthonormal projections (like those used by real-world image capture devices) we can approximate this by assuming each pixel represents light within a fixed angle of the real world. If we know this angle (call it <code>p</code> and measure it in radians), then the real-world distance covered by a pixel is just <code>sin(p) .* d</code>, or approximately <code>p .* d</code> where <code>d</code> is the depth of the image at each pixel.

Now if we have this info, we can construct the 3 components of the normal vectors:

<pre><code>width = p .* d;
gradx = m .* cos(g) * width;
grady = m .* sin(g) * width;

normx = - gradx;
normy = - grady;
normz = 1;

len = sqrt(normx .^ 2 + normy .^ 2 + normz .^ 2);
x = normx ./ len;
y = normy ./ len;
z = normz ./ len;
</code></pre>

blocks|key|153832|text|mattnewport建议的是可以在像素着色器中完成。在每个像素着色器中，计算两个向量A和B，向量的叉积将为您提供法线。计算这两个向量的方法如下所示：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|153833|float2+du+//values+sent+to+the+shader+based+on+depth+image's+width+and+height
float2+dv+//normally+du+=+float2(1/width,+0)+and+dv+=+float2(0,+1/height)
float+D+=+sample(depthtex,+uv)
float+D1+=+sample(depthtex,+uv+%2B+du)
float+D2+=+sample(depthtex,+uv+%2B+dv)
float3+A+=+float3(du*width_of_image,+0,+D1-D)
float3+B+=+float3(0,+dv*height_of_image,+D2-D)
float3+normal+=+AXB
return+normal|code-block|syntax|javascript|153834|当深度值中存在不连续时，这将中断。|153835|要计算一个表面是否在像素着色器中是平的，可以使用二阶偏导数。计算二阶导数的方法是通过计算有限差分，然后求出差分，就像这样：|153836|float+D+=+sample(depthtex,+uv)
float+D1+=+sample(depthtex,+uv+%2B+du)
float+D3+=+sample(depthtex,+uv+-+du)

float+dx1+=+(D1+-+D)/du
float+dx2+=+(D+-+D3)/du
float+dxx+=+(dx2+-+dx1)/du|153837|用同样的方法，你必须计算dyy,+dxy+and+dyx。如果为dxx+=+dyy+=+dxy+=+dyx+=+0.，则曲面是平坦的|offset|length|style|CODE|153838|通常，您会选择du和dv为深度图像的1/width和1/height。|153839|所有这些事情都发生在GPU上，这使得一切都变得非常快。但是如果你不关心这一点，你也可以在CPU中运行这个方法。唯一的问题是你要替换一个像sample这样的函数，并实现你自己的版本。它将深度图像和u，v值作为输入，并返回采样点的深度值。|153840|编辑：|153841|这是一个假设的采样函数，它在CPU上执行最近邻采样。|153842|float+Sample(const+Texture&+texture,+vector_2d+uv){
++++return+texture.data[(int)(uv.x+*+texture.width+%2B+0.5)][(int)(uv.y+*+texture.height+%2B+0.5];
}|153843|entityMap^0|0|0|0|0|0|C|G|W|Q|0|0|1W|6|0|0|0|0^^$0|@$1|2|3|4|5|6|7|14|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|15|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|16|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|17|8|@]|9|@]|A|$]]|$1|K|3|L|5|D|7|18|8|@]|9|@]|A|$E|F]]|$1|M|3|N|5|6|7|19|8|@$O|1A|P|1B|Q|R]|$O|1C|P|1D|Q|R]]|9|@]|A|$]]|$1|S|3|T|5|6|7|1E|8|@]|9|@]|A|$]]|$1|U|3|V|5|6|7|1F|8|@$O|1G|P|1H|Q|R]]|9|@]|A|$]]|$1|W|3|X|5|6|7|1I|8|@]|9|@]|A|$]]|$1|Y|3|Z|5|6|7|1J|8|@]|9|@]|A|$]]|$1|10|3|11|5|D|7|1K|8|@]|9|@]|A|$E|F]]|$1|12|3|-4|5|6|7|1L|8|@]|9|@]|A|$]]]|13|$]]

What mattnewport is suggesting is can be done in a pixel shader. In each pixel shader you calculate two vectors A and B and the cross product of the vectors will give you the normal. The way you calculate the two vectors is like so:

<pre><code>float2 du //values sent to the shader based on depth image's width and height
float2 dv //normally du = float2(1/width, 0) and dv = float2(0, 1/height)
float D = sample(depthtex, uv)
float D1 = sample(depthtex, uv + du)
float D2 = sample(depthtex, uv + dv)
float3 A = float3(du*width_of_image, 0, D1-D)
float3 B = float3(0, dv*height_of_image, D2-D)
float3 normal = AXB
return normal
</code></pre>

This will break when there're discontinuities in the depth values. 

To calculate if a surface is flat in the pixel shader the second order partial derivatives can be used. The way you calculate the second order derivatives is by calculating the finite differences and the finding the difference on that like so:

<pre><code>float D = sample(depthtex, uv)
float D1 = sample(depthtex, uv + du)
float D3 = sample(depthtex, uv - du)

float dx1 = (D1 - D)/du
float dx2 = (D - D3)/du
float dxx = (dx2 - dx1)/du
</code></pre>

In the same way you have to calculate <code>dyy, dxy and dyx</code>. The surface is flat if <code>dxx = dyy = dxy = dyx = 0.</code>

Typically, you'd choose the du and dv to be 1/width and 1/height of the depth image .

All of this stuff happens on the GPU which makes everything really fast. But if you don't care about that you can run this method in the CPU as well. The only issue will be for you to replace a function like <code>sample</code> and implement your own version of that. It will take the depth image and u, v values as input and return a depth value at the sampled point.

Edit:

Here's a hypothetical sampling function that does nearest neighbour sampling on the CPU. 

<pre><code>float Sample(const Texture&amp; texture, vector_2d uv){
 return texture.data[(int)(uv.x * texture.width + 0.5)][(int)(uv.y * texture.height + 0.5];
}
</code></pre>

blocks|key|2954882|text|我将从概念上描述你需要做的事情，并提供opencv相关部分的链接。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2954883|要确定点云中给定(3d)点的法线，请执行以下操作：|2954884|2954885|创建kd-tree或(tree+tree？)表示您的点云，以便您可以高效地计算k个最近的邻居。K的选择应该取决于数据的密度。http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1KDTree.html|ordered-list-item|offset|length|2954886|After+http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search查询给定点p的k个最近邻点，使用它们来找到最佳拟合平面。您可以使用PCA来完成此操作。Set+maxComponents=2。http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1PCA.html+https://github.com/Itseez/opencv/blob/master/samples/cpp/pca.cpp+|2954887|步骤2应该返回两个定义您感兴趣的平面的特征向量。这两个向量的叉积应该是你想要的法向量(估计)。你可以在opencv+(Mat::cross)+http://docs.opencv.org/modules/core/doc/basic_structures.html|2954888|中找到如何计算这个值的信息|2954889|entityMap|0|LINK|mutability|MUTABLE|url|http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1KDTree.html|1|http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search|2|http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1PCA.html|3|https://github.com/Itseez/opencv/blob/master/samples/cpp/pca.cpp|4|http://docs.opencv.org/modules/core/doc/basic_structures.html^0|0|0|0|1Q|1X|0|0|6|2L|1|4I|1U|2|6D|1S|3|0|1Z|1P|4|0|0^^$0|@$1|2|3|4|5|6|7|15|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|16|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|17|8|@]|9|@]|A|$]]|$1|E|3|F|5|G|7|18|8|@]|9|@$H|19|I|1A|1|1B]]|A|$]]|$1|J|3|K|5|G|7|1C|8|@]|9|@$H|1D|I|1E|1|1F]|$H|1G|I|1H|1|1I]|$H|1J|I|1K|1|1L]]|A|$]]|$1|L|3|M|5|G|7|1M|8|@]|9|@$H|1N|I|1O|1|1P]]|A|$]]|$1|N|3|O|5|6|7|1Q|8|@]|9|@]|A|$]]|$1|P|3|-4|5|6|7|1R|8|@]|9|@]|A|$]]]|Q|$R|$5|S|T|U|A|$V|W]]|X|$5|S|T|U|A|$V|Y]]|Z|$5|S|T|U|A|$V|10]]|11|$5|S|T|U|A|$V|12]]|13|$5|S|T|U|A|$V|14]]]]

I will describe what I think you have to do conceptually and provide links to the relevant parts of opencv,

To determine normal of a given (3d) point in a pointcloud:

<ol>
<li>Create a kd-tree or (balltree?) representation of your point cloud so that you can efficiently compute the k nearest neighbors. Your choice of k should depend on the density of your data. <a href="http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search" rel="nofollow">http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search</a>
<a href="http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1KDTree.html" rel="nofollow">http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1KDTree.html</a></li>
<li>After querying for the k-nearest neighbors of a given point p, use them to find a best fit plane. You can use PCA to do this. Set maxComponents=2. 
<a href="http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1PCA.html" rel="nofollow">http://physics.nyu.edu/grierlab/manuals/opencv/classcv_1_1PCA.html</a>
<a href="https://github.com/Itseez/opencv/blob/master/samples/cpp/pca.cpp" rel="nofollow">https://github.com/Itseez/opencv/blob/master/samples/cpp/pca.cpp</a> </li>
<li>Step 2 should return two eigenvectors which define the plane you are interested in. The cross product of these two vectors should be (an estimation of) your desired normal vector. You can find info how to calculate this in opencv (Mat::cross) <a href="http://docs.opencv.org/modules/core/doc/basic_structures.html" rel="nofollow">http://docs.opencv.org/modules/core/doc/basic_structures.html</a></li>
</ol>

blocks|key|2954919|text|你可以参考“有效地组合位置和法线以实现精确的3D几何”中的(5)、(6)和(8)：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2954920|​|2954921|📷|atomic|offset|length|2954922|2954923|2954924|2954925|2954926|entityMap|0|IMAGE|mutability|IMMUTABLE|imageUrl|https://ask.qcloudimg.com/http-save/yehe-900000/88501ee081a98d22c2c5034d8203f3e9.png|imageAlt|1|https://ask.qcloudimg.com/http-save/yehe-900000/f40c1c7a4224800ed42db955d83706c6.png^0|0|0|0|1|0|0|0|0|0|1|1|0|0^^$0|@$1|2|3|4|5|6|7|X|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Y|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|Z|8|@]|9|@$G|10|H|11|1|12]]|A|$]]|$1|I|3|C|5|6|7|13|8|@]|9|@]|A|$]]|$1|J|3|C|5|6|7|14|8|@]|9|@]|A|$]]|$1|K|3|E|5|F|7|15|8|@]|9|@$G|16|H|17|1|18]]|A|$]]|$1|L|3|C|5|6|7|19|8|@]|9|@]|A|$]]|$1|M|3|-4|5|6|7|1A|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|T|U|-4]]|V|$5|P|Q|R|A|$S|W|U|-4]]]]

You can refer to (5), (6) and (8) in &quot;Efficiently Combining Positions and Normals for Precise 3D Geometry&quot;:
<a href="https://i.stack.imgur.com/KHDct.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KHDct.png" alt="enter image description here" /></a>
<a href="https://i.stack.imgur.com/kof18.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kof18.png" alt="enter image description here" /></a>

I have a depth image, that I've generated using 3D CAD data. This depth image can also be taken from a depth imaging sensor such as Microsoft Kinect or any other stereo camera. So basically it is a depth map of points visible in the imaging view. In other words it is segmented point cloud of an object from a certain view.
I would like to determine (estimating will also do) the surface normals of each point, then find tangent plane of that point.
How can I do this? I've did some research and found some techniques but didn't understand them well (I could not implement it). More importantly how can I do this in Matlab or OpenCV? I couldn't manage to do this using <code>surfnorm</code> command. AFAIK it needs a single surface, and I have partial surfaces in my depth image.
This is an example depth image.
<img src="https://i.stack.imgur.com/bEfin.png" alt="enter image description here" />
[EDIT]
What I want to do is, after I get the surface normal at each point I will create tangent planes at those points. Then use those tangent planes to decide if that point is coming from a flat region or not by taking the sum of distances of neighbor points to the tangent plane.

How to estimate/determine surface normals and tangent planes at points of a depth image?

我有一个深度图像，它是我用3DCAD数据生成的。此深度图像也可以从深度成像传感器(如Microsoft Kinect或任何其他立体相机)获取。因此，基本上它是成像视图中可见点的深度图。换句话说，它是从特定视图中分割的对象的点云。我想确定(估计也会做)每个点的曲面法线，然后找到该点的切平面。我该怎么做呢？我做了一些研究，...

问如何估计/确定深度图像点处的曲面法线和切平面？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何估计/确定深度图像点处的曲面法线和切平面？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何估计/确定深度图像点处的曲面法线和切平面？
EN