3D Object Recognition and 6DOF Pose Estimation

.Recognition pipeline

pcl::keypoints focus on CorrespondenceGrouping and Hypothesis Verification. In contrast to registration, we simultaneously deal with several models.

Other options in PCL: 1, LINEMOD [HinterstoisserPAMI2012] 2, ORR [PapazovACCV2010] 3. Segmentation + global features

Correspondence grouping

1 Given a set of correspondences (models - scene), group them together in geometrically consistent clusters from which the pose of the models can be extracted.

2 In contrast to RANSAC based methods, allows simultaneous recognition of multiple objects.

3 Usually applied on recognition pipelines based on local features.

Hypothesis Verification

1, given a set of object hypotheses with a 6DoF pose.

2. Aims at removing false positives while keeping true positives.

3. Allows merging hypotheses coming from different pipelines in a principled way.

From 3D models to 2.5D data

Simulate input from depth/3D sensors.

Code

typedef pcl::PointCloud<pcl::PointXYZ>::Ptr CloudPtr;

pcl::apps::RenderViewsTesselatedSphere render_views;

render_views.setResolution (resolution_);

render_views.setTesselationLevel (1);

render_views.addModelFromPolyData (mapper); //vtk model

render_views.setGenOrganized(false);

render_views.generateViews ();

std::vector< CloudPtr > views;

std::vector < Eigen::Matrix4f > poses;

render_views.getViews (views);

render_views.getPoses (poses);

1. Correspondence Grouping

Incrementally build clusters of correspondences that are geometrically consistent:

All elements in cluster are geom. consistent to each other.

Parameters:

1 epsilon: keypoint inaccuracy, noise

2 gc_min_size : minimum cluster size (at least 3)

3 How to use it within PCL?

m_s_corrs are correspondences with indices to m_keypoints and s_keypoints.

Code

pcl::CorrespondencesPtr m_s_corrs; //fill it

std::vector<pcl::Correspondences> clusters;

pcl::GeometricConsistencyGrouping<PT, PT> gc_clusterer;

gc_clusterer.setGCSize (cg_size);

gc_clusterer.setGCThreshold (cg_thres);

gc_clusterer.setInputCloud (m_keypoints);

gc_clusterer.setSceneCloud (s_keypoints);

gc_clusterer.setModelSceneCorrespondences (m_s_corrs);

gc_clusterer.cluster (clusters);

2.Hough 3D voting

1.Correspondence votes are accumulated in a 3D Hough space. [TombariIPSJ2012]

2. Each point associated with a repeatable RF, RFs used to:

1. reduce voting space from 6 to 3D...

2. by reorienting the voting location

3. Local maxima in the Hough space identify object instances (handles the presence of multiple instances of the same model in the scene)

How to use it within PCL? m_s_corrs are correspondences with indices to m_keypoints and s_keypoints

Code

typedef pcl::ReferenceFrame RFType; pcl::PointCloud<RFType>::Ptr model_rf; //fill with RFs

pcl::PointCloud<RFType>::Ptr scene_rf; //fill with RFs pcl::CorrespondencesPtr m_s_corrs; //fill it std::vector<pcl::Correspondences> clusters; pcl::Hough3DGrouping<PT, PT, RFType, RFType> hc; hc.setHoughBinSize (cg_size); hc.setHoughThreshold (cg_thres); hc.setUseInterpolation (true); hc.setUseDistanceWeight (false); hc.setInputCloud (m_keypoints); hc.setInputRf (model_rf); hc.setSceneCloud (s_keypoints); hc.setSceneRf (scene_rf); hc.setModelSceneCorrespondences (m_s_corrs); hc.cluster (clusters);

3 Hypothesis Verification

1. Keep along the recognition pipeline as many hypotheses as possible and use HV to select those best "explaining the scene".

2. A hypothesis Mi is a model aligned to the scene S.

3. Main goal: Remove FPs without rejecting TPs.

Keep along the recognition pipeline as many hypotheses as possible and use HV to select those best "explaining the scene".

A hypothesis Mi is a model aligned to the scene S.

Main goal: Remove FPs without rejecting TPs.

3 options in PCL:

Greedy [AldomaDAGM12]

Conflict Graph [PapazovACCV11]

Global HV [AldomaECCV12]

3.1HV: Greedy

Reasoning about occlusions to handle occluded objects (in common with all 3 methods).

1. For each hypothesis Mi, count #inliers and #outliers.

2. Greedily select the best hypothesis (#inliers - λ · #outliers) ...

3. and update the inliers count for successive hypotheses, resort and repeat.

4. Mi selected if #inliers - λ · #outliers > 0.

Code

pcl::GreedyVerification<pcl::PointXYZ, pcl::PointXYZ> greedy_hv(lamb

greedy_hv.setResolution (0.005f);

greedy_hv.setInlierThreshold (0.005f);

greedy_hv.setSceneCloud (scene);

greedy_hv.addModels (aligned_hypotheses, true);

greedy_hv.verify ();

std::vector<bool> mask_hv;

greedy_hv.getMask (mask_hv);

3.2. HV: Conflict Graph

1. First, a sequential stage that discards hypotheses based on percen tage of inliers and outliers.

2. From the remaining hypotheses, some are selected based on a non-maxima suppression stage on a conflict graph.

3. Two hypothesis are in conflict if they share the same space.

Code

pcl::PapazovHV<pcl::PointXYZ, pcl::PointXYZ> papazov;

papazov.setResolution (0.005f);

papazov.setInlierThreshold (0.005f);

papazov.setSupportThreshold (0.08f); //inliers

papazov.setPenaltyThreshold (0.05f); //outliers

papazov.setConflictThreshold (0.02f);

papazov.setSceneCloud (scene);

papazov.addModels (aligned_hypotheses, true);

papazov.verify ();

std::vector<bool> mask_hv;

3.3 HV: Global HV

1. Consider the two possible states of a single hypothesis

xi = { 0; 1} (inactive/active).

2. By switching the state of an hypothesis, we can evaluate a global cost function that tell us how good the current solution X = {x1; x2; :::; xn} is.

3. Formally, we are looking for a solution X~ such that:

considers the whole set of hypotheses (M) as a global scene model instead of considering each model hypothesis separately.

Note: global descriptor matching often does not yield automatically the object pose!

EXAMPLE

THE END