3D Vision Technology - 3D Matching

Video: Pick-and-place application with 3D matching

With 3D matching, it is possible to recognize arbitrary 3D objects and to determine their 3D pose even with only one camera or depth sensor.

The 3D matching of MVTec HALCON is particularly used for 3D alignment, i.e., determining the 3D position and orientation of 3D objects from common or distance images, e.g., within automotive and robotics applications, pick-and-place applications (see an example in the video on the right), and bin picking.

The object's 3D position and orientation is determined by matching multiple 2D views of a known 3D object.

A further possibility is the measuring of geometric features and locating defects on complex 3D objects after 3D alignment.

To determine the 3D pose from common images, shape-based 3D matching is used. It extends the technology of shape-based matching to 3D by using multiple 2D views of the 3D object, represented by its CAD model. To determine the 3D pose from distance images, surface-based 3D matching is used. It combines 3D point cloud data and edge information from distance images. This allows the robust pose determination even for smooth objects without prominent edges, which would not show significant gray value edges in common images. To gain highest accuracy, both methods provide a refinement of the pose in the full 3D space.

Localizing a known object.

Control of the robot’s grasping process.

A reference object is compared with a test object to detect bent parts.

The 3D surface inspection is done based on surface-based matching and object comparison.