• Author: Vadim Pisarevsky
• Link: The feature request
• Status: Draft
• Platforms: All
• Complexity: a few man-months

In opencv_contrib there is rgbd module that does some point cloud processing. Besides, there is well-known Point Cloud Library under BSD license, which contains a lot of good stuff that can be ported to OpenCV. While we do not need to move every single piece of the code around the world into OpenCV, it makes sense to build a better foundation for 3D vision and robotics in OpenCV.

Create a dedicated pclouds module in the main repository with some assorted algorithms from rgbd, calib3d and maybe PCL. Some initial set of functionality may include:

• algorithm to compute surface normals
• RGBD/visual odometry (compute camera trajectory based on the sequence of RGBD frames it observes)
• registration of (establishing correspondence between) RGB data and depth data coming from different sensors
• 3D filtering (smooth surfaces)
• 3D feature detection (corners)
• maybe kinect fusion (?)
• maybe DTAM
• ICP or such
• surface registration (there is dedicated module in opencv_contrib, but it implements a patented algorithm, maybe there are non-patented alternatives)
• 3D shape fitting
• structure from motion? (there is dedicated sfm module in opencv_contrib for that)
• building blocks for SLAM (or make it a separate module?)
• ...

Overall, this is a loosely covered part of CV in OpenCV, so it's unlikely that many people use the existing rgbd module.

Perhaps, it should be more than 1 module and perhaps sfm and some other parts should stay in opencv_contrib for now.

TBD