3D RECONSTRUCTION WITH OPENCV AND POINT CLOUD LIBRARY

Hi everybody!

I bring you a sample of how to reconstruct a scene in 3D using OpenCV and Point Cloud Library (PCL) with a simple program and an example scene.

All we need is the left image of our stereo camera:

(You can implement your own cheap stereo webcam following this post: OpenCV Stereo Webcam)

The disparity map generated with your preferred Stereo Matching algorithm:

(For example you can use OpenCV's stereoBM algorithm: OpenCV StereoBM)

And the reprojection matrix (Q) obtained at calibration time:

<?xml version="1.0"?>
<opencv_storage>
<Q type_id="opencv-matrix">
  <rows>4</rows>
  <cols>4</cols>
  <dt>d</dt>
  <data>
    1. 0. 0. -2.9615028381347656e+02
    0. 1. 0. -2.3373317337036133e+02
    0. 0. 0. 5.6446880931501073e+02
    0. 0. -1.1340974198400260e-01 4.1658568844268817e+00
  </data></Q>
</opencv_storage>

(You can get the matrix Q for your own stereo camera following the instructions in this post: OpenCV Camera Calibration)

Now download the source code (I highly recommend to read the source code to understand what is going on, don't worry there is comments :P):

[NOTE]: You will need to have installed OpenCV Library (you can get it here) and Point Cloud Library (you can get it here). Also you will need CMake to generate the Makefiles.

Once you have downloaded the source code and installed the dependencies, just run:

tar xzvf OpenCVReprojectImageToPointCloud-1.0.tgz
cd OpenCVReprojectImageToPointCloud
cmake .
make
./OpenCVReprojectImageToPointCloud rgb-image.ppm disparity-image.pgm Q.xml

You should see something similar to the following video:

I hope you enjoy it!
[UPDATE: 04/02/2012] I have released a bug-fix. Thanks to Chris for pointing it out.