OPENCV: STEREO MATCHING

In a previous post I talked about how to calibrate a stereo camera using OpenCV. Today, I would like to talk about the next step. Once your stereo camera is calibrated you can estimate the 3D position (relative to the camera) of any object given its position in the left and right image. For that, we need to calculate the stereo disparity for that object (stereo disparity = the difference in image location of an object seen by the left and right camera). If we want to know the 3D position of all points in a stereo pair of images, then we want to compute a dense disparity map. And that is what this post goes about.

A dense disparity map looks like this:

 I am not going to explain the details or the math behind it, I am more of a practical kind of guy. So let's start.

Basically OpenCV provides 2 methods to calculate a dense disparity map:

• cvFindStereoCorrespondenceBM: Fast (can process several images per second), but if parameters not tuned then the results are poor.
• cvFindStereoCorrespondenceGC: Really Slow (takes several seconds, even minutes per image), but gets very accurate results.

In this post I will focus on cvFindStereoCorrespondenceBM, this method is based on Konolige's Block Matching Algorithm. The OpenCV call looks like this:

void cvFindStereoCorrespondenceBM(const CvArr* left, const CvArr* right, CvArr* disparity, CvStereoBMState* state)

The structure CvStereoBMState contains all the parameters that are applicable to the algorithm. There is a bunch of them (pre-filtering, Sum of Absolute Difference windows size, disparity-related, post-filtering...). So, to make it easy, I implemented a small Gtk application that takes 2 images (left image and right image), calculates the disparity map using cvFindStereoCorrespondenceBM and allows you to play with the parameters.

The application is written in C and can be downloaded here: StereoBMTuner-1.0. The application depends on the libraries gtk+-2.0, gmodule-2.0 and opencv. Be sure to have them installed in your system.

Once the file is downloaded just execute:

tar xzvf stereoBMTunner-1.0.tgz
cd StereoBMTunner
make
./main

The last command will execute the application

As you can appreciate, the disparity map generated using the default parameters is hardly similar to the first image on this post. But, you can tune the parameters until you get a clearer disparity map. This video shows the use of the application:

Once the parameters are tuned, the disparity map is much better

It is still not perfect, but it is not so bad either.

Now, to use this application with your own couple of images the only thing you need to do is execute the application like this:

./main -left /path/to/my/image/left -right /path/to/my/image/right

And that's it. Please leave a comment if you found this useful, have any problems, questions, suggestions, impressions, etc...
25JKKNMXU6FE