This repository contains code to find disparity and depth map of two image sequences of a given subject by leveraging the concepts of epipolar geometry, Fundamental Matrix, Essential Matrix and its decomposition to get Rotation and Translation matrices, epipolar lines, rectification correspondence using SSD, and triangulation.
- Compare the two images and select the set of matching feature. Tune the Lowe's ration to reject the outliers
- Estimate the Fundamental matrix using the obtained matching feature. Use the RANSAC to make your estimation more robust. Enforce the rank 2 condition for the fundamental matrix.
- Estimate the Essential matrix(E) from the Fundamental matrix(F) and instrinsic camera parameter.
- Decompose the E into a translation T and rotation R
- Disambugiate the T and R using triangulation.
- Apply perspective transfomation to make sure that the epipolar lines are horizontal for both the images. This will limit the search space to horizontal line during the corrospondace matching process in the later stage.
- For each epipolar line, apply the sliding window with SSD to find the corrospondence and calulate disparity.
- Rescale the disparity to be from 0-255 for visualization
- Using the disparity calculated above, compute the depth map. The resultant image has a depth image instead of disparity.
- Python 2.0 or above
- OpenCV
- NumPy
-
Epipolar line corrosponding to the obtained matching features
-
Disparity Map
-
Depth heat map
