‎"Behind every stack of books there is a flood of knowledge."

Projective Geometry for Image Analysis


3D Reconstruction from Multiple Images

Suppose that a fixed scene is seen by two or more perspective cameras. We are interested in geometric issues, so we will suppose that the correspondences between visible points in different images are already known. However it must be pointed out that matching is a fundamental and extremely difficult issue in vision, which can not be dismissed so lightly in practice.

So in this section, we suppose that n 3D points Ai are observed by m cameras with projection matrices $P_{j}, j= 1,\ldots,m$. Neither the point positions nor the camera projections are given. Only the projections aij of the ith point in the jthimage are known.

Projective Reconstruction

Simple parameter counting shows that we have 2nm independent measurements and only 11 m + 3 n unknowns, so with enough points and images the problem ought to be soluble. However, the solution can never be unique as we always have the freedom to change the 3D coordinate system we use. In fact, in homogeneous coordinates the equations become


a_{ij} \sim P_{j} A_{i} \qquad i=1, \ldots n,~~j=1, \ldots,m
\end{displaymath} (5.4)



So we always have the freedom to apply a nonsingular $4\times 4$transformation H to both projections $P_j \to P_j H^{-1}$ and world points $A_i\to H A_i$. Hence, without further constraints, reconstruction is only ever possible up to an unknown projective deformation of the 3D world. However, modulo this fundamental ambiguity, the solution is in general unique.

One simple way to obtain the solution is to work in a projective basis tied to the 3D points [3]. Five of the visible points (no four of them coplanar) can be selected for this purpose.



Exercise 5.1   : Given the epipolar geometry, show how we can decide whether four points are coplanar or not, by just considering their images in a stereo pair. Hint: consider the intersection of a pair of image lines, each linking two of the four points.


An alternative to this is to select the projection center of the first camera as the coordinate origin, the projection center of the second camera as the unit point, and complete the basis with three other visible 3D points A1A2A3 such that no four of the five points are coplanar.



Exercise 5.2   : Design an image-based test to check whether three points are coplanar with the center of projection. Derive a test that checks that two points are not coplanar with the base line of a stereo pair, assuming that the epipolar geometry is known. Deduce a straightforward test to check that the above five points form a valid 3D projective basis.


Let a1a2a3 and a1a2a3 respectively be the projections in image 1 and image 2 of the 3D points A1A2A3. Make a projective transformation of each image so that these three points and the epipoles become a standard basis:


\begin{displaymath}a_1 = a'_1 =
1 \\ 0 \\ 0
e = e' = \left(\begin{array}{c} 1\\ 1\\ 1 \end{array}\right)



Also fix the 3D coordinates of A1A2A3 to be respectively


\begin{displaymath}A_1 = \left(\begin{array}{c} 1\\ 0\\ 0\\ 0 \end{array}\right)...
A_3 = \left(\begin{array}{c} 0\\ 0\\ 1\\ 0 \end{array}\right)



It follows that the two projection matrices can be written:


\begin{displaymath}P = \left(
1 & 0 & 0 & 0 \\
0 & 1 & 0 & ...
...& -1 \\
0 & 1 & 0 & -1 \\
0 & 0 & 1 & -1
\end{array} \right)





Exercise 5.3   : Show that the projections have these forms. (NB: Given only the projections of A1,A2,A3, each row of P,P‘ could have a different scale factor since point projections are only defined up to scale. It is the projections of the epipoles that fix these scale factors to be equal).


Since the projection matrices are now known, 3D reconstruction is relatively straightforward. This is just a simple, tutorial example so we will not bother to work out the details. In any case, for precise results, a least squares fit has to be obtained starting from this initial algebraic solution (e.g. by bundle adjustment).


Affine Reconstruction

Section 4.1 described the advantages of recovering affine space and provided some methods of computing the location of the plane at infinity $\Pi_{\infty}$. The easiest way to proceed is to use prior information, for instance the knowledge that lines in the scene are parallel or that a point is the half way between two others.

Prior constraints on the camera motion can also be used. For example, a translating camera is equivalent to a translating scene. Observing different images of the same point gives a line in the direction of motion. Intersecting several of these lines gives the point at infinity in the motion direction, and hence one constraint on the affine structure.

On the other hand, any line through two scene points translates into a line parallel to itself, and the intersection of these two lines gives further constraints on the affine structure. Given three such point pairs we have in all four points at infinity, and the projective reconstruction of these allows the ideal points to be recovered — see [16] for details and experimental results.


Euclidean Reconstruction

We have not implemented the specific suggestions of section 4.2 for the recovery of Euclidean structure from a projective reconstruction by observing known scene angles, circles,…However we have developed a more brute force approach, finding a projective transformation H in equation (5.4) that maps the projective reconstruction to one that satisfies a set of redundant Euclidean constraints. This simultaneously minimizes the error in the projection equations and the violation of the constraints. The equations are highly nonlinear and a good initial guess for the structure is needed. In our experiments this was obtained by assuming parallel projection, which is linear and allows easy reconstruction using SVD decomposition ([26]).

Figures 5.3 and 5.4 show an example of this process.


Figure 5.3: The house scene: the three images used for reconstruction together with the extracted corners. The reference frame used for the reconstruction is defined by the five points marked with white disks in image 3




Figure 5.4: Euclidean reconstruction of an indoor scene using the known relative positions of five points. To make the results easier to see, the reconstructed points are joined with segments.
\centerline{general view}



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Virtual Fashion Technology

Virtual Fashion Education


"chúng tôi chỉ là tôi tớ của anh em, vì Đức Kitô" (2Cr 4,5b)


News About Tech, Money and Innovation


Modern art using the GPU

Theme Showcase

Find the perfect theme for your blog.


Learn to Learn

Gocomay's Blog

Con tằm đến thác vẫn còn vương tơ

Toán cho Vật lý

Khoa Vật lý, Đại học Sư phạm Tp.HCM - ĐT :(08)-38352020 - 109

Maths 4 Physics & more...

Blog Toán Cao Cấp (M4Ps)

Bucket List Publications

Indulge- Travel, Adventure, & New Experiences


‎"Behind every stack of books there is a flood of knowledge."

The Blog

The latest news on and the WordPress community.

%d bloggers like this: