Reconstruction from uncalibrated affine silhouettes (with Paul McIlroy)

This work addresses the problem of model building from multiple affine silhouette views of an object in an uncontrolled environment such as an aircraft in flight. Each pair of silhouette views provides two outer epipolar tangency constraints on the relative motion between the cameras. For a scaled orthographic camera model with six degrees of freedom we show that it is possible to recover structure and motion from six or more silhouette views by solving the outer epipolar tangency constraints simultaneously. This work won the Best Student Poster award at BMVC 2009 

[2009 BMVC Paper]
This work addresses the problem of model building from multiple affine silhouette views of an object in an uncontrolled environment such as an aircraft in flight. Each pair of silhouette views provides two outer epipolar tangency constraints on the relative motion between the cameras. For a scaled orthographic camera model with six degrees of freedom we show that it is possible to recover structure and motion from six or more silhouette views by solving the outer epipolar tangency constraints simultaneously.

Fig.1: Reconstruction from silhouettes. Left to right: camera view; silhouette view; reconstructed model as output from a 3D printer in an acrylic-based photopolymer.

Each scaled orthographic camera has six parameters to be determined, three for the rotation relative to the world, two for the image coordinates of the principal point and one for the scale. Thus there are a total of 6N parameters in the N-camera system. The world reference frame may be chosen arbitrarily such that the first camera's rotation matrix is the identity, the principal point is in the centre of the image and it has unit scale. This constrains the world origin to lie on the optical axis of the first camera. By fixing one of the principal point coordinates in a second camera we fully specify the 3D position of the world origin. We have arbitrarily chosen seven of the 6N camera parameters in the system to define the world coordinate frame, leaving (6N-7) free parameters to be solved using the constraints from the outer epipolar tangents between camera pairs. There are ½N(N-1) camera pairs in the system with each camera pair providing two unique constraints from the outer epipolar tangents, so for a unique solution the number of cameras, N, must satisfy (6N-7) ≥ N(N-1). It is therefore possible to recover motion using the epipolar tangency constraint from a minimum of six silhouette views.

Fig.2: Epipolar tangency constraints. Left to right: epipolar tangent planes; residual errors for a single camera pair; tangencies from multiple camera pairs.

The camera motion is obtained by an optimization process which minimises a cost function based on the epipolar tangents. The cost function is the sum of the squared residuals given by the perpendicular distance between the epipolar tangent point on the silhouette outline and the epipolar line corresponding to the epipolar tangent point in the second camera as illustrated in Figure 2. The visual hull associated with each set of silhouettes is recovered by space carving using the camera configuration determined by the optimization. Figure 3 shows the reconstructed model of an SR-71 from high resolution images obtained in a laboratory setting.

Fig.3: Reconstructed SR-71 model.