ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition (with Qi Pan and Gerhard Reitmayr)

The generation of 3D models of real objects is very useful for many computer vision applications. This paper introduces ProFORMA, a system designed to enable on-line reconstruction of textured 3D objects rotated by a user's hand. Partial models are created very rapidly and displayed to the user to aid view planning, as well as used by the system to robustly track the object pose. The system works by calculating the Delaunay tetrahedralisation of a point cloud obtained from on-line structure from motion estimation which is then carved using a recursive probabilistic algorithm to rapidly obtain the surface mesh. This work won the Best Demo prize at ISMAR 2009.

[2009 BMVC Paper]

Off-line model reconstruction relies on an image collection phase and a slow reconstruction phase, requiring a long time to verify a model obtained from an image sequence is acceptable. We propose a new model acquisition system, called ProFORMA, which generates a 3D model on-line as the input sequence is being collected. As the user rotates the object in front of a stationary camera, a partial model is reconstructed and displayed to the user to assist view planning. The model is also used by the system to robustly track the pose of the object. Models are rapidly produced through a Delaunay tetrahedralisation of points obtained from on-line structure from motion estimation, followed by a probabilistic tetrahedron carving step to obtain a textured surface mesh of the object.







Figure 1: Left to right (a) Object rotated by hand in front of camera. (b) Point cloud obtained from on-line structure from motion estimation followed by bundle adjustment. (c) Delaunay Tetrahedralisation of point cloud, partitioning the convex hull into tetrahedra. (d) Carved mesh obtained from recursive probabilisitic tetrahedron carving. (e) Texture-mapped surface mesh.

Figure 2: Results of reconstruction for various objects.

<







Acknowledgements 

I am very grateful to Tom Drummond, Gerhard Reitmayr and Ed Rosten for their support, guidance and interesting discussions, as well as various fragments of code. Many thanks to Ethan Eade for his bundle adjustment and five-point code, without which the first version of this system would not have been possible.