Sensor fusion and occlusion refinement for tablet-based AR (with Georg Klein)

Tablet PCs offer an alternative to head-mounted displays for delivering augmented reality. This work shows a tablet-based AR application which combines inside-out edge tracking and outside-in LED tracking for robust registration: the inside-out system provides a high level of registration accuracy while the outside-in system provides robustness and recovery from shake and camera occlusions. To provide a high level of rendering quality, we look at the specific case of virtual graphics occluded by real objects for which we have a model. We show that instead of just clipping virtual graphics using the real geometry projected into the z-buffer, an individual treatment and refinement of occluding edges produces a far more convincing integration of real and virtual objects. 

[2004 ISMAR Paper]