Development and Improvement of Optical Tracking Methods towards Registering the Deformations of 3D Non- Rigid Bodies in Real Time for Augmented Reality Applications

Abstract

Augmented Reality (AR) is a technology that aims to embed virtual objects in the real world, showing to the user the set of objects (virtual and real) as a single world. For that purpose, it is necessary to offer a perfect alignment between virtual and real objects, which increases the effectiveness of AR. The solution to this problem is known as tracking. The object tracking consists in determining at any time the position and orientation of the camera relative to the scene. Optical sensors are most commonly used to overcome the tracking problem due to their low cost implementation. However, it is often difficult to provide robustness, accuracy and low computational cost at the same time. This thesis tackles the improvement and development of the main optical tracking techniques, primarily focused on detecting the deformations of the bodies. First, it has been achieved the tracking of rigid and non-rigid planar surfaces through a monocular camera, and then, the object deformation estimation with a more complex device as a RGB-D camera has been developed. Surface tracking systems such as those based on markers have the problem of not being able to handle occlusions. Thus, this thesis raises a new marker design that offers robustness against occlusions. Furthermore, in order to handle the deformations of surfaces, a solution that recovers the camera pose and the non-rigid surface simultaneously is proposed. Continuing with the deformation handling, it has also developed a robust tracking system for reconstructing the 3D shape of deformable objects using two different physical formulations. One offers a correct physical behaviour with a low computational cost, whereas the other achieves higher levels of accuracy at the expense of higher processing time. In addition, all the presented solutions have the common factor that all are executed in real time, which is a key property for a fluently visual feedback of an AR application.