Novel View Synthesis and Augmented Reality for Assisting Human Action Learning
Fabian Lorenzo Dayrit · Institutional Repositories DataBase (IRDB) · 2017
When people wish to learn an action, such as in sports or in dance, for example, the most common way to do so is by imitating someone else performing the action.This can take one of two forms: either the learner observes a real, in-person teacher, or the learner watches a recording of the teacher performing the action.In-person observation allows the learner to view the action from any point of view, but it requires the teacher to be there.The video may be watched separate from the teacher, but it is limited to the original capturing point of view.We want to combine the advantages of these two by creating a new way to view such actions.This study revolves around capturing human actions using depth cameras and rendering the actions from a novel viewpoint, focusing on the motion of the actions and not on the location or context.We call such novel views of actions, reenactments.We wish to use reenactments to help users comprehend and learn actions.We explore practical ways of capturing and rendering reenactments that may be done using consumer depth cameras at home.The challenge is in adequately representing unseen areas and in defining consistent correspondences on the subject's body across the motion sequence.This thesis proposes two methods to represent reenactments: by a set of rigid body parts, and by a deformable statistical body model.For both of these methods, we have implemented an application and have conducted a user study to evaluate the reenactment quality, as well as the application's effectiveness, ease of use, and appeal.