Unsupervised Workflow Extraction from First-Person Video of Mechanical Assembly
Truong-An Pham, Yu Xiao · 2018
Recently, Augmented Reality (AR) applications have proved to help improve the efficiency in accomplishing assembly tasks. However, due to the lack of approaches to automatic workflow extraction, the existing AR-based assembly assistance applications require manual authoring, which hampers scalability. Moreover, most of these applications only support information visualization and video documentation. To tackle the challenge of scalability and to enable more intelligent functionalities, such as real-time quality control, we propose in this paper a novel solution for unsupervised workflow extraction from first-person video of mechanical assembly, without any pre-labeled training data or pre-trained classifiers. Our proposed system automatically discovers a sequence of operations from the input video, and describes the extracted workflow process with semantics. Preliminary evaluation demonstrates the feasibility of our solution and highlights the technical challenges.