Parallel, distributed, and differential processing system for human activity sensing flows
Takamichi Toda, Sozo Inoue, Lin Li · 2013
In this paper, we propose a parallel distributed processing system for data-analytic project including human activity sensing flows, which manages dependency among data and analytic programs, and re-execute updated programs and dependent programs for updated data/programs. In the system, a data analyzer can specify the dependency and parts for requiring distributed parallel processing using Hadoop Streaming, and they can be processed only for updated and the dependent part, with flexibly selecting parallel or sequential execution on the fly. The specification can also specify repeated execution of a single program with different data, while their dependencies are checked separately at execution. We describe the mathematical model, the system design, the usage, and the experimental result applying to the essential process in human activity sensing.