A Toolkit for Tracking and Mediating Parametric Objects upon Commodity Mobile Devices
Shantanu Thatte · 2014
The large number of mobile devices introduced in the market in recent years provide numerous interaction opportunities. We used commercially available products to create an interface for interaction with diverse datasets. We extended this scenario to include multiple physical instances, realizing a real-time multi-party interaction session. Tangibles act as tools and containers, in a fashion that requires their physical location and orientation to be evaluated continuously during a session. We present a technique to detect an object using only the touchscreen of a mobile device as an input, by means of touch patterns created on the screen. This helps avoid additional embedded electronics and allows an object to be used as a tangible. We also conducted various other experiments using sensors available on mobile devices to detect objects. We also discuss the architecture and implementation of a low latency backend to relay and propagate events to all participating devices. This includes a low latency memory based key-value store implemented using Redis, and persistent storage using MongoDB. The server side scripts are implemented using the fast non-blocking Node.js Javascript engine to minimize impact on latency imparted during data processing.