Database and Caching Support for Adaptive Visualization of Large Sensor Data
Sapan Tanted, Anshul Agarwal, Shinjan Mitra, Chaitra Bahuman, Krithi Ramamritham · 2020
Rapid deployment of Internet of Things (IoT) has led to ubiquitous and pervasive sensing of objects in the physical world, such as artifacts in buildings, agriculture, cities, the electric grid, etc. Meaningful visualization of large amounts of sensor data demands user-friendly, convenient and flexible tools. In this paper, we discuss the design, implementation and performance of a novel distributed caching & aggregation mechanism to handle the visualization of sensor data, which is time series data. Its features include a) bitmap indexing for capturing the dynamics of the cached data b) exploiting recency of data usage when making cache insertion and replacement decisions and c) integrating existing databases and open-source visualization platforms to provide quick and effective distributed caching solutions to handle time-series data. We evaluate our system on real-world data generated by sensors deployed in an academic building and demonstrate empirically that the system adapts to evolving workload patterns and makes it attractive for a variety of workloads.