Efficient Data Compression for IoT Devices using Huffman Coding Based Techniques
Amlan Chatterjee, Rushabh Jitendrakumar Shah, Khondker Shajadul Hasan · 2018
In the era of Big Data and Internet of Things, devices ubiquitously sense and gather data at a rapid pace. Various methods have been proposed to speed up the analysis of the data and also mining it for information. However, considering the resource constraints in majority of Internet of Things, this is challenging. The analysis of data can be handled using a massive array of compute nodes; but the data from the devices need to be transferred to the servers to start the process. This involves transmitting the data over the network. Of course, with the huge quantity of data, this requires significant energy and resources to do so. Therefore, in order to address such issues with data analytics using Internet of Things, using data compression techniques is a viable option. Since graphs represent most real world data, including data gathered by Internet of Things, methods to compress graphs have been in the forefront of such endeavors. In this paper we propose techniques to compress graphs by finding specific patterns and replacing those with identifiers that are of variable length, an idea inspired by Huffman Coding. Specifically, given a graph G = (V, E), where V is the set of vertices and E is the set of edges, and |V| = n, we propose methods to reduce the space requirements of the graph by compressing the adjacency matrix representation of the same. The proposed methods show up to 80% reduction is the space required to store the graphs as compared to using the adjacency matrix. The methods can also be applied to other representations as well. The proposed techniques help solve the issues related to transfer of data over the network in case of resource limited Internet of Things, and addresses the challenges of data analytics in this domain.