Context-Aware Telemetry Data Compression for Cost Optimization in Distributed Systems
Faiz Gouri, Chaitanya Krishnama, Raghavender Puchhakayala, Sudarshan Kotha · 2025
Telemetry data generated by distributed systems, such as cloud infrastructure and loT networks, is growing exponentially, leading to significant storage and processing costs. Traditional compression techniques apply a uniform approach, often compromising data fidelity or failing to optimize costs effectively. This paper proposes a context-aware telemetry data compression framework that dynamically classifies and compresses data based on its priority, usage patterns, and business impact. By leveraging machine learning for data classification and adaptive compression algorithms, the framework ensures that critical data is preserved with high fidelity while aggressively compressing non-critical data to reduce costs. Experimental results demonstrate a 40% reduction in storage costs and a 25% improvement in query performance for critical data. The proposed solution is scalable, cost-effective, and suitable for realtime telemetry systems.