Designing ML-based Approximate Query Processing Services on Time-Varying Large Dataset for Distributed Systems
Ki-Hyuk Nam, Sung-Soo Kim, Choon Seo Park, Taekyong Nam, Taewhi Lee · 2022 13th International Conference on Information and Communication Technology Convergence (ICTC) · 2022
Approximate query processing (AQP) has been well established for big data analytics to complement performance degradation due to the ever-increasing size of datasets. The evolution of machine learning technologies creates another opportunity for improving the AQP. There are two architectural aspects that should be considered for AQP-based data analytics services to embrace the trends. First, the services should support rapidly changing data. Second, the systems should manage the life-cycle of the machine learning process and accommodate the diversity of ML technologies. This paper discusses the requirements and design considerations for ML-based AQP services on time-varying large datasets for distributed environments in a system-independent way.