Controlling the Correctness of Aggregation Operations During Sessions of Interactive Analytic Queries
Eric Simon, Bernd Amann, Rutian Liu, Stéphane Gançarski · Journal of Data and Information Quality · 2023
We present a comprehensive set of conditions and rules to control the correctness of aggregation queries within an interactive data analysis session. The goal is to extend self-service data preparation and Business Intelligence (BI) tools to automatically detect semantically incorrect aggregate queries on analytic tables and views built by using the common analytic operations including filter, project, join, aggregate, union, difference, and pivot. We introduceaggregable propertiesto describe for any attribute of an analytic table, which aggregation functions correctly aggregate the attribute along which sets of dimension attributes. These properties can also be used to formally identify attributes that aresummarizablewith respect to some aggregation function along a given set of dimension attributes. This is particularly helpful to detect incorrect aggregations of measures obtained through the use of non-distributive aggregation functions like average and count. We extend the notion of summarizability by introducing a newgeneralized summarizability conditionto control the aggregation of attributes after any analytic operation. Finally, we definepropagation rulesthat transform aggregable properties of the query input tables into new aggregable properties for the result tables, preserving summarizability and generalized summarizability.