Meet Charles, big data query advisor
Thibault Sellam, Martin L. Kersten · Data Archiving and Networked Services (DANS) · 2013
In scientific data management and business analytics, the most in-formative queries are a holy grail. Data collection becomes in-creasingly simpler, yet data exploration gets significantly harder. Exploratory querying is likely to return an empty or an overwhelm-ing result set. On the other hand, data mining algorithms require extensive preparation, ample time and do not scale well. In this paper, we address this challenge at its core, i.e., how to query the query space associated with a given database. The space con-sidered is formed by conjunctive predicates. To express them, we introduce the Segmentation Description Language (SDL). The user provides a query. Charles, our query advisory system, breaks its extent into meaningful segments and returns the subsequent SDL descriptions. This provides insight into the set described and offers the user directions for further exploration. We introduce a novel algorithm to generate SDL answers. We eval-uate them using four orthogonal criteria: homogeneity, simplicity, breadth, and entropy. A prototype implementation has been con-structed and the landscape of follow-up research is sketched. 1.