On discovering semantics of user-defined functions in data processing workflows
Michał Bodziony, Hubert Krzyżanowski, Łukasz Pięta, Robert Wrembel · 2021
User defined functions (UDFs) are frequent components of SQL queries and data processing workflows (DPWs). In both of these applications, UDFs are typically treated as black boxes, i.e., their semantics and performance characteristics are unknown, which prevents from optimizing execution plans of queries and DPWs. In this paper, we present an initial alternative solution for learning the semantics of simple UDFs, followed by its experimental evaluation. Our semantics discovery method is based on three steps: (1) discovering the type of an operation that a UDF implements, (2) discovering the types of restrictions on attributes and logical operators that combine the restrictions, and (3) representing the semantics of an original code of a UDF.