IMBridge: Impedance Mismatch Mitigation between Database Engine and Prediction Query Execution
Chenyang Zhang, Junxiong Peng, Chen Xu, Quanqing Xu, Chuanhui Yang · 2024
Prediction queries that apply machine learning (ML) models to perform analysis on data stored in the database are prevalent with the advance of research. Thanks to the prosperity of ML frameworks in Python, current database systems introduce Python UDFs into query engines for inference invocation. However, there are impedance mismatches between database engines and prediction query execution with this approach. In particular, the database engine is oblivious to the semantics within prediction functions, which incurs the repetitive inference context setup. Moreover, the evaluation of the prediction function is coupled with the operator, which results in an undesirable inference batch size with low inference throughput. To mitigate these, we propose a system called IMBridge, which leverages aprediction function rewriter to eliminate redundant inference context setup and introduces adecoupled prediction operator to ensure that the evaluation batch size matches the desirable inference batch size. In this demonstration, we will showcase how IMBridge addresses these mismatches and boosts prediction query execution.