Accelerating Solution of Generalized Linear Models by Solving Normal Equation Using GPGPU on a Large Real-World Tall-Skinny Data Set
Tran Van Sang, Ryōsuke Kobayashi, Rie Shigetomi Yamaguchi, Toshiyuki Nakata · 2019
The amount of data available has grown rapidly in recently years, not least in the context of Industrialization 4.0 and the advent of Cyber Physical Systems (CPS) and IoT devices. Thus, Machine Learning and Big Data analysis are being taken seriously as promising solutions to cope with the challenges of exponentially growing data. A widely known and widely used approach to quantify the relationship between a dependent variable and multiple numerical predictors is the Generalized Linear Model (GLM). In this paper, we introduce an approach to accelerate the GLM's fitting algorithm. Our approach is involved in two steps. First, reimplement GLM fitting algorithm which applies Normal Equation [1] method when solving the Linear Least Squares Equation. Then, we port the implemented GLM fitting to be executable with GPGPU. When Normal Equation method is applied to tall-skinny data, which is typically found in log data of CPS, solving the post Linear Least Squares equation becomes trivial, and the computational burden is transferred from solving that equation to matrix multiplication which can be parallelized in a straightforward manner. In an experiment employing actual user log access data, the Normal Equation method was executed 1.9 times faster when being executed in CPU, combined with 16.8-fold acceleration by GPGPU, leading to being 31.3 times faster overall.