Evaluation of Traditional Machine Learning and Mixed Effect Machine Learning Model Performances by Simulation Study
Ebru TURGAL · Turkiye Klinikleri Journal of Biostatistics · 2022
Objective: The aim of this study is to examine the effectiveness of linear mixed-effects (LME) model, one of the traditional models used in the classification of clustered data, and mixed-effects machine learning models, which are the latest approaches. Material and Methods: For the simulation, various data sets were created with different number of groups (250, 500, 1000) and different sample sizes (5000, 10000, 15000). Within the scope of the simulation, LME model, mixed-effects random forest (MERF) and Gaussian process boosting (GPBoost) models were compared in terms of root mean square error (RMSE) on two functions. Results: When the error variance (EV) is 4 for the linear function, sample size is small and the number of groups is high, RMSE of MERF model is smaller. In all other scenarios, RMSE of the linear model was smaller and. In cases where EV for the nonlinear function is 1, the sample size is small and the number of groups is high, RMSE of MERF is smaller. In all other scenarios (while EV was 1), RMSE of GPBoost model was small, p0.05 for MERF. In cases where EV is 4 for the nonlinear function and the sample size and groups is high, RMSE of MERF is smaller. In all other scenarios (while EV was 4), RMSE of GPBoost model was smaller. Conclusion: As a conclusion, for a nonlinear function, GPBoost performed better than MERF and LME methods in terms of RMSE and time. However, when a linear function is considered, LME gives a better result.