Efficiently Exploring Multilevel Data with Recursive Partitioning
Daniel Patrick Martin · 2015
1 introduction 1 2 recursive partitioning and ensemble methods 5 2.1 Classification and Regression Trees 6 2.1.1 Understanding the bias-variance tradeoff 7 2.1.2 Pruning decision trees with cross-validation 9 2.1.3 CART with a categorical outcome 10 2.1.4 Pros and cons of CART 12 2.2 Conditional Inference Trees 13 2.2.1 Pros and cons of conditional inference trees 16 2.3 Random Forests 18 2.3.1 Out-of-bag samples 18 2.3.2 Variable importance 19 2.3.3 Partial dependence plots 19 2.3.4 Conditional inference forests 22 2.3.5 Pros and cons of random forests 23 2.4 Handling Missing Data 24 3 recursive partitioning and multilevel data 27 3.1 Previous Research 27 3.2 Current Problems 30 3.2.1 Multilevel issues with CART 30 3.2.2 Multilevel issues with conditional inference 31 3.2.3 Multilevel issues with forests 32 4 simulation phase 33 4.1 Statistical Techniques and Implementation 33 4.1.1 Classification and regression trees (CART) 33 4.1.2 Conditional inference trees (CTREE) 34 4.1.3 Random forests using classification and regression trees (CART forest) 34 4.1.4 Random forests using conditional inference trees (CFOREST) 34 4.1.5 Multilevel regression 35 4.2 Simulation Conditions 35 4.3 Data Generation 37 4.4 Evaluation Criteria 39 4.4.1 Proportion variation explained 39 4.4.2 Variable importance 40 4.5 Simulation Results 40 4.5.1 Proportion variation results 40 4.5.2 Variable importance results 41 4.5.3 Special conditions results 44 4.6 Discussion 45 5 application phase 49 v vi contents 5.1 High School and Beyond Survey 49 5.1.1 Step 1: ICC 50 5.1.2 Step 2: Estimate proportion of variation explained 5.1.3 Step 3: Examine variable importance and predicted value plots 50 5.1.4 Conclusion 53 5.2 Responsive Classroom Efficacy Study 54 5.2.1 Initial missingness step 58 5.2.2 Step 1: ICC 58 5.2.3 Step 2: Estimate proportion of variation explained 5.2.4 Step 3: Examine variable importance and predicted value plots 59 5.2.5 Conclusion 61 5.3 My Teaching Partner-Secondary Study 63 5.3.1 Initial missingness step 66 5.3.2 Step 1: ICC 66 5.3.3 Step 2: Estimate proportion of variation explained 5.3.4 Step 3: Examine variable importance and predicted value plots 67 5.3.5 Conclusion 69 6 dissemination phase 73 6.1 R Package 73 6.2 Workshop 75 7 general discussion 81 7.1 Simulation Phase 81 7.2 Application Phase 81 7.3 Dissemination Phase 82 7.4 Limitations and Future Directions 82 7.5 Conclusion 84 a appendix 85 a.1 Proper Calculation of the ICC for a Multilevel Simulation 85 a.2 Extra Plots for the High School and Beyond Survey appendix