Detectable Effect Cluster Analysis: A Novel Machine-Learning Based Clinical Trial Subgroup Analysis Tool
Danielle Beaulieu, Albert A. Taylor, Andrew Conklin, Jonavelle Cuerdo, Dustin Pierce, Mike Keymer, David L. Ennist · 2019
Introduction We previously developed machine-learning based ALS predictive models, including a time to 50% expected VC (VC50) model. Here we report on the use of the VC50 model to develop a novel subgroup analysis tool that we call “Detectable Effect Cluster” (DEC) analysis. Methods The VC50 model was used to rank-order a randomly selected group of 300 PRO-ACT patients (the “trial” population) by predicted log-likelihood. The trial was split 1:1 into “treatment” and “placebo” arms and a simulated 20% slowing in rate of ALSFRS-R progression correlated to creatinine level was applied to the treatment arm. Risk groups using predicted log-likelihood thresholds were defined by systematically expanding subgroups in 2% increments until all participants were included, thus building a series of 1,250 subgroups. To analyze all subgroups, a matrix was plotted in which each block was derived using distinct upper and lower percentile limits. For each subgroup created in this way we performed a statistical analysis, thus developing a “heat map” to reveal combinations of patient subgroups within which statistically significant effect sizes ( p <0.05) were detectable. Results DEC heatmap matrices displaying the results from a simulated clinical trial were plotted for mean square error (MSE), treatment effect (TE), effect size and p -value. The plots revealed differentially stratified patient subgroups characterized by reduced MSE or elevated TE and effect size. The p -value heat map clearly defined a zone, a “hot spot,” comprised of patients in the medium-to low risk group strata with a p -value less than 0.05. Conclusions Detectable Effect Cluster (DEC) analysis shows great promise in identifying subgroups within failed trials that could have formed the basis for successful trials. Importantly this approach allows investigators to explore the possibility of detecting subgroups in which a significant effect size is detectable both via reduced MSE or increased TE. One can also envision an adaptive trial with broad inclusion criteria, the purpose of which is to apply DEC analysis to identify a subgroup with a demonstrable treatment effect. Acknowledgements Data used in this study were obtained from PRO-ACT. This work was partially funded by grants from the ALS Association, and the NIH (NCATS).