Comparing algorithms for characterizing treatment effect heterogeneity in randomized trials.
benchmarking
machine learning
simulation
subgroup analysis
subgroup identification
treatment effect heterogeneity
Journal
Biometrical journal. Biometrische Zeitschrift
ISSN: 1521-4036
Titre abrégé: Biom J
Pays: Germany
ID NLM: 7708048
Informations de publication
Date de publication:
27 Nov 2022
27 Nov 2022
Historique:
revised:
04
10
2022
received:
25
10
2021
accepted:
16
10
2022
entrez:
27
11
2022
pubmed:
28
11
2022
medline:
28
11
2022
Statut:
aheadofprint
Résumé
The identification and estimation of heterogeneous treatment effects in biomedical clinical trials are challenging, because trials are typically planned to assess the treatment effect in the overall trial population. Nevertheless, the identification of how the treatment effect may vary across subgroups is of major importance for drug development. In this work, we review some existing simulation work and perform a simulation study to evaluate recent methods for identifying and estimating the heterogeneous treatments effects using various metrics and scenarios relevant for drug development. Our focus is not only on a comparison of the methods in general, but on how well these methods perform in simulation scenarios that reflect real clinical trials. We provide the R package benchtm that can be used to simulate synthetic biomarker distributions based on real clinical trial data and to create interpretable scenarios to benchmark methods for identification and estimation of treatment effect heterogeneity.
Identifiants
pubmed: 36437036
doi: 10.1002/bimj.202100337
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2022 Wiley-VCH GmbH.
Références
Alemayehu, D., Chen, Y., & Markatou, M. (2018). A comparative study of subgroup identification methods for differential treatment effect: Performance metrics and recommendations. Statistical Methods in Medical Research, 27(12), 3658-3678.
Alosh, M., Huque, M. F., Bretz, F., & D'Agostino Sr, R. B. (2017). Tutorial on statistical considerations on subgroup analysis in confirmatory clinical trials. Statistics in Medicine, 36(8), 1334-1360.
Athey, S., & Imbens, G. (2016). Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27), 7353-7360.
Athey, S., Tibshirani, J., & Wager, S. (2019). Generalized random forests. The Annals of Statistics, 47(2), 1148-1178.
Athey, S., & Wager, S. (2019). Estimating treatment effects with causal forests: An application. Observational Studies, 5(2), 37-51.
Battioui, C., Shen, L., & Ruberg, S. J. (2014). A resampling-based ensemble tree method to identify patient subgroups with enhanced treatment effect. Proceedings of 2014 joint statistical meetings. American Statistical Association, Alexandria, VA, USA, pp. 4013-4023.
Boulesteix, A.-L., Groenwold, R. H., Abrahamowicz, M., Binder, H., Briel, M., Hornung, R., Morris, T. P., Rahnenführer, J., & Sauerbrei, W. (2020a). Introduction to statistical simulations in health research. BMJ Open, 10(12), e039921.
Boulesteix, A.-L., Hoffmann, S., Charlton, A., & Seibold, H. (2020b). A replication crisis in methodological research? Significance, 17(5), 18-21.
Boulesteix, A.-L., Lauer, S., & Eugster, M. J. (2013). A plea for neutral comparison studies in computational sciences. PLoS One, 8(4), e61562.
Chen, G., Zhong, H., Belousov, A., & Devanarayan, V. (2015). A prim approach to predictive-signature development for patient stratification. Statistics in Medicine, 34(2), 317-342. https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.6343
Chen, J., & Hsiang, C.-W. (2019). Causal random forests model using instrumental variable quantile regression. Econometrics, 7(4), 49.
Chen, S., Tian, L., Cai, T., & Yu, M. (2017). A general statistical framework for subgroup identification and comparative treatment scoring. Biometrics, 73(4), 1199-1209.
Chernozhukov, V., Demirer, M., Duflo, E., & Fernández-Val, I. (2018). Generic machine learning inference on heterogeneous treatment effects in randomized experiments, with an application to immunization in India (No. w24678). National Bureau of Economic Research.
Cole, S. R., Edwards, J. K., & Greenland, S. (2021). Surprise! American Journal of Epidemiology, 190(2), 191-193. https://doi.org/10.1093/aje/kwaa136
Dmitrienko, A., Muysers, C., Fritsch, A., & Lipkovich, I. (2016). General guidance on exploratory and confirmatory subgroup analysis in late-stage clinical trials. Journal of Biopharmaceutical Statistics, 26(1), 71-98.
Doove, L. L., Dusseldorp, E., Van Deun, K., & Van Mechelen, I. (2014). A comparison of five recursive partitioning methods to find person subgroups involved in meaningful treatment-subgroup interactions. Advances in Data Analysis and Classification, 8(4), 403-425.
Dusseldorp, E., Conversano, C., & Van Os, B. J. (2010). Combining an additive and tree-based regression model simultaneously: Stima. Journal of Computational and Graphical Statistics, 19(3), 514-530.
Dusseldorp, E., & Van Mechelen, I. (2014). Qualitative interaction trees: A tool to identify qualitative treatment-subgroup interactions. Statistics in Medicine, 33(2), 219-237.
Egami, N., & Imai, K. (2018). Causal interaction in factorial experiments: Application to conjoint analysis. Journal of the American Statistical Association, 114(526), 529-540.
European Medicines Agency. (2019). Guideline on the investigation of subgroups in confirmatory clinical trials. EMA/CHMP/539146.
Foster, J. C., Taylor, J. M., & Ruberg, S. J. (2011). Subgroup identification from randomized clinical trial data. Statistics in Medicine, 30(24), 2867-2880.
Friedman, J., & Fisher, N. I. (1999). Bump hunting in high-dimensional data. Statistics and Computing, 9(2), 123-143.
Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1.
Friedman, J., Hastie, T., Tibshirani, R., Narasimhan, B., Tay, K., Simon, N., & Qian, J. (2021). Package ‘glmnet'. CRAN R Repositary.
Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning, (Vol. 1). Springer Series in Statistics.
Fu, H., Zhou, J., & Faries, D. E. (2016). Estimating optimal treatment regimes via subgroup identification in randomized control trials and observational studies. Statistics in Medicine, 35(19), 3285-3302.
Garge, N. R., Bobashev, G., & Eggleston, B. (2013). Random forest methodology for model-based recursive partitioning: The mobforest package for R. BMC Bioinformatics, 14(1), 1-8.
Gewandter, J. S., McDermott, M. P., He, H., Gao, S., Cai, X., Farrar, J. T., Katz, N. P., Markman, J. D., Senn, S., Turk, D. C., & Dworkin, R. H. (2019). Demonstrating heterogeneity of treatment effects among patients: An overlooked but important step toward precision medicine. Clinical Pharmacology & Therapeutics, 106(1), 204-210.
Harrell, F., & Slaughter, J. C. (2021). Biostatistics for biomedical research. http://hbiostat.org/doc/bbr.pdf (Updated February 6, 2021)
Hastie, T., & Qian, J. (2014). Glmnet vignette. Retrieved June, 9(2016), 1-30.
Hothorn, T., & Zeileis, A. (2015). partykit: A modular toolkit for recursive partytioning in R. The Journal of Machine Learning Research, 16(1), 3905-3909.
Huang, X., Sun, Y., Trow, P., Chatterjee, S., Chakravartty, A., Tian, L., & Devanarayan, V. (2017). Patient subgroup identification for clinical drug development. Statistics in Medicine, 36(9), 1414-1428.
Huber, C., Benda, N., & Friede, T. (2019). A comparison of subgroup identification methods in clinical drug development: Simulation study and regulatory considerations. Pharmaceutical Statistics, 18(5), 600-626.
Huling, J. D., & Yu, M. (2021). Subgroup identification using the personalized package. Journal of Statistical Software, 98, 1-60.
Imai, K., & Ratkovic, M. (2013). Estimating treatment effect heterogeneity in randomized program evaluation. The Annals of Applied Statistics, 7(1), 443-470.
Krzykalla, J., Benner, A., & Kopp-Schneider, A. (2020). Exploratory identification of predictive biomarkers in randomized trials with normal endpoints. Statistics in Medicine, 39(7), 923-939.
Laber, E. B., & Zhao, Y.-Q. (2015). Tree-based methods for individualized treatment regimes. Biometrika, 102(3), 501-514.
Levy, J., van der Laan, M., Hubbard, A., & Pirracchio, R. (2021). A fundamental measure of treatment effect heterogeneity. Journal of Causal Inference, 9(1), 83-108. https://doi.org/10.1515/jci-2019-0003
Lipkovich, I., Dmitrienko, A., & D'Agostini, R. B. (2017). Tutorial in biostatistics: Data-driven subgroup identification and analysis in clinical trials. Statistics in Medicine, 36(1), 136-196. https://doi.org/10.1002/sim.7064
Lipkovich, I., Dmitrienko, A., Denne, J., & Enas, G. (2011). Subgroup identification based on differential effect search-A recursive partitioning method for establishing response to treatment in patient subpopulations. Statistics in Medicine, 30(21), 2601-2621.
Liu, Y., Tang, S.-Y., Man, M., Li, Y. G., Ruberg, S. J., Kaizar, E., & Hsu, J. C. (2016). Thresholding of a continuous companion diagnostic test confident of efficacy in targeted population. Statistics in Biopharmaceutical Research, 8(3), 325-333.
Loh, W.-Y., Cao, L., & Zhou, P. (2019). Subgroup identification for precision medicine: A comparative review of 13 methods. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(5), e1326.
Loh, W.-Y., & Zhou, P. (2020). The guide approach to subgroup identification. In: Ting, N., Cappelleri, J., Ho, S., Chen, G. (eds) Design and analysis of subgroups with biopharmaceutical applications, (pp. 147-165). Springer, Cham.
Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074-2102.
Muysers, C., Dmitrienko, A., Kulmann, H., Kirsch, B., Lippert, S., Schmelter, T., Schulz, A., Mentenich, N., Schmitz, H., Schaefers, M., Meinhardt, G., Keil, T., & Roll, S. (2020). A systematic approach for post hoc subgroup analyses with applications in clinical case studies. Therapeutic Innovation & Regulatory Science, 54(3), 507-518.
Natekin, A., & Knoll, A. (2013). Gradient boosting machines, a tutorial. Frontiers in Neurorobotics, 7, 21.
Nowok, B., Raab, G. M., & Dibben, C. (2016). synthpop: Bespoke creation of synthetic data in R. Journal of Statistical Software, 74(11), 1-26.
Patel, S., Hee, S. W., Mistry, D., Jordan, J., Brown, S., Dritsaki, M., Ellard, D. R., Friede, T., Lamb, S. E., Lord, J., Madan, J., Morris, T., Stallard, N., Tysall, C., Willis, A., Underwood, M., & the Repository Group (2016). Identifying back pain subgroups: Developing and applying approaches using individual patient data collected within clinical trials. Programme Grants for Applied Research, 4(10), 1-278.
Personalized Medicine Coalition. (2021). Personalized medicine at FDA the scope and significance of progress in 2020. https://www.personalizedmedicinecoalition.org/Userfiles/PMC-Corporate/file/PM_at_FDA_The_Scope_Significance_of_Progress_in_2020.pdf
Qian, M., & Murphy, S. A. (2011). Performance guarantees for individualized treatment rules. Annals of Statistics, 39(2), 1180.
Royston, P., & Sauerbrei, W. (2013). Interaction of treatment with a continuous variable: Simulation study of significance level for several methods of analysis. Statistics in Medicine, 32(22), 3788-3803.
Royston, P., & Sauerbrei, W. (2014). Interaction of treatment with a continuous variable: Simulation study of power for several methods of analysis. Statistics in medicine, 33(27), 4695-4708.
Ruberg, S. J. (2021). Assessing and communicating heterogeneity of treatment effects for patient subpopulations: The hardest problem there is. Pharmaceutical Statistics, 20(5), 939-944.
Ruberg, S. J., & Shen, L. (2015). Personalized medicine: Four perspectives of tailored medicine. Statistics in Biopharmaceutical Research, 7, 214-229.
Sauerbrei, W., & Royston, P. (2022). Investigating treatment-effect modification by a continuous covariate in IPD meta-analysis: An approach using fractional polynomials. BMC Medical Research Methodology, 22(1), 1-13.
Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C. H., Devasenapathy, N., Hayward, R. A., Gagnier, J., Borenstein, M., van der Heijden, G. J., Dahabreh, I. J., Sun, X., Sauerbrei, W., Walsh, M., Ioannidis, J. P. A., Thabane, L., & Guyatt, G. H. (2020). Development of the instrument to assess the credibility of effect modification analyses (iceman) in randomized controlled trials and meta-analyses. CMAJ, 192(32), E901-E906.
Schandelmaier, S., Chang, Y., Devasenapathy, N., Devji, T., Kwong, J. S., Colunga Lozano, L. E., Lee, Y., Agarwal, A., Bhatnagar, N., Ewald, H., Zhang, Y., Sun, X., Thabane, L., Walsh, M., Briel, M., & Guyatt, G. H. (2019). A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses. Journal of Clinical Epidemiology, 113, 159-167. https://www.sciencedirect.com/science/article/pii/S0895435618308576
Sechidis, K., Papangelou, K., Metcalfe, P. D., Svensson, D., Weatherall, J., & Brown, G. (2018). Distinguishing prognostic and predictive biomarkers: An information theoretic approach. Bioinformatics, 34(19), 3365-3376.
Seibold, H., Zeileis, A., & Hothorn, T. (2016). Model-based recursive partitioning for subgroup analyses. International Journal of Biostatistics, 12(1), 45-63.
Song, F., & Bachmann, M. O. (2016). Cumulative subgroup analysis to reduce waste in clinical research for individualised medicine. BMC Medicine, 14, 197.
Strobl, C., Boulesteix, A.-L., Zeileis, A., & Hothorn, T. (2007). Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1), 25.
Su, X., Zhou, T., Yan, X., Fan, J., & Yang, S. (2008). Interaction trees with censored survival data. The International Journal of Biostatistics, 4(1), 1-26.
Sun, S., Bornkamp, B., Sechidis, K., Mirshani, A., Lu, J., & Chen, Y. (2022). R package benchtm (Version 1.2.0) [Computer software]. https://github.com/Sophie-Sun/benchtm
Sur, P., Chen, Y., & Candès, E. J. (2019). The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled chi-square. Probability Theory and Related Fields, 175(1), 487-558.
Thomas, M., Bornkamp, B., & Seibold, H. (2018). Subgroup identification in dose-finding trials via model-based recursive partitioning. Statistics in Medicine, 37(10), 1608-1624. https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.7594
Tian, L., Alizadeh, A. A., Gentles, A. J., & Tibshirani, R. (2014). A simple method for estimating interactions between a treatment and a large number of covariates. Journal of the American Statistical Association, 109(508), 1517-1532.
Tian, L., & Tibshirani, R. (2011). Adaptive index models for marker-based risk stratification. Biostatistics, 12(1), 68-86.
Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288.
Tong, C. (2019). Statistical inference enables bad science; Statistical thinking enables good science. The American Statistician, 73(sup1), 246-261.
VanderWeele, T. J., Luedtke, A. R., van der Laan, M. J., & Kessler, R. C. (2019). Selecting optimal subgroups for treatment using many covariates. Epidemiology (Cambridge, MA), 30(3), 334.
Wang, L. (2005). Support vector machines: Theory and applications, (Vol. 177). Springer Science & Business Media.
Watson, J. A., & Holmes, C. C. (2020). Machine learning analysis plans for randomised controlled trials: Detecting treatment effect heterogeneity with strict control of type I error. Trials, 21(1), 156.
Xu, Y., Yu, M., Zhao, Y.-Q., Li, Q., Wang, S., & Shao, J. (2015). Regularized outcome weighted subgroup identification for differential treatment effects. Biometrics, 71(3), 645-653.
Zhao, L., Tian, L., Cai, T., Claggett, B., & Wei, L.-J. (2013). Effectively selecting a target population for a future comparative study. Journal of the American Statistical Association, 108(502), 527-539.
Zink, R. C., Shen, L., Wolfinger, R. D., & Showalter, H. (2015). Assessment of methods to identify patient subgroups with enhanced treatment response in randomized clinical trials. In: Chen, Z., Liu, A., Qu, Y., Tang, L., Ting, N., Tsong, Y. (eds). Applied statistics in biomedicine and clinical trials design, (pp. 395-410). Springer.