Estimating individualized treatment effects using an individual participant data meta-analysis.
Individual patient data
Individualized treatment effects
Meta-analysis
Personalized medicine
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
25 Mar 2024
25 Mar 2024
Historique:
received:
13
02
2023
accepted:
15
03
2024
medline:
26
3
2024
pubmed:
26
3
2024
entrez:
26
3
2024
Statut:
epublish
Résumé
One key aspect of personalized medicine is to identify individuals who benefit from an intervention. Some approaches have been developed to estimate individualized treatment effects (ITE) with a single randomized control trial (RCT) or observational data, but they are often underpowered for the ITE estimation. Using individual participant data meta-analyses (IPD-MA) might solve this problem. Few studies have investigated how to develop risk prediction models with IPD-MA, and it remains unclear how to combine those methods with approaches used for ITE estimation. In this article, we compared different approaches using both simulated and real data with binary and time-to-event outcomes to estimate the individualized treatment effects from an IPD-MA in a one-stage approach. We compared five one-stage models: naive model (NA), random intercept (RI), stratified intercept (SI), rank-1 (R1), and fully stratified (FS), built with two different strategies, the S-learner and the T-learner constructed with a Monte Carlo simulation study in which we explored different scenarios with a binary or a time-to-event outcome. To evaluate the performance of the models, we used the c-statistic for benefit, the calibration of predictions, and the mean squared error. The different models were also used on the INDANA IPD-MA, comparing an anti-hypertensive treatment to no treatment or placebo ( Simulation results showed that using the S-learner led to better ITE estimation performances for both binary and time-to-event outcomes. None of the risk models stand out and had significantly better results. For the INDANA dataset with a binary outcome, the naive and the random intercept models had the best performances. For the choice of the strategy, using interactions with treatment (the S-learner) is preferable. For the choice of the method, no approach is better than the other.
Sections du résumé
BACKGROUND
BACKGROUND
One key aspect of personalized medicine is to identify individuals who benefit from an intervention. Some approaches have been developed to estimate individualized treatment effects (ITE) with a single randomized control trial (RCT) or observational data, but they are often underpowered for the ITE estimation. Using individual participant data meta-analyses (IPD-MA) might solve this problem. Few studies have investigated how to develop risk prediction models with IPD-MA, and it remains unclear how to combine those methods with approaches used for ITE estimation. In this article, we compared different approaches using both simulated and real data with binary and time-to-event outcomes to estimate the individualized treatment effects from an IPD-MA in a one-stage approach.
METHODS
METHODS
We compared five one-stage models: naive model (NA), random intercept (RI), stratified intercept (SI), rank-1 (R1), and fully stratified (FS), built with two different strategies, the S-learner and the T-learner constructed with a Monte Carlo simulation study in which we explored different scenarios with a binary or a time-to-event outcome. To evaluate the performance of the models, we used the c-statistic for benefit, the calibration of predictions, and the mean squared error. The different models were also used on the INDANA IPD-MA, comparing an anti-hypertensive treatment to no treatment or placebo (
RESULTS
RESULTS
Simulation results showed that using the S-learner led to better ITE estimation performances for both binary and time-to-event outcomes. None of the risk models stand out and had significantly better results. For the INDANA dataset with a binary outcome, the naive and the random intercept models had the best performances.
CONCLUSIONS
CONCLUSIONS
For the choice of the strategy, using interactions with treatment (the S-learner) is preferable. For the choice of the method, no approach is better than the other.
Identifiants
pubmed: 38528447
doi: 10.1186/s12874-024-02202-9
pii: 10.1186/s12874-024-02202-9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
74Subventions
Organisme : Agence Nationale de la Recherche
ID : ANR-18-CE36-0010-01
Organisme : Agence Nationale de la Recherche
ID : ANR-18-CE36-0010-01
Informations de copyright
© 2024. The Author(s).
Références
Ballarini NM, Rosenkranz GK, Jaki T, Konig F, Posch M. Subgroup identification in clinical trials via the predicted individual treatment effect. PLoS One. 2018;13(10):e0205971. https://doi.org/10.1371/journal.pone.0205971 .
doi: 10.1371/journal.pone.0205971
pubmed: 30335831
pmcid: 6193713
Farooq V, van Klaveren D, Steyerberg EW, Meliga E, Vergouwe Y, Chieffo A, et al. Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II. Lancet. 2013;381(9867):639–50. https://doi.org/10.1016/S0140-6736(13)60108-7 .
doi: 10.1016/S0140-6736(13)60108-7
pubmed: 23439103
Debray TP, Moons KG, Ahmed I, Koffijberg H, Riley RD. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat Med. 2013;32(18):3158–80. https://doi.org/10.1002/sim.5732 .
doi: 10.1002/sim.5732
pubmed: 23307585
Steyerberg EW, Nieboer D, Debray TPA, van Houwelingen HC. Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: An overview and illustration. Stat Med. 2019;38(22):4290–309. https://doi.org/10.1002/sim.8296 .
doi: 10.1002/sim.8296
pubmed: 31373722
pmcid: 6772012
Fisher DJ, Carpenter JR, Morris TP, Freeman SC, Tierney JF. Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? BMJ. 2017;356:j573. https://doi.org/10.1136/bmj.j573 .
doi: 10.1136/bmj.j573
pubmed: 28258124
pmcid: 5421441
Chalkou K, Steyerberg E, Egger M, Manca A, Pellegrini F, Salanti G. A two-stage prediction model for heterogeneous effects of many treatment options: application to drugs for Multiple Sclerosis. 2020. https://arxiv.org/pdf/2004.13464.pdf . Accessed 3 June 2021.
Seo M, White IR, Furukawa TA, Imai H, Valgimigli M, Egger M, et al. Comparing methods for estimating patient-specific treatment effects in individual patient data meta-analysis. Stat Med. 2020;40(6):1553–73. https://doi.org/10.1002/sim.8859 .
doi: 10.1002/sim.8859
pubmed: 33368415
pmcid: 7898845
Künzel SR, Sekhon JS, Bickel PJ, Yu B. Metalearners for estimating heterogeneous treatment effects using machine learning. Proc Natl Acad Sci U S A. 2019;116(10):4156–65. https://doi.org/10.1073/pnas.1804597116 .
doi: 10.1073/pnas.1804597116
pubmed: 30770453
pmcid: 6410831
Kennedy EH. Optimal doubly robust estimation of heterogeneous causal effects. arXiv. 2020. https://doi.org/10.48550/ARXIV.2004.14497 .
Susan Athey, Julie Tibshirani, Stefan Wager "Generalized random forests. Ann Stat Ann Statist. 2019;47(2):1148–78.
Guo X, Ni A. Contrast weighted learning for robust optimal treatment rule estimation. Stat Med. 2022;sim.9574. https://doi.org/10.1002/sim.9574 .
Chen S, Tian L, Cai T, Yu M. A general statistical framework for subgroup identification and comparative treatment scoring. Biometrics. 2017;73(4):1199–209. https://doi.org/10.1111/biom.12676 .
doi: 10.1111/biom.12676
pubmed: 28211943
pmcid: 5561419
Gueyffier F, Boutitie F, Boissel J, Coope J, Cutler J, Ekbom T, et al. INDANA: a meta-analysis on individual patient data in hypertension. Protocol and preliminary results. Thérapie. 1995;50:353–562.
pubmed: 7482389
Royston P, Parmar MKB, Sylvester R. Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer. Stat Med. 2004;23(6):907–26. https://doi.org/10.1002/sim.1691 .
doi: 10.1002/sim.1691
pubmed: 15027080
Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. https://doi.org/10.1371/journal.pmed.1001381 .
doi: 10.1371/journal.pmed.1001381
pubmed: 23393430
pmcid: 3564751
van Klaveren D, Steyerberg EW, Serruys PW, Kent DM. The proposed ‘concordance-statistic for benefit’ provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol. 2018;94:59–68. https://doi.org/10.1016/j.jclinepi.2017.10.021 .
doi: 10.1016/j.jclinepi.2017.10.021
pubmed: 29132832
Riley RD, Debray TPA, Fisher D, Hattle M, Marlin N, Hoogland J, et al. Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: Statistical recommendations for conduct and planning. Stat Med. 2020;39(15):2115–37. https://doi.org/10.1002/sim.8516 .
doi: 10.1002/sim.8516
pubmed: 32350891
pmcid: 7401032
Belias M, Rovers MM, Reitsma JB, Debray TPA, IntHout J. Statistical approaches to identify subgroups in meta-analysis of individual participant data: a simulation study. BMC Med Res Methodol. 2019;19(1):183. https://doi.org/10.1186/s12874-019-0817-6 .
doi: 10.1186/s12874-019-0817-6
pubmed: 31477023
pmcid: 6720416
Lim M, Hastie T. Learning interactions through hierarchical group-lasso regularization. 2013. https://arxiv.org/abs/1308.2719 . Accessed 3 June 2021.
Pocock SJ, McCormack V, Gueyffier F, Boutitie F, Fagard RH, Boissel JP. A score for predicting risk of death from cardiovascular disease in adults with raised blood pressure, based on individual patient data from randomised controlled trials. BMJ. 2001;323(7304):75–81. https://doi.org/10.1136/bmj.323.7304.75 .
doi: 10.1136/bmj.323.7304.75
pubmed: 11451781
pmcid: 34541
Quartagno M, Carpenter JR. Multiple imputation for IPD meta-analysis: allowing for heterogeneity and studies with missing covariates. Stat Med. 2016;35(17):2938–54. https://doi.org/10.1002/sim.6837 .
doi: 10.1002/sim.6837
pubmed: 26681666
White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30(4):377–99. https://doi.org/10.1002/sim.4067 .
doi: 10.1002/sim.4067
pubmed: 21225900
Vergouwe Y, Royston P, Moons KGM, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol. 2010;63(2):205–14. https://doi.org/10.1016/j.jclinepi.2009.03.017 .
doi: 10.1016/j.jclinepi.2009.03.017
pubmed: 19596181
Wood AM, Royston P, White IR. The estimation and use of predictions for the assessment of model performance using large samples with multiply imputed data. Biom J. 2015;57(4):614–32. https://doi.org/10.1002/bimj.201400004 .
doi: 10.1002/bimj.201400004
pubmed: 25630926
pmcid: 4515100
Imai K, Li ML. Experimental Evaluation of Individualized Treatment Rules. J Am Stat Assoc. 2021;118(541):242–56. https://doi.org/10.1080/01621459.2021.1923511 .
doi: 10.1080/01621459.2021.1923511
van Klaveren D, Balan TA, Steyerberg EW, Kent DM. Models with interactions overestimated heterogeneity of treatment effects and were prone to treatment mistargeting. J Clin Epidemiol. 2019;114:72–83. https://doi.org/10.1016/j.jclinepi.2019.05.029 .
doi: 10.1016/j.jclinepi.2019.05.029
pubmed: 31195109
pmcid: 7497896
Nie X, Wager S. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika. 2020;asaa076. https://doi.org/10.1093/biomet/asaa076 .
Tian L, Alizadeh AA, Gentles AJ, Tibshirani R. A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates. J Am Stat Assoc. 2014;109(508):1517–32. https://doi.org/10.1080/01621459.2014.951443 .
doi: 10.1080/01621459.2014.951443
pubmed: 25729117
pmcid: 4338439
Park H, Petkova E, Tarpey T, Ogden RT. A Single-Index Model With a Surface-Link for Optimizing Individualized Dose Rules. J Comput Graph Stat. 2022;31(2):553–62. https://doi.org/10.1080/10618600.2021.1923521 .
doi: 10.1080/10618600.2021.1923521
pubmed: 35873662
Zhang B, Tsiatis AA, Laber EB, Davidian M. A robust method for estimating optimal treatment regimes. Biometrics. 2012;68(4):1010–8. https://doi.org/10.1111/j.1541-0420.2012.01763.x .
doi: 10.1111/j.1541-0420.2012.01763.x
pubmed: 22550953
pmcid: 3556998
Zhao YQ, Zeng D, Laber EB, Song R, Yuan M, Kosorok MR. Doubly Robust Learning for Estimating Individualized Treatment with Censored Data. Biometrika. 2015;102(1):151–68. https://doi.org/10.1093/biomet/asu050 .
doi: 10.1093/biomet/asu050
pubmed: 25937641
Mo W, Qi Z, Liu Y. Learning Optimal Distributionally Robust Individualized Treatment Rules. J Am Stat Assoc. 2021;116(534):659–74. https://doi.org/10.1080/01621459.2020.1796359 .
doi: 10.1080/01621459.2020.1796359
pubmed: 34177007
Zhao YQ, Zeng D, Tangen CM, Leblanc ML. Robustifying trial-derived optimal treatment rules for a target population. Electron J Stat. 2019;13(1):1717–43. https://doi.org/10.1214/19-EJS1540 .
doi: 10.1214/19-EJS1540
pubmed: 31440323
pmcid: 6705616
Mistry D, Stallard N, Underwood M. A recursive partitioning approach for subgroup identification in individual patient data meta-analysis. Stat Med. 2018;37(9):1550–61. https://doi.org/10.1002/sim.7609 .
doi: 10.1002/sim.7609
pubmed: 29383818
pmcid: 5900744
Wu Y, Jiang X, Kim J, Ohno-Machado L. Grid Binary LOgistic REgression (GLORE): building shared models without sharing data. J Am Med Inform Assoc. 2012;19(5):758–64. https://doi.org/10.1136/amiajnl-2012-000862 .
doi: 10.1136/amiajnl-2012-000862
pubmed: 22511014
pmcid: 3422844
Lu CL, Wang S, Ji Z, Wu Y, Xiong L, Jiang X, et al. WebDISCO: a web service for distributed cox model learning without patient-level data sharing. J Am Med Inform Assoc. 2015;22(6):1212–9. https://doi.org/10.1093/jamia/ocv083 .
doi: 10.1093/jamia/ocv083
pubmed: 26159465
pmcid: 5009917
Zhao Y, Fang X, Simchi-Levi D. Uplift modeling with multiple treatments and general response types. 2017. http://arxiv.org/abs/1705.08492 . Accessed 5 July 2023.
Acharki N, Lugo R, Bertoncello A, Garnier J. Comparison of meta-learners for estimating multi-valued treatment heterogeneous effects. 2023. http://arxiv.org/abs/2205.14714 . Accessed 11 Apr 2023.