Developing prediction models to estimate the risk of two survival outcomes both occurring: A comparison of techniques.
clinical prediction model
multiple outcome
multivariate
simulation
survival analysis
time-to-event
Journal
Statistics in medicine
ISSN: 1097-0258
Titre abrégé: Stat Med
Pays: England
ID NLM: 8215016
Informations de publication
Date de publication:
15 08 2023
15 08 2023
Historique:
revised:
21
03
2023
received:
25
10
2022
accepted:
26
04
2023
medline:
18
7
2023
pubmed:
23
5
2023
entrez:
23
5
2023
Statut:
ppublish
Résumé
This study considers the prediction of the time until two survival outcomes have both occurred. We compared a variety of analytical methods motivated by a typical clinical problem of multimorbidity prognosis. We considered five methods: product (multiply marginal risks), dual-outcome (directly model the time until both events occur), multistate models (msm), and a range of copula and frailty models. We assessed calibration and discrimination under a variety of simulated data scenarios, varying outcome prevalence, and the amount of residual correlation. The simulation focused on model misspecification and statistical power. Using data from the Clinical Practice Research Datalink, we compared model performance when predicting the risk of cardiovascular disease and type 2 diabetes both occurring. Discrimination was similar for all methods. The product method was poorly calibrated in the presence of residual correlation. The msm and dual-outcome models were the most robust to model misspecification but suffered a drop in performance at small sample sizes due to overfitting, which the copula and frailty model were less susceptible to. The copula and frailty model's performance were highly dependent on the underlying data structure. In the clinical example, the product method was poorly calibrated when adjusting for 8 major cardiovascular risk factors. We recommend the dual-outcome method for predicting the risk of two survival outcomes both occurring. It was the most robust to model misspecification, although was also the most prone to overfitting. The clinical example motivates the use of the methods considered in this study.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
3184-3207Subventions
Organisme : Medical Research Council
ID : MR/T025085/1
Pays : United Kingdom
Organisme : Department of Health
Pays : United Kingdom
Informations de copyright
© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
Références
Steyerberg EW. In: Gail M, Jonathan S, Singer B, eds. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. 2nd ed. Cham: Springer; 2019. doi:10.1007/978-3-030-16399-0
Riley RD, van der Windt D, Croft P, Moons KGM. Prognosis Research in Healthcare: Concepts, Methods, and Impact. Oxford, UK: Oxford University Press; 2019.
van Smeden M, Reitsma JB, Riley RD, Collins GS, Moons KG. Clinical prediction models: diagnosis vs prognosis. J Clin Epidemiol. 2021;132:142-145. doi:10.1016/j.jclinepi.2021.01.009
Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system: risk prediction of hospital mortality for critically III hospitalized adults. Chest. 1991;100(6):1619-1636. doi:10.1378/chest.100.6.1619
Jentzer JC, Bennett C, Wiley BM, et al. Predictive value of the sequential organ failure assessment score for mortality in a contemporary cardiac intensive care unit population. J Am Heart Assoc. 2018;7(6):e008169. doi:10.1161/JAHA.117.008169
Hippisley-Cox J, Coupland C, Brindle P. Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017;357(3):j2099. 10.1136/bmj.j2099
D'Agostino RB, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham heart study. Circulation. 2008;117(6):743-753. doi:10.1161/CIRCULATIONAHA.107.699579
Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-382. doi:10.1136/thorax.58.5.377
McAllister KSL, Ludman PF, Hulme W, et al. A contemporary risk model for predicting 30-day mortality following percutaneous coronary intervention in England and Wales. Int J Cardiol. 2016;210:125-132. doi:10.1016/j.ijcard.2016.02.085
Kanis JA, Johnell O, Oden A, Johansson H, McCloskey E. FRAX and the assessment of fracture probability in men and women from the UK. Osteoporos Int. 2008;19(4):385-397. doi:10.1007/s00198-007-0543-5
Caldas C, Greenberg DC, Kearins O, et al. Erratum to: PREDICT: a new UK prognostic model that predicts survival following surgery for invasive breast cancer. Breast Cancer Res. 2010;12:1-10. doi:10.1186/bcr2480
Nashef SAM, Roques F, Sharples LD, et al. Euroscore II. Eur J Cardio-Thoracic Surg. 2012;41(4):734-745. doi:10.1093/ejcts/ezs043
Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of clinical classification schemes for predicting stroke: results from the National Registry of atrial fibrillation. J Am Med Assoc. 2001;285(22):2864-2870. doi:10.1001/jama.285.22.2864
Lip GYH, Nieuwlaat R, Pisters R, Lane DA, Crijns HJGM. Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the euro heart survey on atrial fibrillation. Chest. 2010;137(2):263-272. doi:10.1378/chest.09-1584
Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJGM, Lip GYH. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the euro heart survey. Chest. 2010;138(5):1093-1100. doi:10.1378/chest.10-0134
Chao TF, Lip GYH, Lin YJ, et al. Incident risk factors and major bleeding in patients with atrial fibrillation treated with Oral anticoagulants: a comparison of baseline, follow-up and Delta HAS-BLED scores with an approach focused on modifiable bleeding risk factors. Thromb Haemost. 2018;118(4):768-777. doi:10.1055/s-0038-1636534
Chao TF, Lip GYH, Liu CJ, et al. Relationship of aging and incident comorbidities to stroke risk in patients with atrial fibrillation. J Am Coll Cardiol. 2018;71(2):122-132. doi:10.1016/j.jacc.2017.10.085
Wu X, Baig A, Kasymjanova G, et al. Pattern of local recurrence and distant metastasis in breast cancer by molecular subtype. Cureus. 2016;8(12):e924. doi:10.7759/cureus.924
Peng L, Hong X, Yuan Q, Lu L, Wang Q, Chen W. Prediction of local recurrence and distant metastasis using radiomics analysis of pretreatment nasopharyngeal [18F]FDG PET/CT images. Ann Nucl Med. 2021;35(4):458-468. doi:10.1007/s12149-021-01585-9
Faria SC, Devine CE, Rao B, Sagebiel T, Bhosale P. Imaging and staging of endometrial cancer. Semin Ultrasound CT MRI. 2019;40(4):287-294. doi:10.1053/j.sult.2019.04.001
Avanzo M, Stancanello J, Pirrone G, Sartor G. Radiomics and deep learning in lung cancer. Strahlentherapie Und Onkol. 2020;196(10):879-887. doi:10.1007/s00066-020-01625-9
Uijen A, van de Lisdonk E. Multimorbidity in primary care: prevalence and trend over the last 20 years. Eur J Gen Pract. 2008;14(SUPPL. 1):28-32. doi:10.1080/13814780802436093
Dhalwani NN, O'Donovan G, Zaccardi F, et al. Long terms trends of multimorbidity and association with physical activity in older English population. Int J Behav Nutr Phys Act. 2016;13(1):1-9. doi:10.1186/s12966-016-0330-9
van Oostrom SH, Gijsen R, Stirbu I, et al. Time trends in prevalence of chronic diseases and multimorbidity not only due to aging: data from general practices and health surveys. PLoS One. 2016;11(8):1-14. doi:10.1371/journal.pone.0160264
Hall M, Dondo TB, Yan AT, et al. Multimorbidity and survival for patients with acute myocardial infarction in England and Wales: latent class analysis of a nationwide population-based cohort. PLoS Med. 2018;15(3):1-18. doi:10.1371/journal.pmed.1002501
Tran J, Norton R, Conrad N, et al. Patterns and temporal trends of comorbidity among adult patients with incident cardiovascular disease in the UK between 2000 and 2014: a population-based cohort study. PLoS Med. 2018;15(3):1-23. doi:10.1371/journal.pmed.1002513
The Academy of Medical Sciences. Multimorbidity: A Priority for Global Health Research; 2018. https://acmedsci.ac.uk/file-download/82222577
Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380(9836):37-43. doi:10.1016/S0140-6736(12)60240-2
Blodgett JM, Rockwood K, Theou O. Changes in the severity and lethality of age-related health deficit accumulation in the USA between 1999 and 2018: a population-based cohort study. Lancet Heal Longev. 2021;2(2):e96-e104. doi:10.1016/S2666-7568(20)30059-3
European Observatory on Health Systems and Policies, Rijken M, Struckmann V, Dyakova M, Melchiorre MG. ICARE4EU: improving care for people with multiple chronic conditions in Europe. Eurohealth. 2013;19(3):29-31.
Taylor AW, Price K, Gill TK, et al. Multimorbidity: not just an older person's issue. BMC Public Health. 2010;10:718. doi:10.1186/1471-2458-10-718
Afshar S, Roderick PJ, Kowal P, Dimitrov BD, Hill AG. Multimorbidity and the inequalities of global ageing: a cross-sectional study of 28 countries using the world health surveys. BMC Public Health. 2015;15(1):1-10. doi:10.1186/s12889-015-2008-7
Arokiasamy P, Uttamacharya U, Jain K, et al. The impact of multimorbidity on adult physical and mental health in low- and middle-income countries: what does the study on global ageing and adult health (SAGE) reveal? BMC Med. 2015;13(1):1-16. doi:10.1186/s12916-015-0402-8
Garin N, Koyanagi A, Chatterji S, et al. Global multimorbidity patterns: a cross-sectional, population-based, multi-country study. J Gerontol A. 2016;71(2):205-214. doi:10.1093/gerona/glv128
Lip GYH, Genaidy A, Tran G, Marroquin P, Estes C, Sloop S. Improving stroke risk prediction in the general population: a comparative assessment of common clinical rules, a new multimorbid index, and machine-learning-based algorithms. Thromb Haemost. 2021;122(1):142-150. doi:10.1055/a-1467-2993
Nopp S, Spielvogel C, Schmaldienst S, et al. Bleeding risk assessment in end-stage kidney disease: validation of existing risk scores and evaluation of a machine learning-based approach. Thromb Haemost. 2022;122(09):1558-1566. doi:10.1055/a-1754-7551
Pintilie M. Competing Risks: A Practical Perspective. Hoboken, NJ: John Wiley & Sons; 2006.
Geskus RB. Data Analysis with Competing Risks and Intermediate States. Boca Raton: CRC Press; 2016.
Beyersmann J, Allignol A, Schumacher M. Competing Risks and Multistate Models with R. New York, NY: Springer New York; 2012.
Putter H, Fiocco M, Geskus RB. Tutorial in biostatistics: competing risks and multi-state models. Stat Med. 2007;26(11):2389-2430. doi:10.1002/sim
Austin PC, Lee DS, Fine JP. Introduction to the analysis of survival data in the presence of competing risks. Circulation. 2016;133(6):601-609. doi:10.1161/CIRCULATIONAHA.115.017719
Wolbers M, Koller MT, Witteman JCM, Steyerberg EW. Prognostic models with competing risks. Epidemiology. 2009;20(4):555-561. doi:10.1097/EDE.0b013e3181a39056
Govindarajulu US, D'Agostino RB. Review of current advances in survival analysis and frailty models. Wiley Interdiscip Rev Comput Stat. 2020;12(6):1-11. doi:10.1002/wics.1504
Bakoyannis G, Touloumi G. Practical methods for competing risks data: a review. Stat Methods Med Res. 2012;21(3):257-272. doi:10.1177/0962280210394479
Shihl JH, Louis TA. Inferences on the association parameter in copula models for bivariate survival data. Biometrics. 1995;51(4):1384-1399.
Goethals K, Janssen P, Duchateau L. Frailty models and copulas: similarities and differences. J Appl Stat. 2008;35(9):1071-1079. doi:10.1080/02664760802271389
Durrleman V, Nikeghbali A, Roncalli T. Which copula is the right one? Tech Rep Groupe Rech Opérationnelle, Crédit Lyon; 2000. doi:10.2139/ssrn.1123135
Marra G, Radice R. Bivariate copula additive models for location, scale and shape. Comput Stat Data Anal. 2017;112:99-113. doi:10.1016/j.csda.2017.03.004
Emura T, Matsui S, Rondeau V. Survival Analysis with Correlated Endpoints: Joint Frailty-Copula Models. Singapore: Springer Singapore; 2019. doi:10.1007/978-981-13-3516-7
Georges P, Arnaud-Guilhem A, Emeric N, Guillaume Q, Thierry R. Multivariate Survival Modelling: A Unified Approach with Copulas; 2001. doi:10.2139/ssrn.1032559
Nelsen RB. An Introduction to Copulas. 2nd ed. New York: Springer Pulishing Company; 2006.
Marra G, Radice R. GJRM: Generalised Joint Regression Modelling; 2017. https://cran.r-project.org/web/packages/GJRM/index.html
Marra G, Radice R. Joint regression modeling framework for analyzing bivariate binary data in R. Depend Model. 2017;5(1):268-294. doi:10.1515/demo-2017-0016
Vaupel JW, Manton KG, Stallard E. The impact of heterogeneity in individual frailty on the dynamics of mortality. Demography. 1979;16(3):439-454.
Duchateau L, Janssen P. The Frailty Model. New York: Springer; 2008.
Wienke A. Frailty Models in Survival Analysis. Chapman and Hall/CRC Biostatistics Series. Boca Raton: CRC Press; 2011.
Balan TA, Putter H. A tutorial on frailty models. Stat Methods Med Res. 2020;29(11):3424-3454. doi:10.1177/0962280220921889
Aalen OO, Johansen S. An empirical transition matrix for non-homogeneous Markov chains based on censored observations. Scand J Stat. 1978;5(3):141-150.
Putter H, Spitoni C. Non-parametric estimation of transition probabilities in non-Markov multi-state models: the landmark Aalen-Johansen estimator. Stat Methods Med Res. 2018;27(7):2081-2092. doi:10.1177/0962280216674497
Titman AC. Transition probability estimates for non-Markov multi-state models. Biometrics. 2015;71(4):1034-1041. doi:10.1111/biom.12349
de Uña-Álvarez J, Meira-Machado L. Nonparametric estimation of transition probabilities in the non-Markov illness-death model: a comparative study. Biometrics. 2015;71(2):364-375. doi:10.1111/biom.12288
Pepe MS. Inference for events with dependent risks in multiple endpoint studies. J Am Stat Assoc. 1991;86(415):770-778. doi:10.1080/01621459.1991.10475108
de Wreede LC, Fiocco M, Putter H. The mstate package for estimation and prediction in non- and semi-parametric multi-state and competing risks models. Comput Methods Programs Biomed. 2010;99(3):261-274. doi:10.1016/j.cmpb.2010.01.001
de Wreede LC, Fiocco M, Putter H. Mstate: an R package for the analysis of competing risks and multi-state models. J Stat Softw. 2011;38(7):1-30.
Duchateau L, Janssen P. Extensions of the frailty model. The Frailty Model. New York, NY: Springer; 2008. doi:10.1007/978-0-387-72 835-3_7
Do Ha I, Jeong J-H, Lee YL. Statistical Modelling of Survival Data with Random Effects: H-Likelihood Approach. Singapore: Springer; 2017. doi:10.1007/978-981-10-6557-6
Rueten-Budde AJ, Putter H, Fiocco M. Investigating hospital heterogeneity with a competing risks frailty model. Stat Med. 2019;38(2):269-288. doi:10.1002/sim.8002
Katsahian S, Resche-Rigon M, Chevret S, Porcher R. Analysing multicentre competing risks data with a mixed proportional hazards model for the subdistribution. Stat Med. 2006;25(24):4267-4278. doi:10.1002/sim.2684
Scheike TH, Sun Y, Zhang MJ, Jensen TK. A semiparametric random effects model for multivariate competing risks data. Biometrika. 2010;97(1):133-145. doi:10.1093/biomet/asp082
Emura T, Shih J-H, Il DH, Wilke RA. Comparison of the marginal hazard model and the sub-distribution hazard model for competing risks under an assumed copula. Stat Methods Med Res. 2020;29(8):2307-2327. doi:10.1177/0962280219892295
Emura T, Chen Y-H. Gene selection for survival data under dependent censoring: a copula-based approach. Stat Methods Med Res. 2016;25(6):2840-2857. doi:10.1177/0962280214533378
Zhu H, Lan Y, Ning J, Shen Y. Semiparametric copula-based regression modeling of semi-competing risks data. Commun Stat Theory Methods. 2021;51(22):7830-7845. doi:10.1080/03610926.2021.1881122
Zhou R, Zhu H, Bondy M, Ning J. Semiparametric model for semi-competing risks data with application to breast cancer study. Lifetime Data Anal. 2016;22(3):456-471. doi:10.1007/s10985-015-9344-x
Lo S, Wilke RA. A copula model for dependent competing risks. J R Stat Soc Ser C Appl Stat. 2010;59(2):359-376. doi:10.1111/j.1467-9876.2009.00695.x
Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074-2102. doi:10.1002/sim.8086
Pate A, GitHub repository. Manchester Predictive Healthcare Group. MRC-Multi-Outcome-Project-4-Joint-Risk-Prediction-Two-Survival-Processes; 2022. https://github.com/manchester-predictive-healthcare-group/CHI-MRC-multi-outcome/tree/main/Project%204%20Joint%20Risk%20Prediction%20Two%20Survival%20Processes
Riley RD, Snell KIE, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med. 2019;38(7):1276-1296. doi:10.1002/sim.7992
Stan Development Team. “RStan: The R Interface to Stan.” R Package Version 2.21.3. 2021. https://mc-stan-org/
van Calster B, Nieboer D, Vergouwe Y, de Cock B, Pencina MJ, Steyerberg EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167-176. doi:10.1016/j.jclinepi.2015.12.005
Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361-387. doi:10.1002/(SICI)1097-0258(19960229)15:4 < 361::AID-SIM168 > 3.0.CO;2-4
Moons KGM, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012;98(9):691-698. doi:10.1136/heartjnl-2011-301247
Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453-473. doi:10.1002/(SICI)1097-0258(20000229)19:4 < 453::AID-SIM350 > 3.0.CO;2-5
Nichols GA, Brown JB. The impact of cardiovascular disease on medical care costs in subjects with and without type 2 diabetes. Diabetes Care. 2002;25(3):482-486. doi:10.2337/diacare.25.3.482
Einarson TR, Acs A, Ludwig C, Panton UH. Economic burden of cardiovascular disease in type 2 diabetes: a systematic review. Value Heal. 2018;21(7):881-890. doi:10.1016/j.jval.2017.12.019
Raghavan S, Vassy JL, Ho YL, et al. Diabetes mellitus-related all-cause and cardiovascular mortality in a national cohort of adults. J Am Heart Assoc. 2019;8(4):e011295. doi:10.1161/JAHA.118.011295
Leon BM. Diabetes and cardiovascular disease: epidemiology, biological mechanisms, treatment recommendations and future research. World J Diabetes. 2015;6(13):1246-1258. doi:10.4239/wjd.v6.i13.1246
Clinical Practice Research Datalink (CPRD). Primary care data for public health research. Accessed September 16, 2021 https://www.cprd.com/primary-care
Wolf A, Dedman D, Campbell J, et al. Data resource profile: clinical practice research datalink (CPRD) aurum. Int J Epidemiol. 2019;48(6):1740-1740G. doi:10.1093/ije/dyz034
van Buuren S, Groothuis-oudshoorn K. mice: multivariate imputation by chained equations. J Stat Softw. 2011;45(3):1-67.
Therneau TM. A Package for Survival Analysis in S_. version 2.38; 2015. https://cran.r-project.org/package=survival
Team SD. RStan: The R Interface to Stan; 2021. https://mc-stan.org/
Austin PC, Harrell FE, van Klaveren D. Graphical calibration curves and the integrated calibration index (ICI) for survival models. Stat Med. 2020;39(21):2714-2742. doi:10.1002/sim.8570
Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517-535. doi:10.1002/sim.5941
Harrell FE. Regression Modeling Strategies. Cham: Springer; 2015.
Gauthier J, Wu QV, Gooley TA. Cubic splines to model relationships between continuous variables and outcomes: a guide for clinicians. Bone Marrow Transplant. 2020;55(4):675-680. doi:10.1038/s41409-019-0679-x
Devlin TF, Weeks BJ. Spline functions for logistic regression modelling. Paper presented at: Proceddings of the 11th Annual SAS Users gr Intnl Conference, vol. 4; 1986:646-651. https://support.sas.com/resources/papers/proceedings-archive/SUGI86/Sugi-11-119DevlinWeeks.pdf.
Rutherford MJ, Crowther MJ, Lambert PC. The use of restricted cubic splines to approximate complex hazard functions in the analysis of time-to-event data: a simulation study. J Stat Comput Simul. 2015;85(4):777-793. doi:10.1080/00949655.2013.845890
Benner A. Multivariable fractional polynomials. https://cran.r-project.org/web/packages/mfp/vignettes/mfp_vignette.pdf Accessed July 24, 2018.
Sauerbrei W, Royston P. Building multivariable prognostic and diagnostic models: transformation of the predictors by using fractional polynomials. J R Stat Soc Ser A Stat Soc. 1999;162(1):71-94. doi:10.1111/1467-985X.00122
Chowdhury MZI, Turin TC. Variable selection strategies and its importance in clinical prediction modelling. Fam Med Community Heal. 2020;8(1):1-7. doi:10.1136/fmch-2019-000262
Heinze G, Wallisch C, Dunkler D. Variable selection - a review and recommendations for the practicing statistician. Biometrical J. 2018;60(3):431-449. doi:10.1002/bimj.201700067
RStudio. Integrated Development for R. RStudio Team; 2020. http://www.rstudio.com/
Narasimhan B, Johnson SG, Hahn T, Bouvier A, Kieu K. Cubature: Adaptive Multivariate Integration over Hypercubes; 2021. https://cran.r-project.org/package=cubature
Potapov S, Adler W, Schmid M. survAUC: Estimators of Prediction Accuracy for Time-To-Event Data; 2012. https://cran.r-project.org/package=survAUC
Wickham H, Francois R, Henry L, Muller K. Dplyr: A Grammar of Data Manipulation; 2022.
Harrell FE Jr. Hmisc: Harrell Miscellaneous; 2021. https://cran.r-project.org/package=Hmisc
Harrell FE. R Package: rms; 2022. https://cran.r-project.org/package=rms
Wickham H. ggplot2: Elegant Graphics for Data Analysis; 2016. https://ggplot2.tidyverse.org
Urbanek S, Horner J. Cairo: R Graphics Device using Cairo Graphics Library for Creating High-Quality Bitmap (PNG, JPEG, TIFF), Vector (PDF, SVG, PostScript) and Display (X11 and Win32) Output; 2022. https://cran.r-project.org/package=Cairo
Signorell A, Aho K, Alfons A, et al. DescTools: Tools for Descriptive Statistics; 2021. https://cran.r-project.org/package=DescTools
Kassambara A. ggpubr: “ggplot2” Based Publication Ready Plots; 2020. https://cran.r-project.org/package=ggpubr
Xie Y. knitr: A General-Purpose Package for Dynamic Report Generation in R; 2021. https://rdrr.io/github/yihui/knitr/man/knitr-package.html
Wickham H. Reshaping data with the reshape package. J Stat Softw. 2007;21(12):1-20.
Blaser N, Vizcaya LS, Estill J, et al. Gems: an R package for simulating from disease progression models. J Stat Softw. 2015;64(10):1-22.
Brilleman SL, Wolfe R, Moreno-Betancur M, Crowther MJ. Simulating survival data using the simsurv R package. J Stat Softw. 2021;97(3):1-27. doi:10.18637/jss.v097.i03