The impact of different censoring methods for analyzing survival using real-world data with linked mortality information: a simulation study.
Bias
Censoring
Estimation
Missing data
Real-world data
Real-world evidence
Simulation
Survival analysis
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
13 Sep 2024
13 Sep 2024
Historique:
received:
25
09
2023
accepted:
20
08
2024
medline:
14
9
2024
pubmed:
14
9
2024
entrez:
13
9
2024
Statut:
epublish
Résumé
Evaluating outcome reliability is critical in real-world evidence studies. Overall survival is a common outcome in these studies; however, its capture in real-world data (RWD) sources is often incomplete and supplemented with linked mortality information from external sources. Conflicting recommendations exist for censoring overall survival in real-world evidence studies. This simulation study aimed to understand the impact of different censoring methods on estimating median survival and log hazard ratios when external mortality information is partially captured. We used Monte Carlo simulation to emulate a non-randomized comparative effectiveness study of two treatments with RWD from electronic health records and linked external mortality data. We simulated the time to death, the time to last database activity, and the time to data cutoff. Death events after the last database activity were attributed to linked external mortality data and randomly set to missing to reflect the sensitivity of contemporary real-world data sources. Two censoring schemes were evaluated: (1) censoring at the last activity date and (2) censoring at the end of data availability (data cutoff) without an observed death. We assessed the performance of each method in estimating median survival and log hazard ratios using bias, coverage, variance, and rejection rate under varying amounts of incomplete mortality information and varying treatment effects, length of follow-up, and sample size. When mortality information was fully captured, median survival estimates were unbiased when censoring at data cutoff and underestimated when censoring at the last activity. When linked mortality information was missing, censoring at the last activity date underestimated the median survival, while censoring at the data cutoff overestimated it. As missing linked mortality information increased, bias decreased when censoring at the last activity date and increased when censoring at data cutoff. Researchers should consider the completeness of linked external mortality information when choosing how to censor the analysis of overall survival using RWD. Substantial bias in median survival estimates can occur if an inappropriate censoring scheme is selected. We advocate for RWD providers to perform validation studies of their mortality data and publish their findings to inform methodological decisions better.
Sections du résumé
BACKGROUND
BACKGROUND
Evaluating outcome reliability is critical in real-world evidence studies. Overall survival is a common outcome in these studies; however, its capture in real-world data (RWD) sources is often incomplete and supplemented with linked mortality information from external sources. Conflicting recommendations exist for censoring overall survival in real-world evidence studies. This simulation study aimed to understand the impact of different censoring methods on estimating median survival and log hazard ratios when external mortality information is partially captured.
METHODS
METHODS
We used Monte Carlo simulation to emulate a non-randomized comparative effectiveness study of two treatments with RWD from electronic health records and linked external mortality data. We simulated the time to death, the time to last database activity, and the time to data cutoff. Death events after the last database activity were attributed to linked external mortality data and randomly set to missing to reflect the sensitivity of contemporary real-world data sources. Two censoring schemes were evaluated: (1) censoring at the last activity date and (2) censoring at the end of data availability (data cutoff) without an observed death. We assessed the performance of each method in estimating median survival and log hazard ratios using bias, coverage, variance, and rejection rate under varying amounts of incomplete mortality information and varying treatment effects, length of follow-up, and sample size.
RESULTS
RESULTS
When mortality information was fully captured, median survival estimates were unbiased when censoring at data cutoff and underestimated when censoring at the last activity. When linked mortality information was missing, censoring at the last activity date underestimated the median survival, while censoring at the data cutoff overestimated it. As missing linked mortality information increased, bias decreased when censoring at the last activity date and increased when censoring at data cutoff.
CONCLUSIONS
CONCLUSIONS
Researchers should consider the completeness of linked external mortality information when choosing how to censor the analysis of overall survival using RWD. Substantial bias in median survival estimates can occur if an inappropriate censoring scheme is selected. We advocate for RWD providers to perform validation studies of their mortality data and publish their findings to inform methodological decisions better.
Identifiants
pubmed: 39272007
doi: 10.1186/s12874-024-02313-3
pii: 10.1186/s12874-024-02313-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
203Informations de copyright
© 2024. The Author(s).
Références
21st Century Cures Act. H.R. 34, 114th Congress. 2016. https://www.gpo.gov/fdsys/pkg/BILLS-114hr34enr/pdf/BILLS-114hr34enr.pdf .
FDA. Considerations for the Use of Real-World Data and RealWorld Evidence to Support Regulatory Decision-Making for Drug and Biological Products 2021. Available from: https://www.fda.gov/media/154714/download
FDA. Real-World Data: Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products Guidance for Industry 2021. Available from: https://www.fda.gov/media/154449/download .
FDA. Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision Making for Drug and Biological Products 2021. Available from: https://www.fda.gov/media/152503/download .
FDA. Data Standards for Drug and Biological Product Submissions Containing Real-World Data 2021. Available from: https://www.fda.gov/media/153341/download .
EMA. Real-world evidence framework to support EU regulatory decision-making: Report on the experience gained with regulator-led studies from September 2021 to February 2023 2023. Available from: https://www.ema.europa.eu/en/documents/report/real-world-evidence-framework-support-eu-regulatory-decision-making-report-experience-gained_en.pdf .
FDA. Framework for FDA’s Real-World Evidence Program 2018. Available from: https://www.fda.gov/media/120060/download?attachment .
Li M, Chen S, Lai Y, Liang Z, Wang J, Shi J, et al. Integrating real-world evidence in the regulatory decision-making process: a systematic analysis of experiences in the US, EU, and China using a logic model. Front Med. 2021;8:8.
Sola-Morales O, Curtis LH, Heidt J, Walsh L, Casso D, Oliveria S, et al. Effectively leveraging RWD for external controls: a systematic literature review of regulatory and HTA decisions. Clin Pharmacol Ther. 2023;114(2):325–55.
doi: 10.1002/cpt.2914
pubmed: 37079433
Wang X, Dormont F, Lorenzato C, Latouche A, Hernandez R, Rouzier R. Current perspectives for external control arms in oncology clinical trials: analysis of EMA approvals 2016–2021. J Cancer Policy. 2023;35: 100403.
doi: 10.1016/j.jcpo.2023.100403
pubmed: 36646208
Seifu Y, Gamalo-Siebers M, Barthel FMS, Lin J, Qiu J, Cooner F, et al. Real-world evidence utilization in clinical development reflected by US product labeling: statistical review. Therapeutic Innov Regul Sci. 2020;54(6):1436–43.
doi: 10.1007/s43441-020-00170-y
Purpura CA, Garry EM, Honig N, Case A, Rassen JA. The role of real-world evidence in FDA-approved new drug and biologics license applications. Clin Pharmacol Ther. 2022;111(1):135–44.
doi: 10.1002/cpt.2474
pubmed: 34726771
Jahanshahi M, Gregg K, Davis G, Ndu A, Miller V, Vockley J, et al. The use of external controls in FDA regulatory decision making. Therapeutic Innov Regul Sci. 2021;55(5):1019–35.
doi: 10.1007/s43441-021-00302-y
Goring S, Taylor A, Müller K, Li TJJ, Korol EE, Levy AR, et al. Characteristics of non-randomised studies using comparisons with external controls submitted for regulatory approval in the USA and Europe: a systematic review. BMJ Open. 2019;9(2):e024895.
doi: 10.1136/bmjopen-2018-024895
pubmed: 30819708
pmcid: 6398650
Carrigan G, Bradbury BD, Brookhart MA, Capra WB, Chia V, Rothman KJ, et al. External comparator groups derived from real-world data used in support of egulatory decision making: use cases and challenges. Curr Epidemiol Rep. 2022;9(4):326–37.
doi: 10.1007/s40471-022-00305-9
Gatto NM, Campbell UB, Rubinstein E, Jaksa A, Mattox P, Mo J, et al. The structured process to identify fit-for-purpose data: a data feasibility assessment framework. Clin Pharmacol Ther. 2022;111(1):122–34.
doi: 10.1002/cpt.2466
pubmed: 34716990
Concato J, Corrigan-Curay J. Real-world evidence – where are we now? N Engl J Med. 2022;386(18):1680–2.
doi: 10.1056/NEJMp2200089
pubmed: 35485775
FDA. FDA approves new dosing regimen for cetuximab 2021. Available from: https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-new-dosing-regimen-cetuximab.
Zhang Q, Gossai A, Monroe S, Nussbaum NC, Parrinello CM. Validation analysis of a composite real-world mortality endpoint for patients with cancer in the United States. Health Serv Res. 2021;56(6):1281–7.
doi: 10.1111/1475-6773.13669
pubmed: 33998685
pmcid: 8586476
Ibrahim JG, Chu H, Chen M-H. Missing data in clinical studies: issues and methods. J Clin Oncol. 2012;30(26):3297–303.
doi: 10.1200/JCO.2011.38.7589
pubmed: 22649133
pmcid: 3948388
Jacobs EJ, Newton CC, Wang Y, Campbell PT, Flanders WD, Gapstur SM. Ghost-time bias from imperfect mortality ascertainment in aging cohorts. Ann Epidemiol. 2018;28(10):691-e63.
doi: 10.1016/j.annepidem.2018.06.002
pubmed: 30057347
Siannis F. Sensitivity analysis for multiple right censoring processes: investigating mortality in psoriatic arthritis. Stat Med. 2011;30(4):356–67.
doi: 10.1002/sim.4117
pubmed: 21225898
Carrigan G, Whipple S, Taylor MD, Torres AZ, Gossai A, Arnieri B, et al. An evaluation of the impact of missing deaths on overall survival analyses of advanced non–small cell lung cancer patients conducted in an electronic health records database. Pharmacoepidemiol Drug Saf. 2019;28(5):572–81.
doi: 10.1002/pds.4758
pubmed: 30873729
pmcid: 6594237
Lesko CR, Edwards JK, Cole SR, Moore RD, Lau B. When to censor? Am J Epidemiol. 2017;187(3):623–32.
doi: 10.1093/aje/kwx281
pmcid: 6248498
Lesko CR, Edwards JK, Moore RD, Lau B. Censoring for loss to follow-up in Time-to-event analyses of composite outcomes or in the presence of competing risks. Epidemiology. 2019;30(6):817–24.
doi: 10.1097/EDE.0000000000001073
pubmed: 31393316
pmcid: 6768756
Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074–102.
doi: 10.1002/sim.8086
pubmed: 30652356
pmcid: 6492164
Ladanie A, Schmitt AM, Speich B, Naudet F, Agarwal A, Pereira TV, et al. Clinical trial evidence supporting US Food and Drug Administration approval of novel cancer therapies between 2000 and 2016. JAMA Netw Open. 2020;3(11):e2024406-e.
doi: 10.1001/jamanetworkopen.2020.24406
Friends of Cancer Research. Considerations for Use of Real-World Evidence in Oncology 2020. Available from: https://friendsofcancerresearch.org/wp-content/uploads/Use_of_Real-World_Evidence_in_Oncology_0.pdf .
Freedman LS. Tables of the number of patients required in clinical trials using the log-rank test. Stat Med. 1982;1:121–9.
doi: 10.1002/sim.4780010204
pubmed: 7187087
Rosner B. Fundamentals of Biostatistics. 6th ed. Belmont: Thomson Brooks/Cole; 2006.
Joshi M, Pustejovsky J. Simhelpers: Helper Functions for Simulation Studies. R package version 0.1.2. 2022. https://CRAN.R-project.org/package=simhelpers .