Effects of simulated sample sizes on the mortality effect estimates in three randomized intensive care unit trials.
Anti-Ulcer Agents
/ therapeutic use
Computer Simulation
/ statistics & numerical data
Critical Care
/ methods
Denmark
Female
Finland
Gastrointestinal Hemorrhage
/ complications
Humans
Hydroxyethyl Starch Derivatives
/ therapeutic use
Iceland
Intensive Care Units
Isotonic Solutions
/ therapeutic use
Male
Middle Aged
Netherlands
Norway
Pantoprazole
/ therapeutic use
Plasma Substitutes
/ therapeutic use
Sample Size
Shock, Septic
/ complications
United Kingdom
Journal
Acta anaesthesiologica Scandinavica
ISSN: 1399-6576
Titre abrégé: Acta Anaesthesiol Scand
Pays: England
ID NLM: 0370270
Informations de publication
Date de publication:
08 2020
08 2020
Historique:
received:
19
11
2019
revised:
06
03
2020
accepted:
09
03
2020
pubmed:
3
4
2020
medline:
5
8
2021
entrez:
3
4
2020
Statut:
ppublish
Résumé
Randomized clinical trials (RCTs) are occasionally stopped prematurely before reaching their planned sample sizes. It has been suggested that early stopped RCTs are associated with under- and overestimation of the effect estimates. We simulated the effect of hypothetical premature stopping of three large RCTs done in the intensive care unit (ICU) setting. In this post hoc study, we simulated the impact of stopping trials early by calculating mortality effect estimates continuously after the inclusion of each individual patient in three large RCTs, that is the 6S trial on hydroxyethyl starch vs Ringer's acetate in sepsis in ICU, the TRISS trial on lower vs higher haemoglobin threshold for transfusion in septic shock in ICU and the SUP-ICU trial on pantoprazole in patients at risk for gastrointestinal bleeding in the ICU. The three trials included a total of 5087 patients; 798 from the 6S trial, 998 from the TRISS trial and 3291 patients from the SUP-ICU trial. The premature mortality effect estimates showed considerable fluctuations until at least 20%-30% of the sample size was included. The premature estimates became stable after inclusion of 205 patients (26% of the final sample size) in the 6S trial, 133 patients(13%) in the TRISS trial and 1926 patients(59%) in the SUP-ICU trial. In this post hoc study of three international RCTs within intensive care, we found that the simulated interim mortality effect estimates showed considerable fluctuations until at least 20%-30% of the sample size was included, but remained instable until the final sample sizes had been included. Thus, this study illustrates the necessity for cautious interpretations of prematurely stopped trials.
Sections du résumé
BACKGROUND
Randomized clinical trials (RCTs) are occasionally stopped prematurely before reaching their planned sample sizes. It has been suggested that early stopped RCTs are associated with under- and overestimation of the effect estimates. We simulated the effect of hypothetical premature stopping of three large RCTs done in the intensive care unit (ICU) setting.
METHODS
In this post hoc study, we simulated the impact of stopping trials early by calculating mortality effect estimates continuously after the inclusion of each individual patient in three large RCTs, that is the 6S trial on hydroxyethyl starch vs Ringer's acetate in sepsis in ICU, the TRISS trial on lower vs higher haemoglobin threshold for transfusion in septic shock in ICU and the SUP-ICU trial on pantoprazole in patients at risk for gastrointestinal bleeding in the ICU.
RESULTS
The three trials included a total of 5087 patients; 798 from the 6S trial, 998 from the TRISS trial and 3291 patients from the SUP-ICU trial. The premature mortality effect estimates showed considerable fluctuations until at least 20%-30% of the sample size was included. The premature estimates became stable after inclusion of 205 patients (26% of the final sample size) in the 6S trial, 133 patients(13%) in the TRISS trial and 1926 patients(59%) in the SUP-ICU trial.
CONCLUSIONS
In this post hoc study of three international RCTs within intensive care, we found that the simulated interim mortality effect estimates showed considerable fluctuations until at least 20%-30% of the sample size was included, but remained instable until the final sample sizes had been included. Thus, this study illustrates the necessity for cautious interpretations of prematurely stopped trials.
Substances chimiques
Anti-Ulcer Agents
0
Hydroxyethyl Starch Derivatives
0
Isotonic Solutions
0
Plasma Substitutes
0
Ringer's acetate
0
Pantoprazole
D8TST4O562
Types de publication
Journal Article
Multicenter Study
Observational Study
Randomized Controlled Trial
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
976-981Informations de copyright
© 2020 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.
Références
Suresh K, Chandrashekara S. Sample size estimation and power analysis for clinical research studies. J Hum Reprod Sci. 2012;5:7-13.
Weber EJ, Hoo ZH. Why sample size estimates? Emerg Med J. 2018;35:755-756.
Ferrarotto R, Redman MW, Gandara DR, Herbst RS, Papadimitrakopoulou VA. Lung-MAP-framework, overview, and design principles. Chin Clin Oncol. 2015;4(3):36.
Leandro G. How to interpret a randomized controlled study stopped early. Intens Care Med. 2013;39:1642-1643.
Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation. Control Clin Trials. 1989;10:209-221.
Montori VM, Devereaux PJ, Adhikari NKJ, et al. Randomized trials stopped early for benefit. JAMA. 2005;294(17):2203.
Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects;systematic review and meta-regression analysis. JAMA. 2010;303:1180-1187.
Freidlin B, Korn EL. Stopping clinical trials early for benefit: impact on estimation. Clin Trials. 2009;6:119-125.
Guyatt GH, Briel M, Glasziou P, Bassler D, Montori VM. Problems of stopping trials early. BMJ. 2012;344:e3863.
Wang H, Rosner GL, Goodman SN. Quantifying over-estimation in early stopped clinical trials and the "freezing effect" on subsequent research. Clin Trials. 2016;13:621-631.
Perner A, Haase N, Guttormsen AB, et al. Hydroxyethyl starch 130/0.42 versus ringer’s acetate in severe sepsis. N Engl J Med. 2012;367:124-134.
Holst LB, Haase N, Wetterslev J, et al. Lower versus higher hemoglobin threshold for transfusion in septic shock. N Engl J Med. 2014;371:1381-1391.
Krag M, Marker S, Perner A, et al. Pantoprazole in patients at risk for gastrointestinal bleeding in the ICU. N Engl J Med. 2018;379:2199-2208.
Kofoed A, Perner A, Marker S, Haase N, Holst LB, Møller MH. Effects of simulated sample sizes on mortality estimates-protocol for a study in 3 randomised ICU trials. Acta Anaesthesiol Scand. 2019;63(8):1098-1101.
von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg. 2014;12:1495-1499.
Krag M, Perner A, Wetterslev J, et al. SUP-ICU investigators the S-I. Stress ulcer prophylaxis with a proton pump inhibitor versus placebo in critically ill patients (SUP-ICU trial): study protocol for a randomised controlled trial. Trials. 2016;17:205.
Holst LB, Haase N, Wetterslev J, et al. Transfusion requirements in septic shock (TRISS) trial - comparing the effects and safety of liberal versus restrictive red blood cell transfusion in septic shock patients in the ICU: protocol for a randomised controlled trial. Trials. 2013;14:150.
Perner A, Haase N, Wetterslev J, et al. Comparing the effect of hydroxyethyl starch 130/0.4 with balanced crystalloid solution on mortality and kidney failure in patients with severe sepsis (6S - Scandinavian Starch for Severe Sepsis/Septic Shock trial): study protocol, design and rationale for a double-blinded, randomised clinical trial. Trials. 2011;12:24.
Fagerland MW, Lydersen S, Laake P. Recommended confidence intervals for two independent binomial proportions. Stat Methods Med Res. 2015;24(2):224-225.
Holzer M, Sterz F, Darby JM, et al. Mild therapeutic hypothermia to improve the neurologic outcome after cardiac arrest. N Engl J Med. 2002;346:549-556.
O’Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics. 1979;35:549-556.
Demets DL, Lan KKG. Interim analysis: the alpha spending function approach. Stat Med. 1994;13:1341-1352.
Gerke O, Vilstrup MH, Halekoh U, Hildebrandt MG, Høilund-Carlsen PF. Group-sequential analysis may allow for early trial termination: illustration by an intra-observer repeatability study. EJNMMI Res. 2017;7:79.
Zhu L, Yu Q, Mercante DE. A Bayesian sequential design for clinical trials with time-to-event outcomes. Stat Biopharm Res. 2019;11:387-397.
Yin G, Lam CK, Shi H. Bayesian randomized clinical trials: from fixed to adaptive design. Contemp Clin Trials. 2017;59:77-86.
Zhu H, Yu Q, Mercante DE. A Bayesian sequential design with binary outcome. Pharm Stat. 2017;16:192-200.
Lewis RJ, Lipsky AM, Berry DA. Bayesian decision-theoretic group sequential clinical trial design based on a quadratic loss function: a frequentist evaluation. Clin Trials. 2007;4:5-14.
Bassler D, Montori VM, Briel M, et al. Reflections on meta-analyses involving trials stopped early for benefit: is there a problem and if so, what is it?Stat. Methods Med Res. 2013;22:159-168.
Schou IM, Marschner IC. Meta-analysis of clinical trials with early stopping: an investigation of potential bias. Stat Med. 2013;32:4859-4874.
Bassler D, Ferreira-Gonzalez I, Briel M, et al. Systematic reviewers neglect bias that results from trials stopped early for benefit. J Clin Epidemiol. 2007;60:869-873.
Fairfield CJ, Connor S, Harrison EM, Wigmore SJ. The harms of early cessation of trials on systematic reviews. Lancet Gastroenterol Hepatol. 2019;4:667.
Tierney JF, Vale C, Riley R, et al. Individual participant data (IPD) meta-analyses of randomised controlled trials: guidance on their use. PLoS Medicine. 2015;12(7):e1001855.
Savović J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157:429.