The Fragility Index of randomized controlled trials in pediatric anesthesiology.
L’indice de fragilité des études randomisées contrôlées en anesthésiologie pédiatrique.
Fragility Index
pediatric anesthesia
randomized controlled trial
research methodology
Journal
Canadian journal of anaesthesia = Journal canadien d'anesthesie
ISSN: 1496-8975
Titre abrégé: Can J Anaesth
Pays: United States
ID NLM: 8701709
Informations de publication
Date de publication:
09 2023
09 2023
Historique:
received:
29
08
2022
accepted:
23
01
2023
revised:
16
01
2023
medline:
24
8
2023
pubmed:
8
6
2023
entrez:
7
6
2023
Statut:
ppublish
Résumé
The P value is a widely used measure of statistical importance but has many drawbacks and limitations, one being that it does not reflect the robustness of the results of a clinical trial. The Fragility Index (FI) was developed as a measure of how many outcome events would need to change to nonevents to render a significant P value nonsignificant (P ≥ 0.05). The FI of trials from other medical specialties is typically < 5. We aimed to determine the FI of pediatric anesthesiology randomized controlled trials (RCT) and to test for association with various characteristics of the included trials. We conducted a comprehensive systematic search of high-impact anesthesia, surgical, and medical journals from the last 25 years for trials comparing an intervention between two groups with a statistically significant P value (< 0.05) for a dichotomous outcome. We also compared FI values for variables that reflect the quality and importance of a trial. The median [interquartile range] FI was 3 [1-7] and correlated positively with the number of participants (r The FI of published trials in pediatric anesthesiology is similarly low as in other medical specialties. Larger trials with more events and P values ≤ 0.01 were associated with a higher FI. RéSUMé: OBJECTIF: La valeur P est une mesure d’importance statistique largement utilisée, mais elle présente de nombreux inconvénients et limites, notamment parce qu’elle ne reflète pas la robustesse des résultats d’une étude clinique. L’indice de fragilité (IF) a été mis au point pour mesurer le nombre d’événements du critère d’évaluation qui devraient se transformer en non-événements pour obtenir une valeur P non significative (P ≥ 0,05). L’IF des études d’autres spécialités médicales est généralement < 5. Notre objectif était de déterminer l’IF des études randomisées contrôlées (ERC) en anesthésiologie pédiatrique et de tester l’association avec diverses caractéristiques des études incluses. MéTHODE: Nous avons réalisé une recherche systématique exhaustive dans les revues d’anesthésie, de chirurgie et médicales à fort impact des 25 dernières années pour trouver des études comparant une intervention entre deux groupes avec une valeur P significative d’un point de vue statistique (< 0,05) pour un résultat dichotomique. Nous avons également comparé les valeurs d’IF pour les variables qui reflètent la qualité et l’importance d’une étude. RéSULTATS: L’IF médian [écart interquartile] était de 3 [1 à 7] et était positivement corrélé avec le nombre de participant·es (r
Autres résumés
Type: Publisher
(fre)
RéSUMé: OBJECTIF: La valeur P est une mesure d’importance statistique largement utilisée, mais elle présente de nombreux inconvénients et limites, notamment parce qu’elle ne reflète pas la robustesse des résultats d’une étude clinique. L’indice de fragilité (IF) a été mis au point pour mesurer le nombre d’événements du critère d’évaluation qui devraient se transformer en non-événements pour obtenir une valeur P non significative (P ≥ 0,05). L’IF des études d’autres spécialités médicales est généralement < 5. Notre objectif était de déterminer l’IF des études randomisées contrôlées (ERC) en anesthésiologie pédiatrique et de tester l’association avec diverses caractéristiques des études incluses. MéTHODE: Nous avons réalisé une recherche systématique exhaustive dans les revues d’anesthésie, de chirurgie et médicales à fort impact des 25 dernières années pour trouver des études comparant une intervention entre deux groupes avec une valeur P significative d’un point de vue statistique (< 0,05) pour un résultat dichotomique. Nous avons également comparé les valeurs d’IF pour les variables qui reflètent la qualité et l’importance d’une étude. RéSULTATS: L’IF médian [écart interquartile] était de 3 [1 à 7] et était positivement corrélé avec le nombre de participant·es (r
Identifiants
pubmed: 37286747
doi: 10.1007/s12630-023-02513-3
pii: 10.1007/s12630-023-02513-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1449-1460Informations de copyright
© 2023. Canadian Anesthesiologists' Society.
Références
Shafer SL, Dexter F. Publication bias, retrospective bias, and reproducibility of significant results in observational studies. Anesth Analg 2012; 114: 931–2. https://doi.org/10.1213/ane.0b013e31824a0b5b
doi: 10.1213/ane.0b013e31824a0b5b
pubmed: 22523409
Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Stat 2016; 70: 129–33. https://doi.org/10.1080/00031305.2016.1154108
doi: 10.1080/00031305.2016.1154108
Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol 2014; 67: 622–8. https://doi.org/10.1016/j.jclinepi.2013.10.019
doi: 10.1016/j.jclinepi.2013.10.019
pubmed: 24508144
Ahmed W, Fowler RA, McCredie VA. Does sample size matter when interpreting the fragility index? Crit Care Med 2016; 44: e1142. https://doi.org/10.1097/ccm.0000000000001976
Mazzinari G, Ball L, Serpa Neto A, et al. The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature. Br J Anaesth 2018; 120: 935–41. https://doi.org/10.1016/j.bja.2018.01.012
doi: 10.1016/j.bja.2018.01.012
pubmed: 29661411
Rickard M, Lorenzo AJ, Hannick JH, Blais AS, Koyle MA, Bägli DJ. Over-reliance on P values in urology: fragility of findings in the hydronephrosis literature calls for systematic reporting of robustness indicators. Urology 2019; 133: 204–10. https://doi.org/10.1016/j.urology.2019.03.045
doi: 10.1016/j.urology.2019.03.045
pubmed: 31374290
Evaniew N, Files C, Smith C, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 2015; 15: 2188–97. https://doi.org/10.1016/j.spinee.2015.06.004
doi: 10.1016/j.spinee.2015.06.004
pubmed: 26072464
Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 2016; 44: 1278–84. https://doi.org/10.1097/ccm.0000000000001670
doi: 10.1097/ccm.0000000000001670
pubmed: 26963326
Khan M, Evaniew N, Gichuru M, et al. The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med 2017; 45: 2164–70. https://doi.org/10.1177/0363546516674469
doi: 10.1177/0363546516674469
pubmed: 27895038
Docherty KF, Campbell RT, Jhund PS, Petrie MC, McMurray JJ. How robust are clinical trials in heart failure? Eur Heart J 2017; 38: 338–45. https://doi.org/10.1093/eurheartj/ehw427
doi: 10.1093/eurheartj/ehw427
pubmed: 27742808
Chow JTY, Turkstra TP, Yim E, Jones PM. Sample size calculations for randomized clinical trials published in anesthesiology journals: a comparison of 2010 versus 2016. Can J Anesth 2018; 65: 611–8. https://doi.org/10.1007/s12630-018-1109-z
doi: 10.1007/s12630-018-1109-z
pubmed: 29569142
Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996; 276: 637–9. https://doi.org/10.1001/jama.276.8.637
doi: 10.1001/jama.276.8.637
pubmed: 8773637
Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366: I4898. https://doi.org/10.1136/bmj.l4898
doi: 10.1136/bmj.l4898
Higgins JP, Savović J, Page MJ, Elberrs RG, Sterne JA. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JP, Thomas J, Chandler J, et al. (Eds.). Cochrane Handbook for Systematic Reviews of Interventions, 2
Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale: L. Erlbaum Associates; 1988.
Ortega JL. Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data. Scientometrics 2018; 116: 2123–38. https://doi.org/10.1007/s11192-018-2838-z
GitHub. rAltmetric: Retrieves altmetric data for any published paper from altmetrics.com. R package version 0.7. Available from URL: https://github.com/ropensci/rAltmetric (accessed February 2023).
Holek M, Bdair F, Khan M, et al. Fragility of clinical trials across research fields: a synthesis of methodological reviews. Contemp Clin Trials 2020; 97: 106151. https://doi.org/10.1016/j.cct.2020.106151
Goerke K, Parke M, Horn J, et al. Are results from randomized trials in anesthesiology robust or fragile? An analysis using the fragility index. Int J Evid Based Healthc 2020; 18: 116–24. https://doi.org/10.1097/xeb.0000000000000200
doi: 10.1097/xeb.0000000000000200
pubmed: 31415254
Turner RM, Bird SM, Higgins JP. The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews. PLoS One 2013; 8: e59202. https://doi.org/10.1371/journal.pone.0059202
Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P. Reporting of sample size calculation in randomised controlled trials: review. BMJ 2009; 338: b1732. https://doi.org/10.1136/bmj.b1732
Daniels JR, Dexter F, Espy JL, Brull SJ. Quantitative assessment of statistical reviews of patient safety research articles. J Patient Saf 2019; 15: 184–90. https://doi.org/10.1097/pts.0000000000000391
doi: 10.1097/pts.0000000000000391
pubmed: 28590949
Dexter F, Shafer SL. Narrative review of statistical reporting checklists, mandatory statistical editing, and rectifying common problems in the reporting of scientific articles. Anesth Analg 2017; 124: 943–7. https://doi.org/10.1213/ane.0000000000001593
doi: 10.1213/ane.0000000000001593
pubmed: 27676281
Smith SM, Dworkin RH. Prospective clinical trial registration: not sufficient, but always necessary. Anaesthesia 2018; 73: 538–41. https://doi.org/10.1111/anae.14189
doi: 10.1111/anae.14189
pubmed: 29292500
Tahamtan I, Safipour Afshar A, Ahamdzadeh K. Factors affecting number of citations: a comprehensive review of the literature. Scientometrics 2016; 107: 1195–225. https://doi.org/10.1007/s11192-016-1889-2
doi: 10.1007/s11192-016-1889-2
Fassoulaki A, Vassi A, Kardasis A, Chantziara V. Altmetrics should not be used for ranking of anaesthesia journals. Br J Anaesth 2018; 121: 514–6. https://doi.org/10.1016/j.bja.2018.05.048
doi: 10.1016/j.bja.2018.05.048
pubmed: 30032902
Shochet LR, Kerr PG, Polkinghorne KR. The fragility of significant results underscores the need of larger randomized controlled trials in nephrology. Kidney Int 2017; 92: 1469–75. https://doi.org/10.1016/j.kint.2017.05.011
doi: 10.1016/j.kint.2017.05.011
pubmed: 28754551
Gnech M, Lovatt CA, McGrath M, et al. Quality of reporting and fragility index for randomized controlled trials in the vesicoureteral reflux literature: where do we stand? J Pediatr Urol 2019; 15: 204–12. https://doi.org/10.1016/j.jpurol.2019.02.014
doi: 10.1016/j.jpurol.2019.02.014
pubmed: 31060965
Costi D, Ellwood J, Wallace A, Ahmed S, Waring L, Cyna A. Transition to propofol after sevoflurane anesthesia to prevent emergence agitation: a randomized controlled trial. Paediatr Anaesth 2015; 25: 517–23. https://doi.org/10.1111/pan.12617
doi: 10.1111/pan.12617
pubmed: 25586124
Caldwell JM, Youssefzadeh K, Limpisvasti O. A method for calculating the fragility index of continuous outcomes. J Clin Epidemiol 2021; 136: 20–5. https://doi.org/10.1016/j.jclinepi.2021.02.023
doi: 10.1016/j.jclinepi.2021.02.023
pubmed: 33684509
Potter GE. Dismantling the Fragility Index: a demonstration of statistical reasoning. Stat Med 2020; 39: 3720–31. https://doi.org/10.1002/sim.8689
doi: 10.1002/sim.8689
pubmed: 32781488