Impact of selective reporting bias on stroke trials: potential compromise in evidence synthesis - A cross-sectional study.
Acute ischemic stroke
Evidence synthesis
Randomized controlled trials
Selective reporting bias
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
28 Oct 2024
28 Oct 2024
Historique:
received:
09
04
2024
accepted:
21
10
2024
medline:
29
10
2024
pubmed:
29
10
2024
entrez:
29
10
2024
Statut:
epublish
Résumé
Accurate reporting of outcomes is crucial for interpreting the results of randomized controlled trials (RCTs). However, selectively reporting outcomes in publications to achieve researchers' anticipated results still occurs frequently. This study aims to investigate the prevalence of selective reporting of outcomes in RCTs on treating acute ischemic stroke (AIS), identify factors contributing to this issue, and assess its potential impact on the degree and direction of intervention effect. A search was conducted in MEDLINE, Embase, and the Cochrane Library to collect interventional RCTs on AIS published from 2020 to 2022. Full texts of RCTs were reviewed, and only those reporting International Clinical Trials Registry Platform primary registry numbers were included. Registration information of the RCTs was extracted from the registry platforms and compared with the publications' details to assess the selective reporting of outcomes. Bayesian multilevel logistic regression was used to analyze the reasons behind selective reporting. Among the total of 159 AIS RCTs identified, 82 (51.6%) were ultimately included, as they reported registration numbers, which encompassed 819 outcomes. Among them, 72 RCTs (87.8%) and 497 outcomes (60.7%) exhibited selective reporting. Omission-type selective reporting (downgrading, omitting, or ambiguously reporting) accounted for 36.4%, while addition-type selective reporting (upgrading, adding, or altering the measurement scope of outcomes) comprised 63.6%. Omission-type selective reporting correlated with negative results (OR: 7.39; 95% CI: 4.08-13.44), whereas addition-type selective reporting correlated with positive results (OR: 2.07; 95% CI: 1.34-3.26) and publication in journals that are not in the top quartile of the Journal Citation Reports (OR: 2.48; 95% CI: 1.15-5.38). Registered interventional AIS RCTs still face significant issues regarding selective reporting of outcomes. Therefore, it is necessary to further evaluate the influence of selective reporting bias on the positive results obtained from individual AIS RCTs and the systematic reviews based on these RCTs.
Sections du résumé
BACKGROUND
BACKGROUND
Accurate reporting of outcomes is crucial for interpreting the results of randomized controlled trials (RCTs). However, selectively reporting outcomes in publications to achieve researchers' anticipated results still occurs frequently. This study aims to investigate the prevalence of selective reporting of outcomes in RCTs on treating acute ischemic stroke (AIS), identify factors contributing to this issue, and assess its potential impact on the degree and direction of intervention effect.
METHODS
METHODS
A search was conducted in MEDLINE, Embase, and the Cochrane Library to collect interventional RCTs on AIS published from 2020 to 2022. Full texts of RCTs were reviewed, and only those reporting International Clinical Trials Registry Platform primary registry numbers were included. Registration information of the RCTs was extracted from the registry platforms and compared with the publications' details to assess the selective reporting of outcomes. Bayesian multilevel logistic regression was used to analyze the reasons behind selective reporting.
RESULTS
RESULTS
Among the total of 159 AIS RCTs identified, 82 (51.6%) were ultimately included, as they reported registration numbers, which encompassed 819 outcomes. Among them, 72 RCTs (87.8%) and 497 outcomes (60.7%) exhibited selective reporting. Omission-type selective reporting (downgrading, omitting, or ambiguously reporting) accounted for 36.4%, while addition-type selective reporting (upgrading, adding, or altering the measurement scope of outcomes) comprised 63.6%. Omission-type selective reporting correlated with negative results (OR: 7.39; 95% CI: 4.08-13.44), whereas addition-type selective reporting correlated with positive results (OR: 2.07; 95% CI: 1.34-3.26) and publication in journals that are not in the top quartile of the Journal Citation Reports (OR: 2.48; 95% CI: 1.15-5.38).
CONCLUSIONS
CONCLUSIONS
Registered interventional AIS RCTs still face significant issues regarding selective reporting of outcomes. Therefore, it is necessary to further evaluate the influence of selective reporting bias on the positive results obtained from individual AIS RCTs and the systematic reviews based on these RCTs.
Identifiants
pubmed: 39468435
doi: 10.1186/s12874-024-02381-5
pii: 10.1186/s12874-024-02381-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
255Informations de copyright
© 2024. The Author(s).
Références
Mortality and global health estimates. Available from: https://www.who.int/data/gho/data/themes/mortality-and-global-health-estimates . Cited 2024 Jan 28.
Ischemic Stroke (Clots) | American Stroke Association. Available from: https://www.stroke.org/en/about-stroke/types-of-stroke/ischemic-stroke-clots . Cited 2024 Jan 28.
Saini V, Guada L, Yavagal DR. Global Epidemiology of Stroke and Access to Acute Ischemic Stroke Interventions. Neurology. 2021;97(20 Suppl 2):S6–16.
pubmed: 34785599
Kenakin TP, Johnson SF. The importance of the alpha-adrenoceptor agonist activity of dobutamine to inotropic selectivity in the anaesthetized cat. Eur J Pharmacol. 1985;111(3):347–54.
doi: 10.1016/0014-2999(85)90641-7
pubmed: 2990956
Ding Q, Liu S, Yao Y, Liu H, Cai T, Han L. Global, Regional, and National Burden of Ischemic Stroke, 1990–2019. Neurology. 2022;98(3):e279–90.
doi: 10.1212/WNL.0000000000013115
pubmed: 34911748
Feigin VL, Norrving B, Mensah GA. Global Burden of Stroke. Circ Res. 2017;120(3):439–48.
doi: 10.1161/CIRCRESAHA.116.308413
pubmed: 28154096
Campbell BCV, Khatri P. Stroke Lancet. 2020;396(10244):129–42.
doi: 10.1016/S0140-6736(20)31179-X
pubmed: 32653056
Acute Ischemic Stroke Resources. Available from: https://www.stroke.org/en/professionals/stroke-resource-library/acute-ischemic-stroke-resources . Cited 2024 July 22.
Mokin M, Kass-Hout T, Kass-Hout O, Dumont TM, Kan P, Snyder KV, et al. Intravenous thrombolysis and endovascular therapy for acute ischemic stroke with internal carotid artery occlusion: a systematic review of clinical outcomes. Stroke. 2012;43(9):2362–8.
doi: 10.1161/STROKEAHA.112.655621
pubmed: 22811456
Nam J, Jing H, O’Reilly D. Intra-arterial thrombolysis vs. standard treatment or intravenous thrombolysis in adults with acute ischemic stroke: a systematic review and meta-analysis. Int J Stroke. 2015;10(1):13–22.
doi: 10.1111/j.1747-4949.2012.00914.x
pubmed: 23294872
Mullen MT, Pisapia JM, Tilwa S, Messé SR, Stein SC. Systematic review of outcome after ischemic stroke due to anterior circulation occlusion treated with intravenous, intra-arterial, or combined intravenous+intra-arterial thrombolysis. Stroke. 2012;43(9):2350–5.
doi: 10.1161/STROKEAHA.111.639211
pubmed: 22811451
Yoshimura S, Sakai N, Yamagami H, Uchida K, Beppu M, Toyoda K, et al. Endovascular Therapy for Acute Stroke with a Large Ischemic Region. N Engl J Med. 2022;386(14):1303–13.
doi: 10.1056/NEJMoa2118191
pubmed: 35138767
Sarraj A, Hassan AE, Abraham MG, Ortega-Gutierrez S, Kasner SE, Hussain MS, et al. Trial of Endovascular Thrombectomy for Large Ischemic Strokes. N Engl J Med. 2023;388(14):1259–71.
doi: 10.1056/NEJMoa2214403
pubmed: 36762865
Chen CJ, Ding D, Starke RM, Mehndiratta P, Crowley RW, Liu KC, et al. Endovascular vs medical management of acute ischemic stroke. Neurology. 2015;85(22):1980–90.
doi: 10.1212/WNL.0000000000002176
pubmed: 26537058
pmcid: 4664128
Fransen PSS, Berkhemer OA, Lingsma HF, Beumer D, van den Berg LA, Yoo AJ, et al. Time to Reperfusion and Treatment Effect for Acute Ischemic Stroke: A Randomized Clinical Trial. JAMA Neurol. 2016;73(2):190–6.
doi: 10.1001/jamaneurol.2015.3886
pubmed: 26716735
Guo Q, Jiang G, Zhao Q, Long Y, Feng K, Gu X, et al. Rapid review: A review of methods and recommendations based on current evidence. J Evid Based Med. 2024;17(2):434–53.
Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;18(343):d5928.
doi: 10.1136/bmj.d5928
Chen T, Li C, Qin R, Wang Y, Yu D, Dodd J, et al. Comparison of Clinical Trial Changes in Primary Outcome and Reported Intervention Effect Size Between Trial Registration and Publication. JAMA Netw Open. 2019;2(7):e197242.
doi: 10.1001/jamanetworkopen.2019.7242
pubmed: 31322690
pmcid: 6646984
ICMJE | Recommendations | Clinical Trials. Available from: https://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html . Cited 2024 Jan 28.
Sendyk DI, Souza NV, César Neto JB, Tatakis DN, Pannuti CM. Selective outcome reporting in root coverage randomized clinical trials. J Clin Periodontol. 2021;48(6):867–77.
doi: 10.1111/jcpe.13451
pubmed: 33745136
Kim DY, Oh S, Yoon HS. Complete and transparent reporting of primary end points of randomized trials in the dermatology literature: A comparison of registered and published primary end points. J Am Acad Dermatol. 2021;85(5):1201–8.
doi: 10.1016/j.jaad.2020.04.067
pubmed: 32334060
Vrljičak Davidović N, Komić L, Mešin I, Kotarac M, Okmažić D, Franić T. Registry versus publication: discrepancy of primary outcomes and possible outcome reporting bias in child and adolescent mental health. Eur Child Adolesc Psychiatry. 2022;31(5):757–69.
doi: 10.1007/s00787-020-01710-5
pubmed: 33459886
Vassar M, Roberts W, Cooper CM, Wayant C, Bibens M. Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials. Addiction. 2020;115(6):1172–9.
doi: 10.1111/add.14902
pubmed: 31743532
Aryan R, Jagroop D, Danells CJ, Rozanski G, Unger J, Huntley AH, et al. Publication Rate and Consistency of Registered Trials of Motor-Based Stroke Rehabilitation. Neurology. 2021;96(13):617–26.
doi: 10.1212/WNL.0000000000011660
pubmed: 33568550
pmcid: 8055311
Jia Y, Huang D, Wen J, Liang J, Qureshi R, Wang Y, et al. Association between switching of primary outcomes and reported trial findings among randomized drug trials from China. J Clin Epidemiol. 2021;132:10–7.
doi: 10.1016/j.jclinepi.2020.11.023
pubmed: 33309887
Singh B, Parsaik AK, Prokop LJ, Mittal MK. Endovascular therapy for acute ischemic stroke: a systematic review and meta-analysis. Mayo Clin Proc. 2013;88(10):1056–65.
doi: 10.1016/j.mayocp.2013.07.015
pubmed: 24079677
Minhas JS, Chithiramohan T, Wang X, Barnes SC, Clough RH, Kadicheeni M, et al. Oral antiplatelet therapy for acute ischaemic stroke. Cochrane Database Syst Rev. 2022;1(1):CD000029.
pubmed: 35028933
Xu C, Yu T, Furuya-Kanamori L, Lin L, Zorzela L, Zhou X, et al. Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study. BMJ. 2022;10(377):e069155.
doi: 10.1136/bmj-2021-069155
Vassar M, Roberts W, Cooper CM, Wayant C, Bibens M. Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials. Addiction. 2020;115(6):1172–9.
doi: 10.1111/add.14902
pubmed: 31743532
Unverzagt S, Prondzinsky R, Peinemann F. Single-center trials tend to provide larger treatment effects than multicenter trials: a systematic review. J Clin Epidemiol. 2013;66(11):1271–80.
doi: 10.1016/j.jclinepi.2013.05.016
pubmed: 23972520
Fuentes J, Armijo-Olivo S, da Costa BR, Ha C, Saltaji H, Arenti C, et al. Does Type of Sponsorship of Randomized Controlled Trials Influence Treatment Effect Size Estimates in Rehabilitation: A Meta-Epidemiological Study. Am J Phys Med Rehabil. 2020;99(10):909–16.
doi: 10.1097/PHM.0000000000001444
pubmed: 32960528
Yousefi-Nooraie R, Shakiba B, Mortaz-Hejri S. Country development and manuscript selection bias: a review of published studies. BMC Med Res Methodol. 2006;1(6):37.
doi: 10.1186/1471-2288-6-37
Zhai X, Cui J, Wang Y, Qu Z, Mu Q, Li P, et al. Quality of Reporting Randomized Controlled Trials in Five Leading Neurology Journals in 2008 and 2013 Using the Modified “Risk of Bias” Tool. World Neurosurg. 2017;99:687–694.e7.
doi: 10.1016/j.wneu.2016.12.136
pubmed: 28089838
Liebeskind DS, Kidwell CS, Sayre JW, Saver JL. Evidence of publication bias in reporting acute stroke clinical trials. Neurology. 2006;67(6):973–9.
doi: 10.1212/01.wnl.0000237331.16541.ac
pubmed: 17000963
Harriman SL, Patel J. When are clinical trials registered? An analysis of prospective versus retrospective registration. Trials. 2016;15(17):187.
doi: 10.1186/s13063-016-1310-8
Vevea JL, Woods CM. Publication bias in research synthesis: sensitivity analysis using a priori weight functions. Psychol Methods. 2005;10(4):428–43.
doi: 10.1037/1082-989X.10.4.428
pubmed: 16392998
Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;2009(1):MR000006.
pubmed: 19160345
pmcid: 8276556
Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.
doi: 10.1001/jama.291.20.2457
pubmed: 15161896
Loder E, Loder S, Cook S. Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years. BMJ Open. 2018Feb 16;8(2):e020037.
doi: 10.1136/bmjopen-2017-020037
pubmed: 29453302
pmcid: 5829901
ICMJE | About ICMJE | Disclosure Forms. Available from: https://www.icmje.org/about-icmje/faqs/conflict-of-interest-disclosure-forms/ . Cited 2024 July 22.
Health C for D and R. FDA. FDA; 2024. Overview of Device Regulation. Available from: https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/overview-device-regulation . Cited 2024 July 21.
Health C for D and R. FDA. FDA; 2024. Device Approvals and Clearances. Available from: https://www.fda.gov/medical-devices/products-and-medical-procedures/device-approvals-and-clearances . Cited 2024 July 21.
Research C for DE and. FDA. FDA; 2023. Development & Approval Process | Drugs. Available from: https://www.fda.gov/drugs/development-approval-process-drugs .
Snyder RJ. Lack of Transparency in Publishing Negative Clinical Trial Results. Clin Podiatr Med Surg. 2020;37(2):385–9.
doi: 10.1016/j.cpm.2019.12.013
pubmed: 32146990
Franco A, Malhotra N, Simonovits G. Underreporting in Psychology Experiments: Evidence From a Study Registry. Social Psychological and Personality Science. 2016;7(1):8–12.
doi: 10.1177/1948550615598377
Won J, Kim S, Bae I, Lee H. Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture. PLoS ONE. 2019;14(10):e0223305.
doi: 10.1371/journal.pone.0223305
pubmed: 31581278
pmcid: 6776391
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;15(340):c365.
doi: 10.1136/bmj.c365