Tolerating bad health research: the continuing scandal.
Methodologists
Randomised trials
Research waste
Risk of bias
Statisticians
Journal
Trials
ISSN: 1745-6215
Titre abrégé: Trials
Pays: England
ID NLM: 101263253
Informations de publication
Date de publication:
02 Jun 2022
02 Jun 2022
Historique:
received:
26
11
2021
accepted:
18
05
2022
entrez:
2
6
2022
pubmed:
3
6
2022
medline:
7
6
2022
Statut:
epublish
Résumé
At the 2015 REWARD/EQUATOR conference on research waste, the late Doug Altman revealed that his only regret about his 1994 BMJ paper 'The scandal of poor medical research' was that he used the word 'poor' rather than 'bad'. But how much research is bad? And what would improve things? We focus on randomised trials and look at scale, participants and cost. We randomly selected up to two quantitative intervention reviews published by all clinical Cochrane Review Groups between May 2020 and April 2021. Data including the risk of bias, number of participants, intervention type and country were extracted for all trials included in selected reviews. High risk of bias trials was classed as bad. The cost of high risk of bias trials was estimated using published estimates of trial cost per participant. We identified 96 reviews authored by 546 reviewers from 49 clinical Cochrane Review Groups that included 1659 trials done in 84 countries. Of the 1640 trials providing risk of bias information, 1013 (62%) were high risk of bias (bad), 494 (30%) unclear and 133 (8%) low risk of bias. Bad trials were spread across all clinical areas and all countries. Well over 220,000 participants (or 56% of all participants) were in bad trials. The low estimate of the cost of bad trials was £726 million; our high estimate was over £8 billion. We have five recommendations: trials should be neither funded (1) nor given ethical approval (2) unless they have a statistician and methodologist; trialists should use a risk of bias tool at design (3); more statisticians and methodologists should be trained and supported (4); there should be more funding into applied methodology research and infrastructure (5). Most randomised trials are bad and most trial participants will be in one. The research community has tolerated this for decades. This has to stop: we need to put rigour and methodology where it belongs - at the centre of our science.
Sections du résumé
BACKGROUND
BACKGROUND
At the 2015 REWARD/EQUATOR conference on research waste, the late Doug Altman revealed that his only regret about his 1994 BMJ paper 'The scandal of poor medical research' was that he used the word 'poor' rather than 'bad'. But how much research is bad? And what would improve things?
MAIN TEXT
METHODS
We focus on randomised trials and look at scale, participants and cost. We randomly selected up to two quantitative intervention reviews published by all clinical Cochrane Review Groups between May 2020 and April 2021. Data including the risk of bias, number of participants, intervention type and country were extracted for all trials included in selected reviews. High risk of bias trials was classed as bad. The cost of high risk of bias trials was estimated using published estimates of trial cost per participant. We identified 96 reviews authored by 546 reviewers from 49 clinical Cochrane Review Groups that included 1659 trials done in 84 countries. Of the 1640 trials providing risk of bias information, 1013 (62%) were high risk of bias (bad), 494 (30%) unclear and 133 (8%) low risk of bias. Bad trials were spread across all clinical areas and all countries. Well over 220,000 participants (or 56% of all participants) were in bad trials. The low estimate of the cost of bad trials was £726 million; our high estimate was over £8 billion. We have five recommendations: trials should be neither funded (1) nor given ethical approval (2) unless they have a statistician and methodologist; trialists should use a risk of bias tool at design (3); more statisticians and methodologists should be trained and supported (4); there should be more funding into applied methodology research and infrastructure (5).
CONCLUSIONS
CONCLUSIONS
Most randomised trials are bad and most trial participants will be in one. The research community has tolerated this for decades. This has to stop: we need to put rigour and methodology where it belongs - at the centre of our science.
Identifiants
pubmed: 35655288
doi: 10.1186/s13063-022-06415-5
pii: 10.1186/s13063-022-06415-5
pmc: PMC9161194
doi:
Types de publication
Letter
Langues
eng
Sous-ensembles de citation
IM
Pagination
458Subventions
Organisme : Chief Scientist Office
ID : HSRU1
Pays : United Kingdom
Organisme : Health Research Board
ID : TMRN-2017-1
Pays : Ireland
Informations de copyright
© 2022. The Author(s).
Références
Evid Based Med. 2017 Aug;22(4):120-122
pubmed: 28720642
BMJ Open. 2020 Jun 11;10(6):e038863
pubmed: 32532786
BMJ Open. 2017 Mar 20;7(3):e015276
pubmed: 28320800
Lancet. 2014 Jan 11;383(9912):156-65
pubmed: 24411644
J Clin Epidemiol. 2021 Oct;138:219-226
pubmed: 34077797
BMJ. 1994 Jan 29;308(6924):283-4
pubmed: 8124111
Lancet. 2014 Jan 11;383(9912):166-75
pubmed: 24411645
Cochrane Database Syst Rev. 2019 Oct 3;10:ED000142
pubmed: 31643080
Lancet. 2009 Jul 4;374(9683):86-9
pubmed: 19525005
Lancet. 2014 Jan 11;383(9912):101-4
pubmed: 24411643
PLoS Med. 2015 Sep 22;12(9):e1001864
pubmed: 26393914
BMJ. 2019 Aug 28;366:l4898
pubmed: 31462531
Trials. 2018 Feb 2;19(1):87
pubmed: 29394905
BMJ. 2015 Mar 24;350:h809
pubmed: 25804210
BMJ. 2012 Nov 28;345:e8104
pubmed: 23190914