What is the role of randomised trials in implementation science?
Cluster randomised controlled trials
Implementation science
Quality improvement
Journal
Trials
ISSN: 1745-6215
Titre abrégé: Trials
Pays: England
ID NLM: 101263253
Informations de publication
Date de publication:
16 Aug 2023
16 Aug 2023
Historique:
received:
08
05
2023
accepted:
04
08
2023
medline:
18
8
2023
pubmed:
17
8
2023
entrez:
16
8
2023
Statut:
epublish
Résumé
There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science. There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies. Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.
Sections du résumé
BACKGROUND
BACKGROUND
There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.
MAIN BODY
METHODS
There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.
CONCLUSION
CONCLUSIONS
Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.
Identifiants
pubmed: 37587521
doi: 10.1186/s13063-023-07578-5
pii: 10.1186/s13063-023-07578-5
pmc: PMC10428627
doi:
Types de publication
Randomized Controlled Trial
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
537Subventions
Organisme : National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care Yorkshire and Humber
ID : NIHR200166
Informations de copyright
© 2023. BioMed Central Ltd., part of Springer Nature.
Références
Seddon ME, Marshall MN, Campbell SM, Roland MO. Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand. Qual Health Care. 2001;10:152–8.
pubmed: 11533422
pmcid: 1743427
doi: 10.1136/qhc.0100152
Steel N, Bachmann M, Maisey S, Shekelle P, Breeze E, Marmot M, Melzer D. Self reported receipt of care consistent with 32 quality indicators: national population survey of adults aged 50 or more in England. Bmj. 2008;337:a957.
pubmed: 18703659
pmcid: 2515887
doi: 10.1136/bmj.a957
Zeitlin J, Manktelow BN, Piedvache A, Cuttini M, Boyle E, van Heijst A, Gadzinowski J, Van Reempts P, Huusom L, Weber T, et al. Use of evidence based practices to improve survival without severe morbidity for very preterm infants: results from the EPICE population based cohort. Bmj. 2016;354:i2976.
pubmed: 27381936
pmcid: 4933797
doi: 10.1136/bmj.i2976
Wright AA, Cronin A, Milne DE, Bookman MA, Burger RA, Cohn DE, Cristea MC, Griggs JJ, Keating NL, Levenback CF, et al. Use and effectiveness of intraperitoneal chemotherapy for treatment of ovarian cancer. J Clin Oncol. 2015;33(26):2841–7.
pubmed: 26240233
pmcid: 4554746
doi: 10.1200/JCO.2015.61.4776
Chung SC, Sundstrom J, Gale CP, James S, Deanfield J, Wallentin L, Timmis A, Jernberg T, Hemingway H. Comparison of hospital variation in acute myocardial infarction care and outcome between Sweden and United Kingdom: population based cohort study using nationwide clinical registries. BMJ. 2015;351:h3913.
pubmed: 26254445
pmcid: 4528190
doi: 10.1136/bmj.h3913
Levine DM, Linder JA, Landon BE. The quality of outpatient care delivered to adults in the United States, 2002 to 2013. JAMA Intern Med. 2016;176(12):1778–90.
Turner GM, Calvert M, Feltham MG, Ryan R, Fitzmaurice D, Cheng KK, Marshall T. Under-prescribing of prevention drugs and primary prevention of stroke and transient ischaemic attack in UK general practice: a retrospective analysis. PLoS Med. 2016;13(11): e1002169.
Bucholz EM, Butala NM, Normand SL, Wang Y, Krumholz HM. Association of guideline-based admission treatments and life expectancy after myocardial infarction in elderly medicare beneficiaries. J Am Coll Cardiol. 2016;67(20):2378–91.
pubmed: 27199062
pmcid: 5097252
doi: 10.1016/j.jacc.2016.03.507
Serumaga B, Ross-Degnan D, Avery A, Elliott R, Majumdar SR, Zhang F, Soumerai SB. Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: interrupted time series study. BMJ. 2011;342:d108.
pubmed: 21266440
pmcid: 3026849
doi: 10.1136/bmj.d108
Campbell SM, Roland MO, Middleton E, Reeves D. Improvements in quality of clinical care in English general practice 1998–2003: longitudinal observational study. BMJ. 2005;331(7525):1121.
pubmed: 16257992
pmcid: 1283276
doi: 10.1136/bmj.38632.611123.AE
Fleming-Dutra KE, Hersh AL, Shapiro DJ, Bartoces M, Enns EA, File TM Jr, Finkelstein JA, Gerber JS, Hyun DY, Linder JA, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA. 2016;315(17):1864–73.
pubmed: 27139059
doi: 10.1001/jama.2016.4151
Foy R, Leaman B, McCrorie C, Petty D, House A, Bennett M, Carder P, Faulkner S, Glidewell L, West R. Prescribed opioids in primary care: cross-sectional and longitudinal analyses of influence of patient and practice characteristics. BMJ Open. 2016;6(5):e010276.
pubmed: 27178970
pmcid: 4874107
doi: 10.1136/bmjopen-2015-010276
Yamey G, Garcia P, Hassan F, Mao W, McDade KK, Pai M, Saha S, Schellekens P, Taylor A, Udayakumar K. It is not too late to achieve global covid-19 vaccine equity. BMJ. 2022;376:e070650.
pubmed: 35331982
pmcid: 8943596
doi: 10.1136/bmj-2022-070650
Cooksey R. A review of UK health research funding. Norwich: HMSO; 2006. Accessed 4 Sept 2022.
Roland M. Linking physicians’ pay to the quality of care – a major experiment in the United Kingdom. N Engl J Med. 2004;351:1448–54.
pubmed: 15459308
doi: 10.1056/NEJMhpr041294
Ryan AM, Krinsky S, Kontopantelis E, Doran T. Long-term evidence for the effect of pay-for-performance in primary care on mortality in the UK: a population study. Lancet. 2016;388(10041):268–74.
Auerbach A, Landefeld C, Shojania K. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;357:608–13.
pubmed: 17687138
doi: 10.1056/NEJMsb070738
Foy R, Eccles M, Grimshaw J. Why does primary care need more implementation research? Family Practice. 2001;18:353–5.
pubmed: 11477039
doi: 10.1093/fampra/18.4.353
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.
pubmed: 22651257
pmcid: 3462671
doi: 10.1186/1748-5908-7-50
Furler J, O’Neal D, Speight J, Manski-Nankervis J-A, Gorelik A, Holmes-Truscott E, Ginnivan L, Young D, Best J, Patterson E, et al. Supporting insulin initiation in type 2 diabetes in primary care: results of the Stepping Up pragmatic cluster randomised controlled clinical trial. BMJ. 2017;356:j783.
pubmed: 28274941
pmcid: 6287657
doi: 10.1136/bmj.j783
Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, Wilsdon J, Matowe L, Needham G, Gilbert F, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001;357:1406–9.
pubmed: 11356439
doi: 10.1016/S0140-6736(00)04564-5
Eccles M, Grimshaw JM, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and quality improvement strategies. Qual Saf Health Care. 2003;12:47–52.
pubmed: 12571345
pmcid: 1743658
doi: 10.1136/qhc.12.1.47
Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, Taljaard M, Wiggers J, Sutherland R, Nathan N, et al. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021;372:m3721.
pubmed: 33461967
pmcid: 7812444
doi: 10.1136/bmj.m3721
Davies HTO (ed), Nutley S (ed), Smith PC (ed). What works? Evidence-based policy and practice in public services. Bristol: Policy Press; 2000.
Berwick DM. The science of improvement. JAMA. 2008;283:2275–80.
Oliver D. Should practical quality improvement have parity of esteem with evidence based medicine? BMJ. 2017;357:j2582.
pubmed: 28559288
doi: 10.1136/bmj.j2582
Lamont T, Barber N, de Pury J, Fulop N, Garfield-Birkbeck S, Lilford R, Mear L, Raine R, Fitzpatrick R. New approaches to evaluating complex health and care systems. BMJ. 2016;352:i154.
pubmed: 26830458
doi: 10.1136/bmj.i154
Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. 2019;17(1):88.
pubmed: 31064388
pmcid: 6505277
doi: 10.1186/s12916-019-1322-9
Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, Boyd KA, Craig N, French DP, McIntosh E, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.
pubmed: 34593508
pmcid: 8482308
doi: 10.1136/bmj.n2061
Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: a critical analysis of the experimental literature. Milbank Q. 1989;67:268–317.
pubmed: 2698446
doi: 10.2307/3350142
Foy R, Hempel S, Rubenstein L, Suttorp M, Seelig M, Shanman R, Shekelle P. Meta-analysis: effect of interactive communication between collaborating primary care physicians and specialists. Ann Intern Med. 2010;152:247–58.
pubmed: 20157139
doi: 10.7326/0003-4819-152-4-201002160-00010
Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015;350:h2750.
pubmed: 26058820
pmcid: 4460815
doi: 10.1136/bmj.h2750
Venkataramani AS, Bor J, Jena AB. Regression discontinuity designs in healthcare research. Bmj. 2016;352:i1216.
pubmed: 26977086
pmcid: 6884311
doi: 10.1136/bmj.i1216
Barnighausen T, Tugwell P, Rottingen JA, Shemilt I, Rockers P, Geldsetzer P, Lavis J, Grimshaw J, Daniels K, Brown A, et al. Quasi-experimental study designs series-paper 4: uses and value. J Clin Epidemiol. 2017;89:21–9.
pubmed: 28365303
doi: 10.1016/j.jclinepi.2017.03.012
Barr B, Zhang X, Green M, Buchan I. A blueprint for synthetic control methodology: a causal inference tool for evaluating natural experiments in population health. BMJ. 2022;379:o2712.
pubmed: 36418028
doi: 10.1136/bmj.o2712
Fretheim A, Zhang F, Ross-Degnan D, Oxman AD, Cheyne H, Foy R, Goodacre S, Herrin J, Kerse N, McKinlay RJ, et al. A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation. J Clin Epidemiol. 2015;68(3):324–33.
pubmed: 25499983
doi: 10.1016/j.jclinepi.2014.10.003
Morales DR, Minchin M, Kontopantelis E, Roland M, Sutton M, Guthrie B. Estimated impact from the withdrawal of primary care financial incentives on selected indicators of quality of care in Scotland: controlled interrupted time series analysis. BMJ. 2023;380:e072098.
pubmed: 36948515
pmcid: 10031759
doi: 10.1136/bmj-2022-072098
Campbell MK, Elbourne DR, Altman DG. for the CG: CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328:702–8.
pubmed: 15031246
pmcid: 381234
doi: 10.1136/bmj.328.7441.702
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
pubmed: 19664226
pmcid: 2736161
doi: 10.1186/1748-5908-4-50
Willis TA, Collinson M, Glidewell L, Farrin AJ, Holland M, Meads D, Hulme C, Petty D, Alderson S, Hartley S, et al. An adaptable implementation package targeting evidence-based indicators in primary care: a pragmatic cluster-randomised evaluation. PLOS Med. 2020;17(2):e1003045.
pubmed: 32109257
pmcid: 7048270
doi: 10.1371/journal.pmed.1003045
Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10:37.
pubmed: 19493350
pmcid: 2700087
doi: 10.1186/1745-6215-10-37
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.
pubmed: 25956159
doi: 10.1136/bmj.h2147
Elouafkaoui P, Young L, Newlands R, Duncan EM, Elders A, Clarkson JE. An audit and feedback intervention for reducing antibiotic prescribing in general dental practice: the RAPiD cluster randomised controlled trial. PLoS Med. 2016;13(8):e1002115.
Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalised causal inference. Boston: Houghton Mifflin Company; 2002.
Fleming PJ, Gilbert R, Azaz Y, Berry PJ, Rudd PT, Stewart A, Hall E. Interaction between bedding and sleeping position in the sudden infant death syndrome: a population based case-control study. BMJ. 1990;301(6743):85–9.
pubmed: 2390588
pmcid: 1663432
doi: 10.1136/bmj.301.6743.85
Cochrane AL. Effectiveness and efficiency: random reflections on health services. London: Nuffield Provincial Hospitals Trust; 1972.
Edwards P, Arango M, Balica L, Cottingham R, El-Sayed H, Farrell B, Fernandes J, Gogichaisvili T, Golden N, Hartzenberg B, et al. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months. Lancet. 2005;365(9475):1957–9.
pubmed: 15936423
doi: 10.1016/S0140-6736(05)66552-X
The AVERT Trial Collaboration group. Efficacy and safety of very early mobilisation within 24 h of stroke onset (AVERT): a randomised controlled trial. Lancet. 2015;386(9988):46–55.
Elshaug AG, Watt AM, Mundy L, Willis CD. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197(10):556–60.
pubmed: 23163685
doi: 10.5694/mja12.11083
Roland M, Guthrie B. Quality and Outcomes Framework: what have we learnt? BMJ. 2016;354:i4060.
pubmed: 27492602
pmcid: 4975019
doi: 10.1136/bmj.i4060
Wilson PM, Boaden R, Harvey G. Plans to accelerate innovation in health systems are less than IDEAL. BMJ Qual Saf. 2016;25(8):572–6.
pubmed: 26700544
doi: 10.1136/bmjqs-2015-004605
Westlake D, Tierney S, Wong G, Mahtani KR. Social prescribing in the NHS—is it too soon to judge its value? BMJ. 2023;380: p699.
doi: 10.1136/bmj.p699
Klein R. The new politics of the National Health Service. London: Longman Publishing Group; 1995.
Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, Macleod M, Wisely J, Chalmers I. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85.
pubmed: 24411646
doi: 10.1016/S0140-6736(13)62297-7
Middleton S, McElduff P, Ward J, Grimshaw JM, Dale S, D’Este C, Drury P, Griffiths R, Cheung NW, Quinn C, et al. Implementation of evidence-based treatment protocols to manage fever, hyperglycaemia, and swallowing dysfunction in acute stroke (QASC): a cluster randomised controlled trial. Lancet. 2011;378(9804):1699–706.
pubmed: 21996470
doi: 10.1016/S0140-6736(11)61485-2
Middleton S, Coughlan K, Mnatzaganian G, Low Choy N, Dale S, Jammali-Blasi A, Levi C, Grimshaw JM, Ward J, Cadilhac DA, et al. Mortality reduction for fever, hyperglycemia, and swallowing nurse-initiated stroke intervention: QASC Trial (Quality in Acute Stroke Care) follow-up. Stroke. 2017;48(5):1331–6.
pubmed: 28389609
doi: 10.1161/STROKEAHA.116.016038
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
pubmed: 18824488
pmcid: 2769032
doi: 10.1136/bmj.a1655
Morris S, Hunter RM, Ramsay AI, Boaden R, McKevitt C, Perry C, Pursani N, Rudd AG, Schwamm LH, Turner SJ, et al. Impact of centralising acute stroke services in English metropolitan areas on mortality and length of hospital stay: difference-in-differences analysis. BMJ. 2014;349:g4757.
pubmed: 25098169
pmcid: 4122734
doi: 10.1136/bmj.g4757
Shojania KG. Conventional evaluations of improvement interventions: more trials or just more tribulations? BMJ Qual Saf. 2013;22(11):881–4.
pubmed: 24077851
doi: 10.1136/bmjqs-2013-002377
Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
Gulliford MC, Prevost AT, Charlton J, Juszczyk D, Soames J, McDermott L, Sultana K, Wright M, Fox R, Hay AD, et al. Effectiveness and safety of electronically delivered prescribing feedback and decision support on antibiotic use for respiratory illness in primary care: REDUCE cluster randomised trial. BMJ. 2019;364:l236.
pubmed: 30755451
pmcid: 6371944
doi: 10.1136/bmj.l236
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350: h1258.
pubmed: 25791983
pmcid: 4366184
doi: 10.1136/bmj.h1258
Foy R, Walker A, Ramsay C, Penney G, Grimshaw J, Francis JJ. Theory-based identification of barriers to quality improvement: induced abortion care. Int J Qual Health Care. 2005;17:147–55.
pubmed: 15665065
doi: 10.1093/intqhc/mzi017
Kwan JL, Lo L, Ferguson J, Goldberg H, Diaz-Martinez JP, Tomlinson G, Grimshaw JM, Shojania KG. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ. 2020;370:m3216.
The Improved Clinical Effectiveness through Behavioural Research Group. Designing theoretically-informed implementation interventions. Implement Sci. 2006;1:4.
doi: 10.1186/1748-5908-1-4
Collins LM, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med. 2007;32(5 Suppl):S112-118.
pubmed: 17466815
pmcid: 2062525
doi: 10.1016/j.amepre.2007.01.022
Eccles MP, Francis J, Foy R, Johnston M, Bamford C, Grimshaw JM, Hughes J, Lecouturier J, Steen N, Whitty PM. Improving professional practice in the disclosure of a diagnosis of dementia: a modeling experiment to evaluate a theory-based intervention. Int J Behav Med. 2009;16(4):377–87.
pubmed: 19424811
doi: 10.1007/s12529-008-9023-3
Gude WT, van Engen-Verheul MM, van der Veer SN, de Keizer NF, Peek N. How does audit and feedback influence intentions of health professionals to improve practice? A laboratory experiment and field study in cardiac rehabilitation. BMJ Qual Saf. 2017;26(4):279–87.
pubmed: 27068999
doi: 10.1136/bmjqs-2015-004795
Kilbourne AM, Almirall D, Goodrich DE, Lai Z, Abraham KM, Nord KM, Bowersox NW. Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. Implement Sci. 2014;9:163.
pubmed: 25544027
pmcid: 4296543
doi: 10.1186/s13012-014-0163-3
Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing–a trial of education, informatics, and financial incentives. N Engl J Med. 2016;374(11):1053–64.
pubmed: 26981935
doi: 10.1056/NEJMsa1508955
Hargreaves JR, Copas AJ, Beard E, Osrin D, Lewis JJ, Davey C, Thompson JA, Baio G, Fielding KL, Prost A. Five questions to consider before conducting a stepped wedge trial. Trials. 2015;16:350.
pubmed: 26279013
pmcid: 4538743
doi: 10.1186/s13063-015-0841-8
Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.
pubmed: 17092344
pmcid: 1636652
doi: 10.1186/1471-2288-6-54
Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, Fielding KL. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358.
pubmed: 26278667
pmcid: 4538923
doi: 10.1186/s13063-015-0838-3
Snooks H, Bailey-Jones K, Burge-Jones D, Dale J, Davies J, Evans B, Farr A, Fitzsimmons D, Harrison J, Heaven M, et al. Fitzsimmons D, Harrison J, Heaven M et al: Health services and delivery research. In: Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC). Southampton: NIHR Journals Library; 2018.
Park JJH, Siden E, Zoratti MJ, Dron L, Harari O, Singer J, Lester RT, Thorlund K, Mills EJ. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials. 2019;20(1):572.
pubmed: 31533793
pmcid: 6751792
doi: 10.1186/s13063-019-3664-1
Edwards H, Redaniel MT, Opmeer B, Peters T, Margelyte R, Sillero Rejon C, Hollingworth W, Craggs P, Hill E, Redwood S, et al. Evaluating an enhanced quality improvement intervention in maternity units: PReCePT trial protocol. BMJ Open Quality. 2021;10(2):e001204.
pubmed: 34031151
pmcid: 8149440
doi: 10.1136/bmjoq-2020-001204
Halpern D, Mason D. Radical incrementalism. Evaluation. 2015;21:143–9.
doi: 10.1177/1356389015578895
Haynes L, Service O, Goldacre B, Torgerson D. Test, learn, adapt: developing public policy with randomised controlled trials. London: Cabinet Office Behavioural Insights Team; 2012.
Ivers NM, Grimshaw JM. Reducing research waste with implementation laboratories. Lancet. 2016;388(10044):547–8.
pubmed: 27511773
doi: 10.1016/S0140-6736(16)31256-9
Grimshaw JM, Ivers N, Linklater S, Foy R, Francis JJ, Gude WT. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual Saf. 2019;28(5):416–23.
Stanworth SJ, Walwyn R, Grant-Casey J, Hartley S, Moreau L, Lorencatto F, Francis J, Gould N, Swart N, Rowley M, et al. Effectiveness of enhanced performance feedback on appropriate use of blood transfusions: a comparison of 2 cluster randomized trials. JAMA Network Open. 2022;5(2):e220364–e220364.
pubmed: 35201305
pmcid: 8874348
doi: 10.1001/jamanetworkopen.2022.0364
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.
pubmed: 19525005
doi: 10.1016/S0140-6736(09)60329-9
Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O’Brien MA, French SD, Young J, Odgaard-Jensen J. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534–41.
pubmed: 24965281
pmcid: 4238192
doi: 10.1007/s11606-014-2913-y
Karanatsios B, Prang K-H, Verbunt E, Yeung JM, Kelaher M, Gibbs P. Defining key design elements of registry-based randomised controlled trials: a scoping review. Trials. 2020;21(1):552.
pubmed: 32571382
pmcid: 7310018
doi: 10.1186/s13063-020-04459-z
Rushforth B, Stokes T, Andrews E, Willis TA, McEachan R, Faulkner S, Foy R. Developing ‘high impact’ guideline-based quality indicators for UK primary care: a multi-stage consensus process. BMC Fam Pract. 2015;16(1):156.
pubmed: 26507739
pmcid: 4624600
doi: 10.1186/s12875-015-0350-6
Foy R, Eccles MP, Hrisos S, Hawthorne G, Steen N, Gibb I, Croal B, Grimshaw J. A cluster randomised trial of educational messages to improve the primary care of diabetes. Implement Sci. 2011;6:129.
pubmed: 22177466
pmcid: 3284425
doi: 10.1186/1748-5908-6-129
McLintock K, Russell AM, Alderson SL, West R, House A, Westerman K, Foy R. The effects of financial incentives for case finding for depression in patients with diabetes and coronary heart disease: interrupted time series analysis. BMJ Open. 2014;4(8):e005178.
pubmed: 25142262
pmcid: 4139660
doi: 10.1136/bmjopen-2014-005178
Ivers N: Supporting audit and feedback to encourage vaccine uptake (NCT05099497). ClinicalTrialsgov 2021.
Yu LM, Bafadhel M, Dorward J, Hayward G, Saville BR, Gbinigie O, Van Hecke O, Ogburn E, Evans PH, Thomas NPB, et al. Inhaled budesonide for COVID-19 in people at high risk of complications in the community in the UK (PRINCIPLE): a randomised, controlled, open-label, adaptive platform trial. Lancet. 2021;398(10303):843–55.
pubmed: 34388395
pmcid: 8354567
doi: 10.1016/S0140-6736(21)01744-X
Foy R, Skrypak M, Alderson S, Ivers NM, McInerney B, Stoddart J, Ingham J, Keenan D. Revitalising audit and feedback to improve patient care. BMJ. 2020;368:m213.
pubmed: 32107249
pmcid: 7190377
doi: 10.1136/bmj.m213
Ruppertsberg AI, Ward V, Ridout A, Foy R. The development and application of audit criteria for assessing knowledge exchange plans in health research grant applications. Implement Sci. 2014;9:93.
pubmed: 25017548
pmcid: 4227283
doi: 10.1186/s13012-014-0093-0
Wilkinson E. RECOVERY trial: the UK covid-19 study resetting expectations for clinical trials. BMJ. 2020;369:m1626.
pubmed: 32345591
doi: 10.1136/bmj.m1626
James M, Ford GA, Robinson T, Al-Shahi Salman R. Rapid mobilisation of research in response to covid-19: a paradigm for the future. BMJ. 2020;369:m2155.
pubmed: 32471855
doi: 10.1136/bmj.m2155
Wieseler B, Neyt M, Kaiser T, Hulstaert F, Windeler J. Replacing RCTs with real world data for regulatory decision making: a self-fulfilling prophecy? BMJ. 2023;380:e073100.
pubmed: 36863730
doi: 10.1136/bmj-2022-073100