What validated instruments, that measure implementation outcomes, are suitable for use in the Paediatric Intensive Care Unit (PICU) setting? A systematic review of systematic reviews.
Evidence-based practice
Healthcare
Instrument validation
Outcomes
Journal
Implementation science : IS
ISSN: 1748-5908
Titre abrégé: Implement Sci
Pays: England
ID NLM: 101258411
Informations de publication
Date de publication:
10 Oct 2024
10 Oct 2024
Historique:
received:
06
02
2024
accepted:
30
06
2024
medline:
11
10
2024
pubmed:
11
10
2024
entrez:
10
10
2024
Statut:
epublish
Résumé
The measurement of implementation outcomes can establish the success of implementing evidence into practice. However, implementation outcomes are seldom measured in acute healthcare settings, such as Paediatric Intensive Care Units (PICU), and if they are used, are likely to be non-validated, site or intervention-specific measures. To address this literature gap, this systematic review of systematic reviews aims to identify validated instruments to measure implementation outcomes of new EBP interventions in a PICU setting. A systematic review of systematic reviews was conducted in two phases. Phase One: Five electronic databases were searched between 06/10/22 and 14/10/22. Systematic reviews were selected using pre-determined eligibility criteria. Methodological quality was assessed using the Critical Appraisal Skills Programme tool and a data extraction table was used to allow further synthesis. Phase Two: Secondary eligibility criteria were used to extract and review instruments from the systematic reviews selected in Phase One. Instruments were analysed and mapped to the Consolidated Framework of Implementation Research (CFIR). Phase One: Searches resulted in 3195 unique papers. Five systematic reviews were eligible for inclusion. All examined the psychometric properties of each instrument, utilising different methods to do so; three considered their pragmatic or usability properties; and one identified instruments that were transferrable to different settings. Each systematic review identified that most included instruments had limited evidence of their validity or reliability and had poor psychometric properties. Phase two: 93 instruments were screened, and nine were eligible for analysis. After analysis and CFIR mapping, two instruments were identified as potentially adaptable to the PICU setting. The methodological quality of implementation outcome measurement instruments is inadequate, warranting further validation research. Two instruments were identified that cover multiple CFIR domains and have scope to be adapted for use when implementing evidence-based practice into the PICU. Further work is needed to adapt and further validate an instrument for use in practice. For transparency of procedures and methods, the protocol for this systematic review was registered with PROSPERO (registration number CRD42022361638L).
Sections du résumé
BACKGROUND/AIMS
OBJECTIVE
The measurement of implementation outcomes can establish the success of implementing evidence into practice. However, implementation outcomes are seldom measured in acute healthcare settings, such as Paediatric Intensive Care Units (PICU), and if they are used, are likely to be non-validated, site or intervention-specific measures. To address this literature gap, this systematic review of systematic reviews aims to identify validated instruments to measure implementation outcomes of new EBP interventions in a PICU setting.
METHODS
METHODS
A systematic review of systematic reviews was conducted in two phases. Phase One: Five electronic databases were searched between 06/10/22 and 14/10/22. Systematic reviews were selected using pre-determined eligibility criteria. Methodological quality was assessed using the Critical Appraisal Skills Programme tool and a data extraction table was used to allow further synthesis. Phase Two: Secondary eligibility criteria were used to extract and review instruments from the systematic reviews selected in Phase One. Instruments were analysed and mapped to the Consolidated Framework of Implementation Research (CFIR).
RESULTS
RESULTS
Phase One: Searches resulted in 3195 unique papers. Five systematic reviews were eligible for inclusion. All examined the psychometric properties of each instrument, utilising different methods to do so; three considered their pragmatic or usability properties; and one identified instruments that were transferrable to different settings. Each systematic review identified that most included instruments had limited evidence of their validity or reliability and had poor psychometric properties. Phase two: 93 instruments were screened, and nine were eligible for analysis. After analysis and CFIR mapping, two instruments were identified as potentially adaptable to the PICU setting.
CONCLUSIONS
CONCLUSIONS
The methodological quality of implementation outcome measurement instruments is inadequate, warranting further validation research. Two instruments were identified that cover multiple CFIR domains and have scope to be adapted for use when implementing evidence-based practice into the PICU. Further work is needed to adapt and further validate an instrument for use in practice.
TRIAL REGISTRATION
BACKGROUND
For transparency of procedures and methods, the protocol for this systematic review was registered with PROSPERO (registration number CRD42022361638L).
Identifiants
pubmed: 39390442
doi: 10.1186/s13012-024-01378-4
pii: 10.1186/s13012-024-01378-4
doi:
Types de publication
Journal Article
Systematic Review
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
70Informations de copyright
© 2024. The Author(s).
Références
Willmeroth T, Wesselborg B, Kuske S. Implementation outcomes and indicators as a new challenge in health services research: a systematic scoping review. Inquiry. 2019;56:004695801986125 The Journal of Health Care Organization, Provision, and Financing.
Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, et al. Ten years of implementation outcomes research: a scoping review. Implement Sci. 2023;18(1):31.
doi: 10.1186/s13012-023-01286-z
pubmed: 37491242
pmcid: 10367273
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research Agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65–76.
doi: 10.1007/s10488-010-0319-7
Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. 2019;17(1):88.
doi: 10.1186/s12916-019-1322-9
pubmed: 31064388
pmcid: 6505277
The C-ICUI, the B. A cluster randomized trial of a multifaceted quality improvement intervention in Brazilian intensive care units: study protocol. Implement Sci. 2015;10(1):8.
doi: 10.1186/s13012-014-0190-0
Steffen KM, Holdsworth LM, Ford MA, Lee GM, Asch SM, Proctor EK. Implementation of clinical practice changes in the PICU: a qualitative study using and refining the iPARIHS framework. Implement Sci. 2021;16(1):15.
doi: 10.1186/s13012-021-01080-9
pubmed: 33509190
pmcid: 7841901
Zimmerman JJ, Anand KJS, Meert KL, Willson DF, Newth CJL, Harrison R, et al. Research as a Standard of Care in the PICU*. Pediatr Crit Care Med. 2016;17(1):e13–21.
doi: 10.1097/PCC.0000000000000562
pubmed: 26513203
pmcid: 5003779
Burgess MG, Brough P, Biggs A, Hawkes AJ. Why interventions fail: a systematic review of occupational health psychology interventions. Int J Stress Manag. 2020;27(2):195–207.
doi: 10.1037/str0000144
Garner S, Docherty M, Somner J, Sharma T, Choudhury M, Clarke M, et al. Reducing ineffective practice: challenges in identifying low-value health care using Cochrane systematic reviews. J Health Serv Res Policy. 2013;18(1):6–12.
doi: 10.1258/jhsrp.2012.012044
pubmed: 23393036
Dodds E, Kudchadkar SR, Choong K, Manning JC. A realist review of the effective implementation of the ICU Liberation Bundle in the paediatric intensive care unit setting. Aust Crit Care. 2022;36(5):837–46. https://pubmed.ncbi.nlm.nih.gov/36581506/ .
Patel R, Eakin M, Wieczorek B, Needham D, Kudchadkar S. Sustainability of a PICU early mobilization program: a qualitative analysis. Crit Care Med. 2019;47(1):672.
doi: 10.1097/01.ccm.0000552135.59636.3e
Khadjesari Z, Boufkhed S, Vitoratou S, Schatte L, Ziemann A, Daskalopoulou C, et al. Implementation outcome instruments for use in physical healthcare settings: a systematic review. Implement Sci. 2020;15(1):66.
doi: 10.1186/s13012-020-01027-6
pubmed: 32811517
pmcid: 7433178
Mettert K, Lewis C, Dorsey C, Halko H, Weiner B. Measuring implementation outcomes: an updated systematic review of measures’ psychometric properties. Implement Res Pract. 2020;1:263348952093664.
Pollack M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overview of Reviews. 2022. In: Cochrane Handbook for Systematic Reviews of Interventions. www.training.cochrane.org/handbook .: Cochrane. Version 6.3. . Available from: https://training.cochrane.org/handbook/current/chapter-v . Cited 7/2/2023.
Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15.
doi: 10.1186/1471-2288-11-15
pubmed: 21291558
pmcid: 3039637
Caird J, Sutcliffe K, Kwan I, Dickson K, Thomas J. Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evid Policy. 2015;11:81–97.
doi: 10.1332/174426514X13988609036850
Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667.
doi: 10.1371/journal.pone.0049667
pubmed: 23166744
pmcid: 3499476
Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;93:e070849.
doi: 10.1136/bmj-2022-070849
Nadelson S, Nadelson LS. Evidence-based practice article reviews using CASP tools: a method for teaching EBP. Worldviews Evid Based Nurs. 2014;11(5):344–6.
doi: 10.1111/wvn.12059
pubmed: 25156640
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
doi: 10.1186/1748-5908-4-50
pubmed: 19664226
pmcid: 2736161
Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020;15(1):47.
doi: 10.1186/s13012-020-01007-w
pubmed: 32560661
pmcid: 7304175
McLoughlin GM, Allen P, Walsh-Bailey C, Brownson RC. A systematic review of school health policy measurement tools: implementation determinants and outcomes. Implement. 2021;2(1):67.
doi: 10.1186/s43058-021-00169-y
Hand BN, Darragh AR, Persch AC. Thoroughness and psychometrics of fidelity measures in occupational and physical therapy: a systematic review. Am J Occup Ther. 2018;72(5):7205205050p1-p10.
doi: 10.5014/ajot.2018.025510
pubmed: 30157017
pmcid: 6114192
Clinton-Mcharg T, Yoong SL, Tzelepis F, Regan T, Fielding A, Skelton E, et al. Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review. Implement Sci. 2016;11(1):148.
doi: 10.1186/s13012-016-0512-5
pubmed: 27821146
pmcid: 5100177
Bullock HL, Lavis JN, Wilson MG, Mulvale G, Miatello A. Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis. Implement Sci. 2021;16(1):18.
doi: 10.1186/s13012-021-01082-7
pubmed: 33588878
pmcid: 7885555
Damschroder LJ, Reardon CM, OpraWiderquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7.
doi: 10.1186/s13012-021-01181-5
pubmed: 35065675
pmcid: 8783408
Khadjesari Z, Vitoratou S, Sevdalis N, Hull L. Implementation outcome assessment instruments used in physical healthcare settings and their measurement properties: a systematic review protocol. BMJ Open. 2017;7(10):e017972.
doi: 10.1136/bmjopen-2017-017972
pubmed: 28993392
pmcid: 5640043
Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E, et al. An updated protocol for a systematic review of implementation-related measures. Syst Rev. 2018;7(1):66.
doi: 10.1186/s13643-018-0728-3
pubmed: 29695295
pmcid: 5918558
Lewis CC, Mettert KD, Stanick CF, Halko HM, Nolen EA, Powell BJ, et al. The psychometric and pragmatic evidence rating scale (PAPERS) for measure development and evaluation. Implement Res Pract. 2021;2:263348952110373.
Mokkink LB, Prinsen CAC, Bouter LM, Vet HCWD, Terwee CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther. 2016;20(2):105–13.
doi: 10.1590/bjpt-rbf.2014.0143
pubmed: 26786084
pmcid: 4900032
American Educational Research Association. American psychological association, national council on measurement in education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.
doi: 10.1186/s13012-017-0635-3
pubmed: 28851459
pmcid: 5576104
Wolf DAPS, Dulmus CN, Maguin E, Fava N. Refining the evidence-based practice attitude scale: an alternative confirmatory factor analysis. Social Work Res. 2014;38(1):47–58.
doi: 10.1093/swr/svu006
Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.
doi: 10.1023/B:MHSR.0000024351.12294.65
pubmed: 15224451
pmcid: 1564126
Allen JD, Towne SD Jr, Maxwell AE, DiMartino L, Leyva B, Bowen DJ, et al. Meausures of organizational characteristics associated with adoption and/or implementation of innovations: a systematic review. BMC Health Serv Res. 2017;17(1):591.
doi: 10.1186/s12913-017-2459-x
pubmed: 28835273
pmcid: 5569532
Dow B, Fearn M, Haralambous B, Tinney J, Hill K, Gibson S. Development and initial testing of the person-Centred health care for older adults survey. Int Psychogeriatr. 2013;25(7):1065–76.
doi: 10.1017/S1041610213000471
pubmed: 23628210
Dykes PC, Hurley A, Cashen M, Bakken S, Duffy ME. Development and Psychometric Evaluation of the Impact of Health Information Technology (I-HIT) Scale. J Am Med Inform Assoc. 2007;14(4):507–14.
doi: 10.1197/jamia.M2367
pubmed: 17460123
pmcid: 2244896
Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53(4):454–8.
doi: 10.1111/j.1365-2648.2006.03739.x
pubmed: 16448488
Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135.
doi: 10.1186/s12874-018-0591-x
pubmed: 30442094
pmcid: 6238372
Cook J, Thompson R, Schnurr P. Perceived Characteristics of Intervention Scale: Development and Psychometric Properties. Assessment. 2015;22(6):704–14.
doi: 10.1177/1073191114561254
pubmed: 25520213
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.
doi: 10.1111/j.0887-378X.2004.00325.x
pubmed: 15595944
pmcid: 2690184
Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.
doi: 10.1136/bmj.n2061
pubmed: 34593508
pmcid: 8482308
Dixon-Woods M, Leslie M, Tarrant C, Bion J. Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci. 2013;8(1):70.
doi: 10.1186/1748-5908-8-70
pubmed: 23786847
pmcid: 3704826