Quantitative measures of health policy implementation determinants and outcomes: a systematic review.
Health policy
Implementation
Implementation science
Measures
Policy implementation
Pragmatic
Psychometric
Public policy
Systematic review
Journal
Implementation science : IS
ISSN: 1748-5908
Titre abrégé: Implement Sci
Pays: England
ID NLM: 101258411
Informations de publication
Date de publication:
19 06 2020
19 06 2020
Historique:
received:
24
03
2020
accepted:
05
06
2020
entrez:
21
6
2020
pubmed:
21
6
2020
medline:
25
5
2021
Statut:
epublish
Résumé
Public policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures. Three frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures. Database searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported. Well-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health. Not registered.
Sections du résumé
BACKGROUND
Public policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures.
METHODS
Three frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures.
RESULTS
Database searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported.
CONCLUSIONS
Well-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health.
REGISTRATION
Not registered.
Identifiants
pubmed: 32560661
doi: 10.1186/s13012-020-01007-w
pii: 10.1186/s13012-020-01007-w
pmc: PMC7304175
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, P.H.S.
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
47Subventions
Organisme : NIMH NIH HHS
ID : R01MH106510
Pays : United States
Organisme : NCI NIH HHS
ID : P50 CA244431
Pays : United States
Organisme : NCI NIH HHS
ID : P50CA244431
Pays : United States
Organisme : NIDDK NIH HHS
ID : P30 DK020579
Pays : United States
Organisme : CDC HHS
ID : U48DP006395-01-00
Pays : United States
Organisme : NIDDK NIH HHS
ID : P30DK020579
Pays : United States
Organisme : NCATS NIH HHS
ID : UL1 TR002345
Pays : United States
Références
Annu Rev Public Health. 2018 Apr 1;39:27-53
pubmed: 29166243
BMC Public Health. 2018 Jul 13;18(1):869
pubmed: 30005611
Int J Equity Health. 2018 Oct 5;17(1):92
pubmed: 30286742
Health Aff (Millwood). 2010 Mar-Apr;29(3):447-53
pubmed: 20194986
J Sch Health. 2013 Aug;83(8):525-32
pubmed: 23834603
Behav Ther. 2018 Jul;49(4):525-537
pubmed: 29937255
J Appl Psychol. 1997 Oct;82(5):812-20
pubmed: 9337610
Acad Med. 2015 Oct;90(10):1302-8
pubmed: 25993282
BMC Health Serv Res. 2015 Feb 28;15:79
pubmed: 25889958
Implement Sci. 2014 Aug 20;9:108
pubmed: 25138688
BMC Public Health. 2016 May 17;16:412
pubmed: 27185039
Implement Sci. 2015 Nov 04;10:155
pubmed: 26537706
Assessment. 2015 Dec;22(6):704-14
pubmed: 25520213
Health Educ Res. 2002 Jun;17(3):315-26
pubmed: 12120847
Implement Sci. 2013 Jun 10;8:63
pubmed: 23758952
Am J Prev Med. 2019 Dec;57(6 Suppl 1):S13-S24
pubmed: 31753276
Health Promot Pract. 2016 Mar;17(2):244-53
pubmed: 26500227
Telemed J E Health. 2017 Jul;23(7):577-589
pubmed: 28177858
Health Promot Int. 2016 Jun;31(2):290-302
pubmed: 25500994
Syst Rev. 2018 Apr 25;7(1):66
pubmed: 29695295
Am J Prev Med. 2012 Sep;43(3):337-50
pubmed: 22898128
J Public Health Manag Pract. 2016 Jan-Feb;22(1):9-19
pubmed: 25822902
Health Policy. 2015 Jun;119(6):749-59
pubmed: 25620775
Qual Life Res. 2009 Oct;18(8):1115-23
pubmed: 19711195
J Appl Psychol. 2000 Feb;85(1):132-42
pubmed: 10740964
Child Abuse Negl. 2012 Sep;36(9):621-32
pubmed: 22980071
Soz Praventivmed. 2003;48(6):379-91
pubmed: 14758749
Transl Behav Med. 2019 Nov 20;:
pubmed: 31747021
J Am Med Inform Assoc. 2010 May-Jun;17(3):237-44
pubmed: 20442140
Health Promot Int. 2006 Dec;21(4):266-73
pubmed: 16855012
Bull World Health Organ. 2008 Feb;86(2):140-6
pubmed: 18297169
Public Health Nutr. 2015 Jan;18(1):81-8
pubmed: 24477275
Implement Sci. 2016 Mar 22;11:42
pubmed: 27000147
Implement Sci. 2016 Jan 04;11:1
pubmed: 26727969
BMJ Open. 2017 Dec 21;7(12):e017772
pubmed: 29273655
Implement Sci. 2013 Jun 04;8:59
pubmed: 23734672
Glob Health Action. 2012;5:17271
pubmed: 22666185
Environ Health. 2012 Jan 27;11:4
pubmed: 22280473
J Public Health Manag Pract. 2008 Mar-Apr;14(2):117-23
pubmed: 18287916
Zdr Varst. 2017 Feb 26;56(2):115-123
pubmed: 28289471
Implement Sci. 2009 Aug 07;4:50
pubmed: 19664226
Health Policy. 2016 Dec;120(12):1363-1377
pubmed: 27720165
Front Public Health. 2018 May 07;6:136
pubmed: 29868544
J Am Diet Assoc. 2008 Sep;108(9):1497-502
pubmed: 18755322
BMJ Open. 2012 Jun 25;2(3):
pubmed: 22734117
Psychol Assess. 2010 Jun;22(2):356-65
pubmed: 20528063
BMJ Glob Health. 2017 Dec 1;3(Suppl 1):e000603
pubmed: 29379648
Tob Control. 2016 May;25(3):313-8
pubmed: 25908597
ERJ Open Res. 2017 May 12;3(2):
pubmed: 28512634
J Nerv Ment Dis. 2017 Feb;205(2):161-170
pubmed: 27668352
Arch Public Health. 2018 Feb 15;76:12
pubmed: 29456843
Am J Prev Med. 2013 Aug;45(2):237-43
pubmed: 23867032
Br J Sports Med. 2015 Jun;49(11):749-56
pubmed: 24682248
Implement Sci. 2013 Jun 10;8:65
pubmed: 23758995
Scand J Caring Sci. 2002 Mar;16(1):3-11
pubmed: 11985743
Eur J Public Health. 2018 Feb 1;28(1):43-49
pubmed: 29016786
Implement Sci. 2013 Mar 24;8:38
pubmed: 23522556
J Subst Abuse Treat. 2002 Jun;22(4):197-209
pubmed: 12072164
Tob Control. 2020 Mar;29(2):231-233
pubmed: 31053650
Am J Prev Med. 2007 Jul;33(1 Suppl):S50-8; quiz S59-65
pubmed: 17584591
Adm Policy Ment Health. 2011 Mar;38(2):65-76
pubmed: 20957426
Implement Sci. 2008 May 16;3:26
pubmed: 18485219
BMC Health Serv Res. 2018 Mar 27;18(1):209
pubmed: 29580251
Hum Resour Health. 2012 Oct 30;10:40
pubmed: 23110690
J Phys Act Health. 2014 Feb;11(2):233-40
pubmed: 23364305
Implement Sci. 2017 Oct 3;12(1):118
pubmed: 28974248
Lancet. 2008 Apr 12;371(9620):1284-93
pubmed: 18406863
Health Promot Pract. 2013 Jul;14(4):514-23
pubmed: 23132841
Health Promot Int. 2016 Dec;31(4):888-898
pubmed: 26276800
Health Educ Behav. 2013 Jun;40(3):257-65
pubmed: 23709579
Implement Sci. 2016 Mar 09;11:31
pubmed: 26956621
Implement Sci. 2013 Feb 17;8:22
pubmed: 23414420
Qual Saf Health Care. 2009 Feb;18 Suppl 1:i22-7
pubmed: 19188457
J Subst Abuse Treat. 2013 Mar;44(3):280-7
pubmed: 22959978
Health Promot Pract. 2009 Apr;10(2):176-85
pubmed: 19372279
Implement Sci. 2010 Feb 12;5:16
pubmed: 21080976