Developing guideline-based key performance indicators for recurrent miscarriage care: lessons from a multi-stage consensus process with a diverse stakeholder group.
Care quality
Delphi technique
Early pregnancy loss
Patient and public involvement
Quality improvement
Quality indicators
Journal
Research involvement and engagement
ISSN: 2056-7529
Titre abrégé: Res Involv Engagem
Pays: England
ID NLM: 101708164
Informations de publication
Date de publication:
14 May 2022
14 May 2022
Historique:
received:
11
01
2022
accepted:
29
04
2022
entrez:
15
5
2022
pubmed:
16
5
2022
medline:
16
5
2022
Statut:
epublish
Résumé
Standardised care pathways tailored to women/couples who experience recurrent miscarriage are needed; however, clinical practice is inconsistent and poorly organised. In this paper, we outline our processes and experiences of developing guideline-based key performance indicators (KPIs) for recurrent miscarriage care with a diverse stakeholder group which will be used to evaluate national services. To date, such exercises have generally only involved clinicians, with the need for greater stakeholder involvement highlighted. Our study involved six stages: (i) identification and synthesis of recommendations for recurrent miscarriage care through a systematic review of clinical practice guidelines; (ii) a two-round modified e-Delphi survey with stakeholders to develop consensus on recommendations and outcomes; (iii) four virtual meetings to develop this consensus further; (iv) development of a list of candidate KPIs; (v) survey to achieve consensus on the final suite of KPIs and a (vi) virtual meeting to agree on the final set of KPIs. Through participatory methods, participants provided feedback on the process of KPI development. From an initial list of 373 recommendations and 14 outcomes, 110 indicators were prioritised for inclusion in the final suite of KPIs: (i) structure of care (n = 20); (ii) counselling and supportive care (n = 7); (iii) investigations (n = 30); treatment (n = 34); outcomes (n = 19). Participants' feedback on the process comprised three main themes: accessibility, richness in diversity, streamlining the development process. It is important and feasible to develop guideline-based KPIs with a diverse stakeholder group. One hundred and ten KPIs were prioritised for inclusion in a suite of guideline-based KPIs for recurrent miscarriage care. Insights into our experiences may help others undertaking similar projects, particularly those undertaken in the absence of a clinical guideline and/or involving a range of stakeholders. Women/couples who have recurrent/repeated miscarriages should receive care that meets their needs, through agreed care pathways. This is often not the case. Key performance indicators (KPIs) are measures of specific elements of care (structures, processes and/or outcomes), which can help us to judge the quality of care given. In this paper, we describe how we worked with women and men with lived experience, doctors, nurses, managers, and others, to develop and agree on a list KPIs for recurrent miscarriage care in Ireland. We will use these to check what services are doing across the country and what could be done better. Participants filled out surveys and took part in meetings to vote on and agree on what KPIs were important to include. They also shared their views and experiences of taking part in this work. Together, we developed 110 KPIs for recurrent miscarriage care. These include measures of how care is structured, counselling and supports, investigations and treatments provided, and health-related outcomes. Participants’ valued the different views that people brought to discussions and what they learned. They suggested ways that the process could be made more participant-friendly. For example, being up-front about the time it would take, explaining medical terms more, and cutting down on the number of items to be rated in surveys. It is important and possible to develop KPIs with different groups, particularly those with lived experience. Learning from our study may help others who want to do similar projects, such as develop KPIs or guidelines for care.
Sections du résumé
BACKGROUND
BACKGROUND
Standardised care pathways tailored to women/couples who experience recurrent miscarriage are needed; however, clinical practice is inconsistent and poorly organised. In this paper, we outline our processes and experiences of developing guideline-based key performance indicators (KPIs) for recurrent miscarriage care with a diverse stakeholder group which will be used to evaluate national services. To date, such exercises have generally only involved clinicians, with the need for greater stakeholder involvement highlighted.
METHODS
METHODS
Our study involved six stages: (i) identification and synthesis of recommendations for recurrent miscarriage care through a systematic review of clinical practice guidelines; (ii) a two-round modified e-Delphi survey with stakeholders to develop consensus on recommendations and outcomes; (iii) four virtual meetings to develop this consensus further; (iv) development of a list of candidate KPIs; (v) survey to achieve consensus on the final suite of KPIs and a (vi) virtual meeting to agree on the final set of KPIs. Through participatory methods, participants provided feedback on the process of KPI development.
RESULTS
RESULTS
From an initial list of 373 recommendations and 14 outcomes, 110 indicators were prioritised for inclusion in the final suite of KPIs: (i) structure of care (n = 20); (ii) counselling and supportive care (n = 7); (iii) investigations (n = 30); treatment (n = 34); outcomes (n = 19). Participants' feedback on the process comprised three main themes: accessibility, richness in diversity, streamlining the development process.
CONCLUSIONS
CONCLUSIONS
It is important and feasible to develop guideline-based KPIs with a diverse stakeholder group. One hundred and ten KPIs were prioritised for inclusion in a suite of guideline-based KPIs for recurrent miscarriage care. Insights into our experiences may help others undertaking similar projects, particularly those undertaken in the absence of a clinical guideline and/or involving a range of stakeholders.
Women/couples who have recurrent/repeated miscarriages should receive care that meets their needs, through agreed care pathways. This is often not the case. Key performance indicators (KPIs) are measures of specific elements of care (structures, processes and/or outcomes), which can help us to judge the quality of care given. In this paper, we describe how we worked with women and men with lived experience, doctors, nurses, managers, and others, to develop and agree on a list KPIs for recurrent miscarriage care in Ireland. We will use these to check what services are doing across the country and what could be done better. Participants filled out surveys and took part in meetings to vote on and agree on what KPIs were important to include. They also shared their views and experiences of taking part in this work. Together, we developed 110 KPIs for recurrent miscarriage care. These include measures of how care is structured, counselling and supports, investigations and treatments provided, and health-related outcomes. Participants’ valued the different views that people brought to discussions and what they learned. They suggested ways that the process could be made more participant-friendly. For example, being up-front about the time it would take, explaining medical terms more, and cutting down on the number of items to be rated in surveys. It is important and possible to develop KPIs with different groups, particularly those with lived experience. Learning from our study may help others who want to do similar projects, such as develop KPIs or guidelines for care.
Autres résumés
Type: plain-language-summary
(eng)
Women/couples who have recurrent/repeated miscarriages should receive care that meets their needs, through agreed care pathways. This is often not the case. Key performance indicators (KPIs) are measures of specific elements of care (structures, processes and/or outcomes), which can help us to judge the quality of care given. In this paper, we describe how we worked with women and men with lived experience, doctors, nurses, managers, and others, to develop and agree on a list KPIs for recurrent miscarriage care in Ireland. We will use these to check what services are doing across the country and what could be done better. Participants filled out surveys and took part in meetings to vote on and agree on what KPIs were important to include. They also shared their views and experiences of taking part in this work. Together, we developed 110 KPIs for recurrent miscarriage care. These include measures of how care is structured, counselling and supports, investigations and treatments provided, and health-related outcomes. Participants’ valued the different views that people brought to discussions and what they learned. They suggested ways that the process could be made more participant-friendly. For example, being up-front about the time it would take, explaining medical terms more, and cutting down on the number of items to be rated in surveys. It is important and possible to develop KPIs with different groups, particularly those with lived experience. Learning from our study may help others who want to do similar projects, such as develop KPIs or guidelines for care.
Identifiants
pubmed: 35568920
doi: 10.1186/s40900-022-00355-9
pii: 10.1186/s40900-022-00355-9
pmc: PMC9107009
doi:
Types de publication
Journal Article
Langues
eng
Pagination
18Subventions
Organisme : Health Research Board
ID : ILP-HSR-2019-011
Pays : Ireland
Commentaires et corrections
Type : ErratumIn
Informations de copyright
© 2022. The Author(s).
Références
Quenby S, Gallos ID, Dhillon-Smith RK, Podesek M, Stephenson MD, Fisher J, et al. Miscarriage matters: the epidemiological, physical, psychological, and economic costs of early pregnancy loss. Lancet. 2021;397(10285):1658–67.
pubmed: 33915094
Coomarasamy A, Dhillon-Smith RK, Papadopoulou A, Al-Memar M, Brewin J, Abrahams VM, et al. Recurrent miscarriage: evidence to accelerate action. Lancet. 2021;397(10285):1675–82.
pubmed: 33915096
Hennessy M, Dennehy R, Meaney S, Linehan L, Devane D, Rice R, et al. Clinical practice guidelines for recurrent miscarriage in high-income countries: a systematic review. Reprod Biomed Online. 2021;42(6):1146–71.
pubmed: 33895080
Kirshenbaum M, Orvieto R. Should we offer in vitro fertilization to couples with unexplained recurrent pregnancy loss? J Clin Med. 2019;8(11):2001.
pmcid: 6912379
Manning R, Iyer J, Bulmer JN, Maheshwari A, Choudhary M. Are we managing women with recurrent miscarriage appropriately? A snapshot survey of clinical practice within the United Kingdom. J Obstet Gynaecol. 2020;41:1–8.
van den Boogaard E, Hermens RPMG, Franssen AMHW, Doornbos JPR, Kremer JAM, van der Veen F, et al. Recurrent miscarriage: do professionals adhere to their guidelines. Hum Reprod. 2013;28(11):2898–904.
pubmed: 23970335
ESHRE Early Pregnancy Guideline Development Group. Guideline on the management of recurrent pregnancy loss. Version 2. Grimbergen: European Society of Human Reproduction and Embryology; 2017.
Health Information and Quality Authority (HIQA). Guidance on developing key performance indicators and minimum data sets to monitor healthcare quality. Dublin: HIQA; 2013.
Mak A, Cheung MW, Cheak AA, Ho RC. Combination of heparin and aspirin is superior to aspirin alone in enhancing live births in patients with recurrent pregnancy loss and positive anti-phospholipid antibodies: a meta-analysis of randomized controlled trials and meta-regression. Rheumatology (Oxford). 2010;49(2):281–8.
Donabedian A. Evaluating the quality of medical care. Milbank Q. 2005;83(4):691–729.
pubmed: 16279964
pmcid: 2690293
Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ (Clin Res Ed). 2003;326(7393):816–9.
Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11(4):358–64.
pubmed: 12468698
pmcid: 1758017
Kötter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci. 2012;7(1):21.
pubmed: 22436067
pmcid: 3368783
Nothacker M, Stokes T, Shaw B, Lindsay P, Sipilä R, Follmann M, et al. Reporting standards for guideline-based performance measures. Implement Sci. 2016;11(1):6.
pubmed: 26772173
pmcid: 4714427
Hommel I, van Gurp PJ, Tack CJ, Wollersheim H, Hulscher MEJL. Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway. BMJ Qual Saf. 2016;25(7):525.
pubmed: 26384710
Davies P, Wye L, Horrocks S, Salisbury C, Sharp D. Developing quality indicators for community services: the case of district nursing. Qual Prim Care. 2011;19(3):155–66.
pubmed: 21781431
Rushforth B, Stokes T, Andrews E, Willis TA, McEachan R, Faulkner S, et al. Developing ‘high impact’ guideline-based quality indicators for UK primary care: a multi-stage consensus process. BMC Fam Pract. 2015;16(1):156.
pubmed: 26507739
pmcid: 4624600
van den Boogaard E, Goddijn M, Leschot NJ, FVD V, Kremer JAM, Hermens RPMG. Development of guideline-based quality indicators for recurrent miscarriage. Reprod Biomed Online. 2010;20(2):267–73.
pubmed: 20113965
van den Berg MMJ, Hajenius PJ, Mol F, Hermens R, van der Veen F, Goddijn M, et al. Guideline-based quality indicators for early pregnancy assessment units. Reprod Biomed Online. 2020;40(3):453–9.
pubmed: 32044275
Kötter T, Schaefer FA, Scherer M, Blozik E. Involving patients in quality indicator development—a systematic review. Patient Prefer Adherence. 2013;7:259–68.
pubmed: 23569365
pmcid: 3616132
Blackwood J, Armstrong MJ, Schaefer C, Graham ID, Knaapen L, Straus SE, et al. How do guideline developers identify, incorporate and report patient preferences? An international cross-sectional survey. BMC Health Serv Res. 2020;20(1):458.
pubmed: 32448198
pmcid: 7247137
van de Bovenkamp HM, Zuiderent-Jerak T. An empirical study of patient participation in guideline development: exploring the potential for articulating patient knowledge in evidence-based epistemic settings. Health Expect Int J Public Particip Health Care Health Policy. 2015;18(5):942–55.
Petkovic J, Riddle A, Akl EA, Khabsa J, Lytvyn L, Atwere P, et al. Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Syst Rev. 2020;9(1):21.
pubmed: 32007104
pmcid: 6995157
Concannon TW, Grant S, Welch V, Petkovic J, Selby J, Crowe S, et al. Practical guidance for involving stakeholders in health research. J Gen Intern Med. 2019;34(3):458–63.
pubmed: 30565151
Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3(1):13.
pubmed: 29062538
pmcid: 5611595
Health Research Authority/INVOLVE. Public involvement in research and research ethics committee review. Nottingham: Health Research Authority/INVOLVE; 2016.
Dixon N. Guide to managing ethical issues in quality improvement or clinical audit projects. London: Healthcare Quality Improvement Partnership Ltd; 2017.
Devane D, Barrett N, Gallen A, O’Reilly MF, Nadin M, Conway G, et al. Identifying and prioritising midwifery care process metrics and indicators: a Delphi survey and stakeholder consensus process. BMC Pregnancy Childbirth. 2019;19(1):198.
pubmed: 31182055
pmcid: 6558705
Siering U, Eikermann M, Hausner E, Hoffmann-Esser W, Neugebauer EA. Appraisal tools for clinical practice guidelines: a systematic review. PLoS ONE. 2013;8(12):e82915.
pubmed: 24349397
pmcid: 3857289
Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–31.
pubmed: 22473437
Hennessy M, Dennehy R, Meaney S, Linehan L, Devane D, O’Donoghue K. Extended data: clinical practice guidelines for recurrent miscarriage in high-income countries: a systematic review. Retrieved from osf.io/xbfyp ; 2021
Smith PP, Dhillon-Smith RK, O’Toole E, Cooper NAM, Coomarasamy A, Clark TJ. Outcomes in prevention and management of miscarriage trials: a systematic review. BJOG Int J Obstet Gynaecol. 2019;126(2):176–89.
Hsu C, Sandford B. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(10):1–8.
Flenady V, Wojcieszek AM, Fjeldheim I, Friberg IK, Nankabirwa V, Jani JV, et al. eRegistries: indicators for the WHO Essential Interventions for reproductive, maternal, newborn and child health. BMC Pregnancy Childbirth. 2016;16(1):293.
pubmed: 27716088
pmcid: 5045645
Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: Sage Publications Ltd; 2013.
Knowles SE, Allen D, Donnelly A, Flynn J, Gallacher K, Lewis A, et al. More than a method: trusting relationships, productive tensions, and two-way learning as mechanisms of authentic co-production. Res Involv Engagem. 2021;7(1):34.
pubmed: 34059159
pmcid: 8165763
Williams O, Robert G, Martin GP, Hanna E, O’Hara J. Is co-production just really good PPI? Making sense of patient and public involvement and co-production networks. In: Bevir M, Waring J, editors. Decentering healthcare networks organizational behaviour in healthcare. Cham: Palgrave Macmillan; 2020. p. 213–37.
Pearce C. The complexities of developing equal relationships in patient and public involvement in health research. Social Theory Health. 2020;19:362–79.
Williams O, Sarre S, Papoulias SC, Knowles S, Robert G, Beresford P, et al. Lost in the shadows: reflections on the dark side of co-production. Health Res Policy Syst. 2020;18(1):43.
pubmed: 32380998
pmcid: 7204208
Khalife D, Ghazeeri G, Kutteh W. Review of current guidelines for recurrent pregnancy loss: new strategies for optimal evaluation of women who may be superfertile. Semin Perinatol. 2019;43(2):105–15.
pubmed: 30642578
Howlett M. Looking at the ‘field’ through a Zoom lens: methodological reflections on conducting online research during a global pandemic. Qual Res. 2021;1468794120985691.
Langley J, Wallace N, Davis A, Gwilt I, Knowles S, Partridge R, et al. COVID co-design does not *HAVE* to be digital! Why ‘which platform should we use?’ should not be your first question. In: Williams O, Tembo D, Ocloo J, Kaur M, Hickey G, Farr M, et al., editors. COVID-19 and co-production in health and social care research, policy and practice volume 2: co-production methods and working together at a distance. Bristol: Policy Press; 2021. p. 85–95.
Fiset VJ, Davies BL, Graham ID, Gifford W, Woodend K. Developing guideline-based quality indicators: assessing gaps in pain management practice. Int J Evid Based Healthc. 2019;17(2):92–105.
pubmed: 30633009
Langendam MW, Piggott T, Nothacker M, Agarwal A, Armstrong D, Baldeh T, et al. Approaches of integrating the development of guidelines and quality indicators: a systematic review. BMC Health Serv Res. 2020;20(1):875.
pubmed: 32938461
pmcid: 7493171
Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011;6(6):e20476.
pubmed: 21694759
pmcid: 3111406
Gage AD, Carnes F, Blossom J, Aluvaala J, Amatya A, Mahat K, et al. In low- and middle-income countries, is delivery in high-quality obstetric facilities geographically feasible? Health Aff. 2019;38(9):1576–84.
Goldenberg RL, McClure EM, Saleem S. Improving pregnancy outcomes in low- and middle-income countries. Reprod Health. 2018;15(Suppl 1):88.
pubmed: 29945628
pmcid: 6019988