Users' experiences with an interactive Evidence to Decision (iEtD) framework: a qualitative analysis.
Clinical decision support
Decision-making
Evidence-based health care
GRADE approach
Journal
BMC medical informatics and decision making
ISSN: 1472-6947
Titre abrégé: BMC Med Inform Decis Mak
Pays: England
ID NLM: 101088682
Informations de publication
Date de publication:
25 05 2021
25 05 2021
Historique:
received:
19
01
2021
accepted:
13
05
2021
entrez:
26
5
2021
pubmed:
27
5
2021
medline:
27
5
2021
Statut:
epublish
Résumé
Evidence to Decision (EtD) frameworks bring clarity, structure and transparency to health care decision making. The interactive Evidence to Decision (iEtD) tool, developed in the context of the DECIDE project and published by Epistemonikos, is a stand-alone online solution for producing and using EtD frameworks. Since its development, little is known about how organizations have been using the iEtD tool and what characterizes users' experiences with it. This missing information is necessary for any teams planning future developments of the iEtD tool. This study aimed to describe users' experiences with the iEtD and identify main barriers and facilitators related to use. We contacted all users registered in the iEtD via email and invited people who identified themselves as having used the solution to a semi-structured interview. Audio recordings were transcribed, and one researcher conducted a directed content analysis of the interviews guided by a user experience framework. Two researchers checked the content independently for accuracy. Out of 860 people contacted, 81 people replied to our introductory email (response rate 9.4%). Twenty of these had used the tool in a real scenario and were invited to an interview. We interviewed all eight users that accepted this invitation (from six countries, four continents). 'Guideline development' was the iEtD use scenario they most commonly identified. Most participants reported an overall positive experience, without major difficulties navigating or using the different sections. They reported having used most of the EtD framework criteria. Participants reported tailoring their frameworks, for instance by adding or deleting criteria, translating to another language, or rewording headings. Several people preferred to produce a Word version rather than working online, due to the burden of completing the framework, or lack of experience with the tool. Some reported difficulties working with the exportable formats, as they needed considerable editing. A very limited number of guideline developers have used the iEtD tool published by Epistemonikos since its development. Although users' general experiences are positive, our work has identified some aspects of the tool that need improvement. Our findings could be also applied to development or improvement of other solutions for producing or using EtD frameworks.
Sections du résumé
BACKGROUND
Evidence to Decision (EtD) frameworks bring clarity, structure and transparency to health care decision making. The interactive Evidence to Decision (iEtD) tool, developed in the context of the DECIDE project and published by Epistemonikos, is a stand-alone online solution for producing and using EtD frameworks. Since its development, little is known about how organizations have been using the iEtD tool and what characterizes users' experiences with it. This missing information is necessary for any teams planning future developments of the iEtD tool.
METHODS
This study aimed to describe users' experiences with the iEtD and identify main barriers and facilitators related to use. We contacted all users registered in the iEtD via email and invited people who identified themselves as having used the solution to a semi-structured interview. Audio recordings were transcribed, and one researcher conducted a directed content analysis of the interviews guided by a user experience framework. Two researchers checked the content independently for accuracy.
RESULTS
Out of 860 people contacted, 81 people replied to our introductory email (response rate 9.4%). Twenty of these had used the tool in a real scenario and were invited to an interview. We interviewed all eight users that accepted this invitation (from six countries, four continents). 'Guideline development' was the iEtD use scenario they most commonly identified. Most participants reported an overall positive experience, without major difficulties navigating or using the different sections. They reported having used most of the EtD framework criteria. Participants reported tailoring their frameworks, for instance by adding or deleting criteria, translating to another language, or rewording headings. Several people preferred to produce a Word version rather than working online, due to the burden of completing the framework, or lack of experience with the tool. Some reported difficulties working with the exportable formats, as they needed considerable editing.
CONCLUSION
A very limited number of guideline developers have used the iEtD tool published by Epistemonikos since its development. Although users' general experiences are positive, our work has identified some aspects of the tool that need improvement. Our findings could be also applied to development or improvement of other solutions for producing or using EtD frameworks.
Identifiants
pubmed: 34034723
doi: 10.1186/s12911-021-01532-8
pii: 10.1186/s12911-021-01532-8
pmc: PMC8146986
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
169Commentaires et corrections
Type : ErratumIn
Références
Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, et al. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125–54.
doi: 10.1111/1468-0009.00005
Rosenbaum SE. Improving the user experience of evidence. A design approach to evidence-informed health care. Oslo: Oslo College of Architecture and Design; 2010.
Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
doi: 10.1136/bmj.i2016
Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, et al. GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. J Clin Epidemiol. 2016;76:89–98.
doi: 10.1016/j.jclinepi.2016.01.032
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci IS. 2012;7:50.
doi: 10.1186/1748-5908-7-50
Liang Z, Howard PF, Leggat SG, Murphy G. A framework to improve evidence-informed decision-making in health service management. Aust Health Rev Publ Aust Hosp Assoc. 2012;36(3):284–9.
doi: 10.1071/AH11051
Rosenbaum SE, Moberg J, Glenton C, Schünemann HJ, Lewin S, Akl E, et al. Developing evidence to decision frameworks and an interactive evidence to decision tool for making and using decisions and recommendations in health care. Glob Chall. 2018;2(9):1700081.
doi: 10.1002/gch2.201700081
Moberg J, Oxman AD, Rosenbaum S, Schunemann HJ, Guyatt G, Flottorp S, et al. The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Res Policy Syst. 2018;16(1):45.
doi: 10.1186/s12961-018-0320-2
Parmelli E, Amato L, Oxman AD, Alonso-Coello P, Brunetti M, Moberg J, et al. GRADE Evidence to Decision (EtD) framework for coverage decisions. Int J Technol Assess Health Care. 2017;33(2):176–82.
doi: 10.1017/S0266462317000447
Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines. BMJ. 2016;353:i2089.
doi: 10.1136/bmj.i2089
DECIDE 2011–2015. Interactive GRADE Evidence to Decision (iEtD) framework [cited]. Available from http://www.decide-collaboration.eu/interactive-evidence-decision-ietd-framework .
Li SA, Alexander PE, Reljic T, Cuker A, Nieuwlaat R, Wiercioch W, et al. Evidence to Decision framework provides a structured “roadmap” for making GRADE guidelines recommendations. J Clin Epidemiol. 2018;104:103–12.
doi: 10.1016/j.jclinepi.2018.09.007
Neumann I, Brignardello-Petersen R, Wiercioch W, Carrasco-Labra A, Cuello C, Akl E, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implement Sci IS. 2016;11:93.
doi: 10.1186/s13012-016-0462-y
Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.
doi: 10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-G
O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.
doi: 10.1097/ACM.0000000000000388
Rosenbaum SE, Glenton C, Cracknell J. User experiences of evidence-based online resources for health professionals: user testing of The Cochrane Library. BMC Med Inform Decis Mak. 2008;8(1):34.
doi: 10.1186/1472-6947-8-34
Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010;63(6):607–19.
doi: 10.1016/j.jclinepi.2009.12.013
Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol. 2010;63(6):620–6.
doi: 10.1016/j.jclinepi.2009.12.014
REK: Norwegian Committees for Medical and Health Research Ethics. Examples of activities that do not require approval from REC. Oslo [cited]. Available from https://helseforskning.etikkom.no/reglerogrutiner/soknadsplikt/sokerikkerek?p_dim=34999&_ikbLanguageCode=us .
The National Committee for Research Ethics in the Social Sciences and the Humanities (NESH). Guidelines for Research Ethics in the Social Sciences, Humanities, Law and Theology [cited]. Available from: https://www.forskningsetikk.no/en/guidelines/social-sciences-humanities-law-and-theology/guidelines-for-research-ethics-in-the-social-sciences-humanities-law-and-theology/ .
Jaspers MWM, Steen T, Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform. 2004;73(11):781–95.
doi: 10.1016/j.ijmedinf.2004.08.003
Eccles DW, Arsal G. The think aloud method: what is it and how do I use it? Qual Res Sport Exerc Health. 2017;9(4):514–31.
doi: 10.1080/2159676X.2017.1331501
Rosenbaum SE, Glenton C, Cracknell J. User experiences of evidence-based online resources for health professionals: user testing of The Cochrane Library. BMC Med Inform Decis Mak. 2008;8:34.
doi: 10.1186/1472-6947-8-34
Rosenbaum SE, Glenton C, Wiysonge CS, Abalos E, Mignini L, Young T, et al. Evidence summaries tailored to health policy-makers in low- and middle-income countries. Bull World Health Organ. 2011;89(1):54–61.
doi: 10.2471/BLT.10.075481
Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
doi: 10.1177/1049732305276687
Assarroudi A, Heshmati Nabavi F, Armat MR, Ebadi A, Vaismoradi M. Directed qualitative content analysis: the description and elaboration of its underpinning methods and data analysis process. J Res Nurs. 2018;23(1):42–55.
doi: 10.1177/1744987117741667
US Dept of Health & Human services User Experience Basics. [Cited]. Available from https://www.usability.gov/what-and-why/user-experience.html .
Glenton C, Lewin S, Gülmezoglu AM. Expanding the evidence base for global recommendations on health systems: strengths and challenges of the OptimizeMNH guidance process. Implement Sci. 2016;11(1):98.
doi: 10.1186/s13012-016-0470-y