Engaging clinicians early during the development of a graphical user display of an intelligent alerting system at the bedside.
Focus groups
Graphical user interface
Modeling
Journal
International journal of medical informatics
ISSN: 1872-8243
Titre abrégé: Int J Med Inform
Pays: Ireland
ID NLM: 9711057
Informations de publication
Date de publication:
03 2022
03 2022
Historique:
received:
16
05
2021
revised:
13
10
2021
accepted:
08
11
2021
pubmed:
2
1
2022
medline:
5
4
2022
entrez:
1
1
2022
Statut:
ppublish
Résumé
Artificial Intelligence (AI) is increasingly used to support bedside clinical decisions, but information must be presented in usable ways within workflow. Graphical User Interfaces (GUI) are front-facing presentations for communicating AI outputs, but clinicians are not routinely invited to participate in their design, hindering AI solution potential. To inform early user-engaged design of a GUI prototype aimed at predicting future Cardiorespiratory Insufficiency (CRI) by exploring clinician methods for identifying at-risk patients, previous experience with implementing new technologies into clinical workflow, and user perspectives on GUI screen changes. We conducted a qualitative focus group study to elicit iterative design feedback from clinical end-users on an early GUI prototype display. Five online focus group sessions were held, each moderated by an expert focus group methodologist. Iterative design changes were made sequentially, and the updated GUI display was presented to the next group of participants. 23 clinicians were recruited (14 nurses, 4 nurse practitioners, 5 physicians; median participant age ∼35 years; 60% female; median clinical experience 8 years). Five themes emerged from thematic content analysis: trend evolution, context (risk evolution relative to vital signs and interventions), evaluation/interpretation/explanation (sub theme: continuity of evaluation), clinician intuition, and clinical operations. Based on these themes, GUI display changes were made. For example, color and scale adjustments, integration of clinical information, and threshold personalization. Early user-engaged design was useful in adjusting GUI presentation of AI output. Next steps involve clinical testing and further design modification of the AI output to optimally facilitate clinician surveillance and decisions. Clinicians should be involved early and often in clinical decision support design to optimize efficacy of AI tools.
Sections du résumé
BACKGROUND
Artificial Intelligence (AI) is increasingly used to support bedside clinical decisions, but information must be presented in usable ways within workflow. Graphical User Interfaces (GUI) are front-facing presentations for communicating AI outputs, but clinicians are not routinely invited to participate in their design, hindering AI solution potential.
PURPOSE
To inform early user-engaged design of a GUI prototype aimed at predicting future Cardiorespiratory Insufficiency (CRI) by exploring clinician methods for identifying at-risk patients, previous experience with implementing new technologies into clinical workflow, and user perspectives on GUI screen changes.
METHODS
We conducted a qualitative focus group study to elicit iterative design feedback from clinical end-users on an early GUI prototype display. Five online focus group sessions were held, each moderated by an expert focus group methodologist. Iterative design changes were made sequentially, and the updated GUI display was presented to the next group of participants.
RESULTS
23 clinicians were recruited (14 nurses, 4 nurse practitioners, 5 physicians; median participant age ∼35 years; 60% female; median clinical experience 8 years). Five themes emerged from thematic content analysis: trend evolution, context (risk evolution relative to vital signs and interventions), evaluation/interpretation/explanation (sub theme: continuity of evaluation), clinician intuition, and clinical operations. Based on these themes, GUI display changes were made. For example, color and scale adjustments, integration of clinical information, and threshold personalization.
CONCLUSIONS
Early user-engaged design was useful in adjusting GUI presentation of AI output. Next steps involve clinical testing and further design modification of the AI output to optimally facilitate clinician surveillance and decisions. Clinicians should be involved early and often in clinical decision support design to optimize efficacy of AI tools.
Identifiants
pubmed: 34973608
pii: S1386-5056(21)00269-0
doi: 10.1016/j.ijmedinf.2021.104643
pmc: PMC9040820
mid: NIHMS1795800
pii:
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Langues
eng
Sous-ensembles de citation
IM
Pagination
104643Subventions
Organisme : NINR NIH HHS
ID : F31 NR019725
Pays : United States
Organisme : NINR NIH HHS
ID : R01 NR013912
Pays : United States
Organisme : NINR NIH HHS
ID : T32 NR008857
Pays : United States
Informations de copyright
Copyright © 2021. Published by Elsevier B.V.
Références
Ann Am Thorac Soc. 2017 Mar;14(3):384-391
pubmed: 28033032
Am J Respir Crit Care Med. 2014 Sep 15;190(6):649-55
pubmed: 25089847
J Clin Monit Comput. 2014 Aug;28(4):329-39
pubmed: 24248424
Qual Health Res. 2012 Oct;22(10):1383-94
pubmed: 22811304
Crit Care Med. 2006 Sep;34(9):2463-78
pubmed: 16878033
AMIA Jt Summits Transl Sci Proc. 2020 May 30;2020:191-200
pubmed: 32477638
Proc Natl Acad Sci U S A. 1991 Mar 15;88(6):2297-301
pubmed: 11607165
Chest. 2018 Nov;154(5):1239-1248
pubmed: 29752973
Am J Physiol Regul Integr Comp Physiol. 2002 Sep;283(3):R789-97
pubmed: 12185014
Nat Commun. 2020 Aug 7;11(1):3966
pubmed: 32769990
Int J Med Inform. 2016 Dec;96:24-37
pubmed: 26847070
JAMA. 2019 Oct 8;322(14):1351-1352
pubmed: 31393527
J Biomed Inform X. 2019 Sep;3:
pubmed: 31423485
Surgery. 2017 May;161(5):1357-1366
pubmed: 27842918
J Am Med Inform Assoc. 2021 Mar 1;28(3):653-663
pubmed: 33325504
J Biomed Inform. 2017 Nov;75:14-21
pubmed: 28942140
Int J Nurs Stud Adv. 2021 Nov;3:100019
pubmed: 33426534
Stud Health Technol Inform. 2010;160(Pt 2):927-31
pubmed: 20841820
Crit Care Nurs Clin North Am. 2018 Jun;30(2):273-287
pubmed: 29724445
BMC Med. 2019 Oct 29;17(1):195
pubmed: 31665002
AMIA Annu Symp Proc. 2015 Nov 05;2015:1841-50
pubmed: 26958283
Crit Care. 2016 Mar 16;20:70
pubmed: 26984263
Crit Care Med. 2016 Feb;44(2):368-74
pubmed: 26771782
Crit Care Med. 2016 Sep;44(9):1639-48
pubmed: 27452809
BMC Med Inform Decis Mak. 2020 Oct 8;20(1):257
pubmed: 33032582