Evaluation of a clinical decision support system for rare diseases: a qualitative study.
Clinical decision support systems
Computer-assisted diagnosis
Rare diseases
Usability
Journal
BMC medical informatics and decision making
ISSN: 1472-6947
Titre abrégé: BMC Med Inform Decis Mak
Pays: England
ID NLM: 101088682
Informations de publication
Date de publication:
18 02 2021
18 02 2021
Historique:
received:
10
12
2020
accepted:
10
02
2021
entrez:
19
2
2021
pubmed:
20
2
2021
medline:
24
4
2021
Statut:
epublish
Résumé
Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS. We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS). A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability. This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.
Sections du résumé
BACKGROUND
Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS.
METHODS
We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS).
RESULTS
A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability.
CONCLUSIONS
This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.
Identifiants
pubmed: 33602191
doi: 10.1186/s12911-021-01435-8
pii: 10.1186/s12911-021-01435-8
pmc: PMC7890997
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
65Références
Clinics (Sao Paulo). 2018;73:e68
pubmed: 29641803
Int J Med Inform. 2017 Oct;106:1-8
pubmed: 28870378
J Med Internet Res. 2019 Mar 19;21(3):e11732
pubmed: 30888324
Int J Med Inform. 2012 Nov;81(11):761-72
pubmed: 22456088
Int J Med Inform. 2014 Sep;83(9):636-47
pubmed: 24981988
JMIR Hum Factors. 2015 Sep 10;2(2):e14
pubmed: 27025540
Adv Exp Med Biol. 2017;1031:25-38
pubmed: 29214564
BMC Med Inform Decis Mak. 2020 Sep 16;20(1):230
pubmed: 32938448
AMIA Annu Symp Proc. 2006;:1167-8
pubmed: 17238783
Qual Quant. 2018;52(4):1893-1907
pubmed: 29937585
Appl Clin Inform. 2018 Jul;9(3):604-634
pubmed: 30112741
Acad Med. 2014 Sep;89(9):1245-51
pubmed: 24979285
Stud Health Technol Inform. 2020 Jun 23;271:176-183
pubmed: 32578561
J Biomed Inform. 2015 Aug;56:284-91
pubmed: 26071683
BMC Med Inform Decis Mak. 2016 Aug 02;16:101
pubmed: 27484923
Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2017 May;60(5):523-531
pubmed: 28289778
NPJ Digit Med. 2020 Feb 6;3:17
pubmed: 32047862
JMIR Hum Factors. 2019 Apr 17;6(2):e12469
pubmed: 30994460
JMIR Med Inform. 2018 Oct 10;6(4):e11301
pubmed: 30305261
Int J Med Inform. 2015 Dec;84(12):1009-18
pubmed: 26391601
J Diabetes Sci Technol. 2013 Jul 01;7(4):1039-56
pubmed: 23911188
Int J Qual Health Care. 2007 Dec;19(6):349-57
pubmed: 17872937
Stud Health Technol Inform. 2015;216:368-71
pubmed: 26262073
J Biomed Inform. 2017 Jan;65:1-21
pubmed: 27856379
BMJ. 1995 Jul 8;311(6997):109-12
pubmed: 7613363
J Biomed Inform. 2004 Feb;37(1):56-76
pubmed: 15016386
Orphanet J Rare Dis. 2020 Sep 24;15(1):263
pubmed: 32972444
Br J Gen Pract. 2016 Nov;66(652):550-551
pubmed: 27789486
Stud Health Technol Inform. 2019 Aug 21;264:1580-1581
pubmed: 31438241
Nat Rev Genet. 2013 Jun;14(6):372
pubmed: 23821785