A combination of two methods for evaluating the usability of a hospital information system.
Heuristic evaluation
Hospital information system
Human-computer interaction
Think aloud
Usability evaluation
Journal
BMC medical informatics and decision making
ISSN: 1472-6947
Titre abrégé: BMC Med Inform Decis Mak
Pays: England
ID NLM: 101088682
Informations de publication
Date de publication:
04 05 2020
04 05 2020
Historique:
received:
09
10
2019
accepted:
29
03
2020
entrez:
6
5
2020
pubmed:
6
5
2020
medline:
17
12
2020
Statut:
epublish
Résumé
None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system. Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods. The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the "Major" level. Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems.
Sections du résumé
BACKGROUND
None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system.
METHODS
Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods.
RESULTS
The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the "Major" level.
CONCLUSION
Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems.
Identifiants
pubmed: 32366248
doi: 10.1186/s12911-020-1083-6
pii: 10.1186/s12911-020-1083-6
pmc: PMC7199374
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
84Références
Stud Health Technol Inform. 2008;136:327-32
pubmed: 18487752
Int J Med Inform. 2014 Jul;83(7):473-83
pubmed: 24874987
J Biomed Inform. 2014 Dec;52:141-50
pubmed: 24333875
Adv Physiol Educ. 2013 Sep;37(3):242-8
pubmed: 24022770
Stud Health Technol Inform. 2009;143:322-7
pubmed: 19380955
Appl Clin Inform. 2016 Feb 17;7(1):89-100
pubmed: 27081409
Int J Med Inform. 2007 Oct;76 Suppl 2:S317-24
pubmed: 17616431
J Med Syst. 2014 Apr;38(4):35
pubmed: 24682671
Int J Technol Assess Health Care. 2018 Jan;34(1):68-77
pubmed: 29455684
J Biomed Inform. 2005 Jun;38(3):189-99
pubmed: 15896692
Int J Med Inform. 2010 Aug;79(8):565-75
pubmed: 20617546
J Biomed Inform. 2018 Apr;80:43-51
pubmed: 29501909
AMIA Annu Symp Proc. 2009 Nov 14;2009:714-8
pubmed: 20351946
J Biomed Inform. 2018 Feb;78:12-23
pubmed: 29305953
J Am Med Inform Assoc. 2017 Apr 1;24(e1):e55-e60
pubmed: 27497799
Int J Med Inform. 2010 Jun;79(6):e135-43
pubmed: 18757234
Comput Inform Nurs. 2017 Jun;35(6):307-314
pubmed: 28060182
Int J Med Inform. 2011 May;80(5):341-50
pubmed: 21435943
Int J Med Inform. 2009 May;78(5):340-53
pubmed: 19046928
Int J Med Inform. 2007 Feb-Mar;76(2-3):157-68
pubmed: 16854617