Assessment of Diagnostic Competences With Standardized Patients Versus Virtual Patients: Experimental Study in the Context of History Taking.
clinical reasoning
medical education
performance-based assessment
simulation
standardized patient
virtual patient
Journal
Journal of medical Internet research
ISSN: 1438-8871
Titre abrégé: J Med Internet Res
Pays: Canada
ID NLM: 100959882
Informations de publication
Date de publication:
04 03 2021
04 03 2021
Historique:
received:
25
06
2020
accepted:
27
12
2020
revised:
01
10
2020
entrez:
4
3
2021
pubmed:
5
3
2021
medline:
30
9
2021
Statut:
epublish
Résumé
Standardized patients (SPs) have been one of the popular assessment methods in clinical teaching for decades, although they are resource intensive. Nowadays, simulated virtual patients (VPs) are increasingly used because they are permanently available and fully scalable to a large audience. However, empirical studies comparing the differential effects of these assessment methods are lacking. Similarly, the relationships between key variables associated with diagnostic competences (ie, diagnostic accuracy and evidence generation) in these assessment methods still require further research. The aim of this study is to compare perceived authenticity, cognitive load, and diagnostic competences in performance-based assessment using SPs and VPs. This study also aims to examine the relationships of perceived authenticity, cognitive load, and quality of evidence generation with diagnostic accuracy. We conducted an experimental study with 86 medical students (mean 26.03 years, SD 4.71) focusing on history taking in dyspnea cases. Participants solved three cases with SPs and three cases with VPs in this repeated measures study. After each case, students provided a diagnosis and rated perceived authenticity and cognitive load. The provided diagnosis was scored in terms of diagnostic accuracy; the questions asked by the medical students were rated with respect to their quality of evidence generation. In addition to regular null hypothesis testing, this study used equivalence testing to investigate the absence of meaningful effects. Perceived authenticity (1-tailed t The results on perceived authenticity demonstrate that learners experience SPs as more authentic than VPs. As higher amounts of intrinsic and extraneous cognitive loads are detrimental to performance, both types of cognitive load must be monitored and manipulated systematically in the assessment. Diagnostic accuracy was higher for SPs than for VPs, which could potentially negatively affect students' grades with VPs. We identify and discuss possible reasons for this performance difference between both assessment methods.
Sections du résumé
BACKGROUND
Standardized patients (SPs) have been one of the popular assessment methods in clinical teaching for decades, although they are resource intensive. Nowadays, simulated virtual patients (VPs) are increasingly used because they are permanently available and fully scalable to a large audience. However, empirical studies comparing the differential effects of these assessment methods are lacking. Similarly, the relationships between key variables associated with diagnostic competences (ie, diagnostic accuracy and evidence generation) in these assessment methods still require further research.
OBJECTIVE
The aim of this study is to compare perceived authenticity, cognitive load, and diagnostic competences in performance-based assessment using SPs and VPs. This study also aims to examine the relationships of perceived authenticity, cognitive load, and quality of evidence generation with diagnostic accuracy.
METHODS
We conducted an experimental study with 86 medical students (mean 26.03 years, SD 4.71) focusing on history taking in dyspnea cases. Participants solved three cases with SPs and three cases with VPs in this repeated measures study. After each case, students provided a diagnosis and rated perceived authenticity and cognitive load. The provided diagnosis was scored in terms of diagnostic accuracy; the questions asked by the medical students were rated with respect to their quality of evidence generation. In addition to regular null hypothesis testing, this study used equivalence testing to investigate the absence of meaningful effects.
RESULTS
Perceived authenticity (1-tailed t
CONCLUSIONS
The results on perceived authenticity demonstrate that learners experience SPs as more authentic than VPs. As higher amounts of intrinsic and extraneous cognitive loads are detrimental to performance, both types of cognitive load must be monitored and manipulated systematically in the assessment. Diagnostic accuracy was higher for SPs than for VPs, which could potentially negatively affect students' grades with VPs. We identify and discuss possible reasons for this performance difference between both assessment methods.
Identifiants
pubmed: 33661122
pii: v23i3e21196
doi: 10.2196/21196
pmc: PMC7974754
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e21196Informations de copyright
©Maximilian C Fink, Victoria Reitmeier, Matthias Stadler, Matthias Siebeck, Frank Fischer, Martin R Fischer. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 04.03.2021.
Références
J Clin Nurs. 2011 Dec;20(23-24):3543-52
pubmed: 21917033
Acad Med. 2014 Mar;89(3):387-92
pubmed: 24448038
Med Educ. 2016 Sep;50(9):955-68
pubmed: 27562895
Nurse Educ Today. 2012 May;32(4):448-52
pubmed: 21565436
J Med Educ. 1964 Aug;39:802-5
pubmed: 14180699
Acad Med. 2002 Jan;77(1):87-90
pubmed: 11788331
BMC Med Educ. 2013 Feb 22;13:28
pubmed: 23433202
Med Educ. 2011 Aug;45(8):807-17
pubmed: 21752077
Med Teach. 2014 May;36(5):371-84
pubmed: 24593808
Am J Pharm Educ. 2006 Apr 15;70(2):33
pubmed: 17149413
Adv Simul (Lond). 2017 Jun 27;2:10
pubmed: 29450011
Ann Intern Med. 1986 Nov;105(5):762-71
pubmed: 3767153
Acad Med. 1993 Jun;68(6):443-51; discussion 451-3
pubmed: 8507309
Acad Med. 2018 Jul;93(7):990-995
pubmed: 29369086
Acad Med. 1990 Sep;65(9 Suppl):S63-7
pubmed: 2400509
Adv Health Sci Educ Theory Pract. 2016 Aug;21(3):505-21
pubmed: 26433730
Br J Gen Pract. 1991 Mar;41(344):97-9
pubmed: 2031767
J Gen Intern Med. 1989 Mar-Apr;4(2):108-14
pubmed: 2709168
Med Teach. 2010;32(10):845-50
pubmed: 20854161
Br Med J. 1975 Feb 22;1(5955):447-51
pubmed: 1115966
Soc Psychol Personal Sci. 2017 May;8(4):355-362
pubmed: 28736600
Med Educ. 2004 Sep;38(9):958-68
pubmed: 15327677
Adv Health Sci Educ Theory Pract. 2019 Oct;24(4):811-825
pubmed: 30456474
Simul Healthc. 2017 Apr;12(2):117-123
pubmed: 28704289
Med Educ. 2012 Jul;46(7):636-47
pubmed: 22616789
JMIR Med Educ. 2020 Mar 12;6(1):e14428
pubmed: 32163036
Med Decis Making. 1991 Oct-Dec;11(4):265-72
pubmed: 1766329
Simul Healthc. 2012 Oct;7(5):308-20
pubmed: 23032751
Acad Med. 2000 Aug;75(8):825-33
pubmed: 10965862
Med Educ. 2008 Aug;42(8):823-9
pubmed: 18564096