Evaluating the Value of Eye-Tracking Augmented Debriefing in Medical Simulation-A Pilot Randomized Controlled Trial.
Journal
Simulation in healthcare : journal of the Society for Simulation in Healthcare
ISSN: 1559-713X
Titre abrégé: Simul Healthc
Pays: United States
ID NLM: 101264408
Informations de publication
Date de publication:
24 Oct 2024
24 Oct 2024
Historique:
medline:
29
10
2024
pubmed:
29
10
2024
entrez:
29
10
2024
Statut:
aheadofprint
Résumé
Debriefing after simulation facilitates reflective thinking and learning. Eye-tracking augmented debriefing (ETAD) may provide advantages over traditional debriefing (TD) by leveraging video replay with first-person perspective. This multisite randomized controlled trial compared the impact of ETAD with TD (without eye-tracking and without video) after simulation on 4 outcomes: (1) resident metacognitive awareness (the primary outcome), (2) cognitive load (CL) of residents and debriefers, (3) alignment of resident self-assessment and debriefer assessment scores, and (4) resident and debriefer perceptions of the debriefing experience. Fifty-four emergency medicine residents from 2 institutions were randomized to the experimental (ETAD) or the control (TD) arm. Residents completed 2 simulation stations followed by debriefing. Before station 1 and after station 2, residents completed a Metacognition Awareness Inventory (MAI). After each station, debriefers and residents rated their CL and completed an assessment of performance. After the stations, residents were interviewed and debriefers participated in a focus group. There were no statistically significant differences in mean MAI change, resident CL, or assessment alignment between residents and debriefers. Debriefer CL was lower in the experimental arm. Interviews identified 4 themes: (1) reflections related to debriefing approach, (2) eye-tracking as a metacognitive sensitizer, (3) translation of metacognition to practice, and (4) ETAD as a strategy to manage CL. Residents reported that eye tracking improved the specificity of feedback. Debriefers relied less on notes, leveraged video timestamps, appreciated the structure of the eye-tracking video, and found the video useful when debriefing poor performers. There were no significant quantitative differences in MAI or resident CL scores; qualitative findings suggest that residents appreciated the benefits of the eye-tracking video review. Debriefers expended less CL and reported less perceived mental effort with the new technology. Future research should leverage longitudinal experimental designs to further understand the impact of eye-tracking facilitated debriefing.
Identifiants
pubmed: 39470255
doi: 10.1097/SIH.0000000000000825
pii: 01266021-990000000-00155
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
Copyright © 2024 Society for Simulation in Healthcare.
Déclaration de conflit d'intérêts
The authors declare no conflict of interest.
Références
Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc 2007;2(2):115–125.
Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc 2015;10(2):106–115.
Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simul Healthc 2016;11(3):209–217. doi:10.1097/SIH.0000000000000148.
doi: 10.1097/SIH.0000000000000148
Szulewski A, Braund H, Egan R, et al. Through the learner's lens: eye-tracking augmented debriefing in medical simulation. J Grad Med Educ 2018;10(3):340–341. doi:10.4300/JGME-D-17-00827.1.
doi: 10.4300/JGME-D-17-00827.1
Szulewski A, Braund H, Egan R, et al. Starting to think like an expert: an analysis of resident cognitive processes during simulation-based resuscitation examinations. Ann Emerg Med 2019;74(5):647–659. doi:10.1016/j.annemergmed.2019.04.002.
doi: 10.1016/j.annemergmed.2019.04.002
White MR, Braund H, Howes D, et al. Getting inside the expert's head: an analysis of physician cognitive processes during trauma resuscitations. Ann Emerg Med 2018;72:289–298.
Wagner M, Gropel P, Bibl K, Olischar M, Auerbach MA, Gross IT. Eye-tracking during simulation-based neonatal airway management. Pediatr Res 2020;87(3):518–522. doi:10.1038/s41390-019-0571-9.
doi: 10.1038/s41390-019-0571-9
Pelaccia T, Tardif J, Triby E, et al. How and when do expert emergency physicians generate and evaluate diagnostic hypotheses? A qualitative study using head-mounted video cued-recall interviews. Ann Emerg Med 2014;64(6):575–585.
Zhang H, Morelius E, Goh SHL, Wang W. Effectiveness of video-assisted debriefing in simulation-based health professions education: a systematic review of quantitative evidence. Nurse Educ 2019;44(3):E1–E6. doi:10.1097/NNE.0000000000000562.
doi: 10.1097/NNE.0000000000000562
Schraw G, Dennison RS. Assessing metacognitive awareness. Contemp Educ Psychol 1994;19(4):460–475.
Turan S, Demirel O, Sayek I. Metacognitive awareness and self-regulated learning skills of medical students in different medical curricula. Med Teach 2009;31(10):e477–e483. doi:10.3109/01421590903193521.
doi: 10.3109/01421590903193521
Young A, Fry J. Metacognitive awareness and academic achievement in college students. J Scholarsh Teach Learn 2008;8(2):1–10.
Ayres P. Using subjective measures to detect variations of intrinsic cognitive load within problems. Learn Instr 2006;16(5):389–400. doi:10.1016/j.learninstruc.2006.09.001.
doi: 10.1016/j.learninstruc.2006.09.001
Paas F, Tuovinen J, Tabbers H, Van Gerven P. Cognitive load measurement as a means to advance cognitive load theory. Educ Psychol 2003;38(1):63–71.
Szulewski A, Gegenfurtner A, Howes DW, Sivilotti MLA, van Merrienboer JJG. Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool. Adv Health Sci Educ Theory Pract 2017;22(4):951–968. doi:10.1007/s10459-016-9725-2.
doi: 10.1007/s10459-016-9725-2
Cierniak G, Scheiter K, Gerjets P. Explaining the split-attention effect: is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Comput Hum Behav 2009;25(2):315–324. doi:10.1016/j.chb.2008.12.020.
doi: 10.1016/j.chb.2008.12.020
Ten Cate O, Hart D, Ankel F. Entrustment decision making in clinical training. Acad Med 2016;91(2):191–198.
Weersink K, Hall AK, Rich J, Szulewski A, Dagnone D. Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring. Adv Simul 2019;4:9.
Hall AK, Dagnone JD, Lacroix L, Pickett W, Klinger DA. Queen's simulation assessment tool: development and validation of an assessment tool for resuscitation objective structured clinical examination stations in emergency medicine. Simul Healthc 2015;10(2):98–105. doi:10.1097/SIH.0000000000000076.
doi: 10.1097/SIH.0000000000000076
Hall AK, Damon Dagnone J, Moore S, et al. Comparison of simulation-based resuscitation performance assessments with in-training evaluation reports in emergency medicine residents: a Canadian multicenter study. AEM Educ Train 2017;1(4):293–300. doi:10.1002/aet2.10055.
doi: 10.1002/aet2.10055
Dagnone D, Hall A, Sebok-Syer S, et al. Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees—a Canadian multi-centred study. Can Med Educ J 2016;7(1):e57–e67.
Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ 2015;49(6):560–575.
Schafer JL. Analysis of Incomplete Multivariate Data. New York, NY: Chapman & Hall; 1997.
Cofie N, Braund H, Dalgarno N. Eight ways to get a grip on intercoder reliability using qualitative-based measures. Can Med Educ J 2022;13(2):73–76. doi:10.36834/cmej.72504.
doi: 10.36834/cmej.72504
Cale AS, Hoffman LA, McNulty MA. Promoting metacognition in an allied health anatomy course. Anat Sci Educ 2023;16(3):473–485. doi:10.1002/ase.2218.
doi: 10.1002/ase.2218
Meguerdichian M, Bajaj K, Ivanhoe R, et al. Impact of the PEARLS healthcare debriefing cognitive aid on facilitator cognitive load, workload, and debriefing quality: a pilot study. Adv Simul (Lond) 2022;7(1):40. doi:10.1186/s41077-022-00236-x.
doi: 10.1186/s41077-022-00236-x
Sweller J, van Merriënboer JJG, Paas F. Cognitive architecture and instructional design: 20 years later. Educ Psychol Rev 2019;31(2):261–292. doi:10.1007/s10648-019-09465-5.
doi: 10.1007/s10648-019-09465-5
Croskerry P, Sinclar D. Emergency medicine: a practice prone to error? CJEM 2001;3(4):271–276. doi:10.1017/S1481803500005765.
doi: 10.1017/S1481803500005765
Sweller J. Cognitive load during problem solving: effects on learning. Cognit Sci 1988;12:257–285.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence. JAMA 2006;296(9):1094–1102.
Scaffidi MA, Grover SC, Carnahan H, et al. Impact of experience on self-assessment accuracy of clinical colonoscopy competence. Gastrointest Endosc 2018;87(3):827–836.
Ganni S, Chmarra MK, Goossens RHM, Jakimowicz JJ. Self-assessment in laparoscopic surgical skills training: is it reliable? Surg Endosc 2017;31(6):2451–2456.
Moorthy K, Munz Y, Adams S, Pandey V, Darzi A. Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater. Am J Surg 2006;192(1):114–118.
Larsen T, Jeppe-Jensen D. The introduction and perception of an OSCE with an element of self- and peer-assessment. Eur J Dent Educ 2008;12(1):2–7.
Greer SK, Jeffe DB, Manga A, Murray DJ, Emke AR. Cognitive load assessment scales in simulation: validity evidence for a novel measure of cognitive load types. Simul Healthc 2023;18(3):172–180. doi:10.1097/SIH.0000000000000665.
doi: 10.1097/SIH.0000000000000665