Effectiveness of hybrid simulation training on medical student performance in whole-task consultation of cardiac patients: The ASSIMILATE EXCELLENCE randomized waitlist-controlled trial.
Cardiology
Clerkship
Composite narrative
Consultation
Hybrid simulation
Whole task
Journal
Advances in simulation (London, England)
ISSN: 2059-0628
Titre abrégé: Adv Simul (Lond)
Pays: England
ID NLM: 101700425
Informations de publication
Date de publication:
01 Oct 2024
01 Oct 2024
Historique:
received:
26
03
2024
accepted:
20
09
2024
medline:
2
10
2024
pubmed:
2
10
2024
entrez:
1
10
2024
Statut:
epublish
Résumé
Assessment of comprehensive consultations in medicine, i.e. a complete history, physical examination, and differential diagnosis, is regarded as authentic tests of clinical competence; however, they have been shown to have low reliability and validity due to variability in the real patients used and subjective examiner grading. In the ASSIMILATE EXCELLENCE study, our aim was to assess the effect(s) of expert tuition with hybrid simulation using a simulated patient wearing a novel auscultation vest, i.e. a hybrid simulated patient, and repeated peer grading using scoring checklists on student learning, performance, and acumen in comprehensive consultations of patients with valvular heart disease. ASSIMILATE EXCELLENCE was a randomized waitlist-controlled trial with blinded outcome assessment undertaken between February 2021 and November 2021. Students at the Royal College of Surgeons in Ireland in either the second or third year of the four-year graduate-entry medical degree programme were randomized to a hybrid simulation training or waitlist control group and undertook three consultation assessments of three different clinical presentations of valvular heart disease (cases: C1-C3) using hybrid simulation. Our primary outcome was the difference in total score between and within groups across time; a secondary outcome was any change in inter-rater reliability across time. Students self-reported their proficiency and confidence in comprehensive consultations using a pre- and post-study survey. Included were 68 students (age 27.6 ± 0.1 years; 74% women). Overall, total score was 39.6% (35.6, 44.9) in C1 and increased to 63.6% (56.7, 66.7) in C3 (P < .001). On intergroup analysis, a significant difference was observed between groups in C2 only (54.2 ± 7.1% vs. 45.6 ± 9.2%; P < .001), a finding that was mainly driven by a difference in physical examination score. On intragroup analysis, significant improvement in total score across time between cases was also observed. Intraclass correlation coefficients for each pair of assessors were excellent (0.885-0.996 [0.806, 0.998]) in all cases. Following participation, students' confidence in comprehensive consultation assessments improved, and they felt more prepared for their future careers. Hybrid simulation-based training improves competence and confidence in medical students undertaking comprehensive consultation assessment of cardiac patients. In addition, weighted scoring checklists improve grading consistency, learning through peer assessment, and feedback. Trial registration ClinicalTrials.gov Identifier: NCT05895799.
Sections du résumé
BACKGROUND
BACKGROUND
Assessment of comprehensive consultations in medicine, i.e. a complete history, physical examination, and differential diagnosis, is regarded as authentic tests of clinical competence; however, they have been shown to have low reliability and validity due to variability in the real patients used and subjective examiner grading. In the ASSIMILATE EXCELLENCE study, our aim was to assess the effect(s) of expert tuition with hybrid simulation using a simulated patient wearing a novel auscultation vest, i.e. a hybrid simulated patient, and repeated peer grading using scoring checklists on student learning, performance, and acumen in comprehensive consultations of patients with valvular heart disease.
METHODS
METHODS
ASSIMILATE EXCELLENCE was a randomized waitlist-controlled trial with blinded outcome assessment undertaken between February 2021 and November 2021. Students at the Royal College of Surgeons in Ireland in either the second or third year of the four-year graduate-entry medical degree programme were randomized to a hybrid simulation training or waitlist control group and undertook three consultation assessments of three different clinical presentations of valvular heart disease (cases: C1-C3) using hybrid simulation. Our primary outcome was the difference in total score between and within groups across time; a secondary outcome was any change in inter-rater reliability across time. Students self-reported their proficiency and confidence in comprehensive consultations using a pre- and post-study survey.
RESULTS
RESULTS
Included were 68 students (age 27.6 ± 0.1 years; 74% women). Overall, total score was 39.6% (35.6, 44.9) in C1 and increased to 63.6% (56.7, 66.7) in C3 (P < .001). On intergroup analysis, a significant difference was observed between groups in C2 only (54.2 ± 7.1% vs. 45.6 ± 9.2%; P < .001), a finding that was mainly driven by a difference in physical examination score. On intragroup analysis, significant improvement in total score across time between cases was also observed. Intraclass correlation coefficients for each pair of assessors were excellent (0.885-0.996 [0.806, 0.998]) in all cases. Following participation, students' confidence in comprehensive consultation assessments improved, and they felt more prepared for their future careers.
CONCLUSIONS
CONCLUSIONS
Hybrid simulation-based training improves competence and confidence in medical students undertaking comprehensive consultation assessment of cardiac patients. In addition, weighted scoring checklists improve grading consistency, learning through peer assessment, and feedback. Trial registration ClinicalTrials.gov Identifier: NCT05895799.
Identifiants
pubmed: 39354536
doi: 10.1186/s41077-024-00314-2
pii: 10.1186/s41077-024-00314-2
doi:
Banques de données
ClinicalTrials.gov
['NCT05895799']
Types de publication
Journal Article
Langues
eng
Pagination
40Informations de copyright
© 2024. The Author(s).
Références
Vukanovic-Criley JM, et al. Competency in cardiac examination skills in medical students, trainees, physicians, and faculty: a multicenter study. Arch Intern Med. 2006;166(6):610–6.
pubmed: 16567598
doi: 10.1001/archinte.166.6.610
van Merriënboer JJG, Kester L, Paas F. Teaching complex rather than simple tasks: balancing intrinsic and germane load to enhance transfer of learning. Applied Cognitive Psychology. 2006;20:343–52.
doi: 10.1002/acp.1250
Daly, M. Whole-task learning using real world simulations: a guide. 2022; Available from: https://www.timeshighereducation.com/campus/wholetask-learning-using-real-world-simulations-guide .
Chizner MA. Cardiac auscultation: rediscovering the lost art. Curr Probl Cardiol. 2008;33(7):326–408.
pubmed: 18513577
doi: 10.1016/j.cpcardiol.2008.03.003
Wiener S, Nathanson M. Physical examination. Frequently observed errors. JAMA. 1976;236(7):852–5.
pubmed: 947266
doi: 10.1001/jama.1976.03270080034028
Wray NP, Friedland JA. Detection and correction of house staff error in physical diagnosis. JAMA. 1983;249(8):1035–7.
pubmed: 6823058
doi: 10.1001/jama.1983.03330320033027
Craige E. Should auscultation be rehabilitated? N Engl J Med. 1988;318(24):1611–3.
pubmed: 3374530
doi: 10.1056/NEJM198806163182409
Fletcher RH, Fletcher SW. Has medicine outgrown physical diagnosis? Ann Intern Med. 1992;117(9):786–7.
pubmed: 1416582
doi: 10.7326/0003-4819-117-9-786
Mangione S, et al. The teaching and practice of cardiac auscultation during internal medicine and cardiology training. A nationwide survey. Ann Intern Med. 1993;119(1):47–54.
pubmed: 8498764
doi: 10.7326/0003-4819-119-1-199307010-00009
Tavel ME. Cardiac auscultation A glorious past–but does it have a future? Circulation. 1996;93(6):1250–3.
pubmed: 8653848
doi: 10.1161/01.CIR.93.6.1250
Mangione S, Nieman LZ. Cardiac auscultatory skills of internal medicine and family practice trainees. A comparison of diagnostic proficiency. JAMA. 1997;278(9):717–22.
pubmed: 9286830
doi: 10.1001/jama.1997.03550090041030
Mangione S. Cardiac auscultatory skills of physicians-in-training: a comparison of three English-speaking countries. Am J Med. 2001;110(3):210–6.
pubmed: 11182108
doi: 10.1016/S0002-9343(00)00673-2
Perlini S, et al. Simulation-guided cardiac auscultation improves medical students’ clinical skills: the Pavia pilot experience. Intern Emerg Med. 2014;9(2):165–72.
pubmed: 22767224
doi: 10.1007/s11739-012-0811-z
Woywodt A, et al. A novel multimedia tool to improve bedside teaching of cardiac auscultation. Postgrad Med J. 2004;80(944):355–7.
pubmed: 15192171
pmcid: 1743036
doi: 10.1136/pgmj.2003.014944
Recommendations for preclerkship clinical skills education for undergraduate medical education. Task force on the clinical skills education of medical students. 2008, Association of American Medical Colleges: Washington DC.
Osborne C, Brown C, Mostafa A. Effectiveness of high- and low-fidelity simulation-based medical education in teaching cardiac auscultation: a systematic review and meta-analysis. Int J Healthcare Simul. 2022;1(3):75–84.
Yardley S, Hookey C, Lefroy J. Designing whole-task learning opportunities for integrated end-of-life care: a practitioner-derived enquiry. Educ Prim Care. 2013;24(6):436–43.
pubmed: 24196600
doi: 10.1080/14739879.2013.11494214
Vandewaetere M, et al. 4C/ID in medical education: how to design an educational program based on whole-task learning: AMEE Guide No. 93. Med Teach. 2015;37(1):4–20.
pubmed: 25053377
doi: 10.3109/0142159X.2014.928407
Mayer RE. Applying the science of learning to medical education. Med Educ. 2010;44(6):543–9.
pubmed: 20604850
doi: 10.1111/j.1365-2923.2010.03624.x
van Merrienboer JJ, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ. 2010;44(1):85–93.
pubmed: 20078759
doi: 10.1111/j.1365-2923.2009.03498.x
Brown WJ, Tortorella RAW. Hybrid medical simulation – a systematic literature review. Smart Learning Environments. 2020;7:16.
doi: 10.1186/s40561-020-00127-6
Lous ML, et al. Hybrid simulation for obstetrics training: a systematic review. Eur J Obstet Gynaecol Reprod Biol. 2020;246:23–8.
doi: 10.1016/j.ejogrb.2019.12.024
Butcher NJ, et al. Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension. JAMA. 2022;328(22):2252–64.
pubmed: 36511921
doi: 10.1001/jama.2022.21022
Hall, R., Simpson, I., The Cardiovascular History and Physical Examination. 3rd (revised) ed. The ESC Textbook of Cardiovascular Medicine. 2019, Oxford, UK.: Oxford University Press.
Talley, N.J., O’Connor, S., Clinical examination: a systematic guide to physical diagnosis. 4th ed. 2001, Oxford, UK: Blackwell Science Ltd.
McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996;1:30–46.
doi: 10.1037/1082-989X.1.1.30
Alsulimani LK. The feasibility of simulation-based high-stakes assessment in emergency medicine settings: a scoping review. J Educ Health Promot. 2021;10:441.
pubmed: 35071647
pmcid: 8719568
doi: 10.4103/jehp.jehp_1127_20
Hatala R, et al. Incorporating simulation technology in a canadian internal medicine specialty examination: a descriptive report. Acad Med. 2005;80(6):554–6.
pubmed: 15917358
doi: 10.1097/00001888-200506000-00007
Petrusa ER. Current challenges and future opportunities for simulation in high-stakes assessment. Simul Healthc. 2009;4(1):3–5.
pubmed: 19212244
doi: 10.1097/SIH.0b013e3181992077
Boulet JR. Summative assessment in medicine: the promise of simulation for high-stakes evaluation. Acad Emerg Med. 2008;15(11):1017–24.
pubmed: 18778377
doi: 10.1111/j.1553-2712.2008.00228.x
Ilgen JS, et al. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161–73.
pubmed: 25626747
doi: 10.1111/medu.12621
Woolliscroft JO, et al. Harvey: the impact of a cardiovascular teaching simulator on student skill acquisition. Med Teach. 1987;9(1):53–7.
pubmed: 3669990
doi: 10.3109/01421598709028980
Jones JS, et al. Assessing bedside cardiologic examination skills using “Harvey”, a cardiology patient simulator. Acad Emerg Med. 1997;4(10):980–5.
pubmed: 9332631
doi: 10.1111/j.1553-2712.1997.tb03664.x
Gordon MS, et al. “Harvey”, the cardiology patient simulator: pilot studies on teaching effectiveness. Am J Cardiol. 1980;45(4):791–6.
pubmed: 7361670
doi: 10.1016/0002-9149(80)90123-X
Sengupta A, et al. Peer-led medical student tutorials using the cardiac simulator ‘Harvey.’ Med Educ. 2007;41(2):219.
pubmed: 17269958
doi: 10.1111/j.1365-2929.2006.02679.x
Friederichs H, et al. Combining simulated patients and simulators: pilot study of hybrid simulation in teaching cardiac auscultation. Adv Physiol Educ. 2014;38(4):343–7.
pubmed: 25434018
doi: 10.1152/advan.00039.2013
Berg D, Berg K. S-Vest: a novel hybrid method to allow standardised patients to put on the objective physical examination findings of a disease. BMJ Innov. 2019;5(2–3):78–81.
pubmed: 32161661
pmcid: 7045787
doi: 10.1136/bmjinnov-2018-000312
Branch, R.M., Kopcha, T.J., Instructional design models., in Handbook of Research on Educational Communications and Technology. 2014, Springer: New York. p. 77-87.
Onyura B, et al. Evidence for curricular and instructional design approaches in undergraduate medical education: an umbrella review. Med Teach. 2016;38(2):150–61.
pubmed: 25665626
doi: 10.3109/0142159X.2015.1009019
Strowd LC, et al. Early pre-clerkship clinical skills assessments predict clerkship performance. Med Sci Educ. 2022;32(2):463–71.
pubmed: 35251766
pmcid: 8886335
doi: 10.1007/s40670-022-01519-8
Howley LD, Wilson WG. Direct observation of students during clerkship rotations: a multiyear descriptive study. Acad Med. 2004;79(3):276–80.
pubmed: 14985204
doi: 10.1097/00001888-200403000-00017
Klamen DL, Borgia PT. Can students’ scores on preclerkship clinical performance examinations predict that they will fail a senior clinical performance examination? Acad Med. 2011;86(4):516–20.
pubmed: 21346501
doi: 10.1097/ACM.0b013e31820de435
Cleland JA, et al. Cohort study on predicting grades: is performance on early MBChB assessments predictive of later undergraduate grades? Med Educ. 2008;42(7):676–83.
pubmed: 18588648
doi: 10.1111/j.1365-2923.2008.03037.x
Martin IG, Jolly B. Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year. Med Educ. 2002;36(5):418–25.
pubmed: 12028391
doi: 10.1046/j.1365-2923.2002.01207.x
Chima M, Dallaghan GB. Does student performance on preclinical OSCEs relate to clerkship grades? Med Educ Online. 2016;21:31724.
pubmed: 27340087
doi: 10.3402/meo.v21.31724
Casey PM, et al. Predictors of medical school clerkship performance: a multispecialty longitudinal analysis of standardized examination scores and clinical assessments. BMC Med Educ. 2016;16:128.
pubmed: 27121276
pmcid: 4848773
doi: 10.1186/s12909-016-0652-y
Cepeda NJ, Vul E, Rohrer D, Wixted JT, Pashler H. Spacing effects in learning: a temporal ridgeline of optimal retention. Psychol Sci. 2008;11:1095–102.
doi: 10.1111/j.1467-9280.2008.02209.x