Searching for an Alternative Method to Evaluate Surgical Knowledge: Does a Drawing and Labeling Quiz Predict In-Service Exam Performance and Enhance Learning?
Journal
The Journal of craniofacial surgery
ISSN: 1536-3732
Titre abrégé: J Craniofac Surg
Pays: United States
ID NLM: 9010410
Informations de publication
Date de publication:
Jun 2020
Jun 2020
Historique:
pubmed:
17
3
2020
medline:
22
10
2020
entrez:
17
3
2020
Statut:
ppublish
Résumé
The best methods to assess surgical knowledge are still debated. The authors used a non-multiple-choice test as a pre- and post-conference assessment to measure residents' knowledge gains with comparison to a standard summative assessment tool. At one didactic conference, plastic surgery residents at a single institution were given a pre-test of drawing and labeling structures in the extensor mechanism of the finger and within the carpal tunnel. The quiz was followed by a lecture on the same material and a subsequent post-test. Scores were correlated with in-service exam performance. Pre-test scores (n = 13) were positively correlated with postgraduate year (PGY) until PGY-3. Performance on labeling structures was higher than performance on the respective drawing prompt. Residents' ability to label structures increased more strongly with PGY than their ability to draw structures. The post-test (n = 8) demonstrated that teaching improves performance on labeling questions (pre-test score = 62%; post-test score = 87%). Improvement was observed across all PGYs. Pretest results were positively correlated with in-service exam performance. Our study suggests that a knowledge test focused on drawing and labeling structures given to surgical residents is a valid, nontraditional method for assessing resident knowledge. Such a quiz would offer programs an alternative method for regularly evaluating residents aside from in-service questions, in order to identify residents who may need targeted training for the in-service exam and to inform teaching plans.Additionally, residents could use quiz feedback to guide study efforts and prime conference-related learning.
Sections du résumé
BACKGROUND
BACKGROUND
The best methods to assess surgical knowledge are still debated. The authors used a non-multiple-choice test as a pre- and post-conference assessment to measure residents' knowledge gains with comparison to a standard summative assessment tool.
METHODS
METHODS
At one didactic conference, plastic surgery residents at a single institution were given a pre-test of drawing and labeling structures in the extensor mechanism of the finger and within the carpal tunnel. The quiz was followed by a lecture on the same material and a subsequent post-test. Scores were correlated with in-service exam performance.
RESULTS
RESULTS
Pre-test scores (n = 13) were positively correlated with postgraduate year (PGY) until PGY-3. Performance on labeling structures was higher than performance on the respective drawing prompt. Residents' ability to label structures increased more strongly with PGY than their ability to draw structures. The post-test (n = 8) demonstrated that teaching improves performance on labeling questions (pre-test score = 62%; post-test score = 87%). Improvement was observed across all PGYs. Pretest results were positively correlated with in-service exam performance.
CONCLUSIONS
CONCLUSIONS
Our study suggests that a knowledge test focused on drawing and labeling structures given to surgical residents is a valid, nontraditional method for assessing resident knowledge. Such a quiz would offer programs an alternative method for regularly evaluating residents aside from in-service questions, in order to identify residents who may need targeted training for the in-service exam and to inform teaching plans.Additionally, residents could use quiz feedback to guide study efforts and prime conference-related learning.
Identifiants
pubmed: 32176016
doi: 10.1097/SCS.0000000000006353
pii: 00001665-202006000-00006
doi:
Types de publication
Journal Article
Langues
eng
Pagination
904-907Références
Coelho G, Warf B, Lyra M, et al. Anatomical pediatric model for craniosynostosis surgical training. Childs Nerv Syst 2014; 30:2009–2014.
Chipman, Jeffrey G, Schmitz CC. Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents. J Am Coll Surg 2009; 209: 370.e2.
Golriz M, Hafezi M, Garoussi C, et al. Do we need animal hands-on courses for transplantation surgery? Clin Transplant 2013; 27:6–15.
Rehim SA, Chung KC. Educational video recording and editing for the hand surgeon. J Hand Surg 2015; 40:1048–1054.
Sharma G, Aycart MA, Najjar PA, et al. A cadaveric procedural anatomy course enhances operative competence. J Surg Res 2016; 201:22–28.
Schwamborn A, Mayer RE, Thillmann H, et al. Drawing as a generative activity and drawing as a prognostic activity. J Educ Psychol 2010; 102:872–879.
Nugraha I. The use of drawing as an alternative assessment tool in biology teaching. Journal of Physics: Conference Series, 1013 012016. 2018.
Ann Van Heest, Matthew Putnam, Julie Agel, et al. Assessment of technical skills of orthopaedic surgery residents performing open carpal tunnel release surgery. J Bone Joint Surg 2009; 91:2811–2817.
Agnew SP, Dumanian GA. A simple test: Please draw the incision. it is not as easy as we may think. J Hand Surg 2008; 33:1894–1898.
Wong S, Weber R. Utilizing Drawing to Evaluate Ability to Perform a Tendon Repair. American Association of Plastic Surgeons 98th Annual Meeting. April 2019.
Carpenter SK. Semantic information activated during retrieval contributes to later retention: support for the mediator effectiveness hypothesis of the testing effect. J Exp Psychol Learn Mem Cogn 2011; 37:1547–1552.
Carpenter S, Pashler H, Vul E. What types of learning are enhanced by a cued recall test? Psychon Bull Rev 2006; 13:826–830.
Carpenter S, Pashler H, Wixted J, et al. The effects of tests on learning and forgetting. Mem Cognit 2008; 36:438–448.
Chan JCK, McDermott KB. Retrieval-induced facilitation. J Exp Psychol Gen 2006; 135:553–571.
Hays MJ, Kornell N, Bjork RA. When and why a failed test potentiates the effectiveness of subsequent study. J Exp Psychol Learn Mem Cogn 2013; 39:290–296.
Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci 2006; 17:249–255.
Karpicke JD, Roediger HL III. The critical importance of retrieval for learning. Science 2008; 319:966–968.
Kornell N. Attempting to answer a meaningful question enhances subsequent learning even when feedback is delayed. J Exp Psychol Learn Mem Cogn 2014; 40:106–114.
Kornell N, Hays MJ, Bjork RA. Unsuccessful retrieval attempts enhance subsequent learning. J Exp Psychol Learn Mem Cogn 2009; 35:989–998.
Kornell N, Klein PJ, Rawson KA. Retrieval attempts enhance learning, but retrieval success (versus failure) does not matter. J Exp Psychol Learn Mem Cogn 2015; 41:283–294.
Mcdaniel M, Roediger H, Mcdermott K. Generalizing test-enhanced learning from the laboratory to the classroom. Psychon Bull Rev 2007; 14:200–206.
Richland LE, Kornell N, Kao LS. The pretesting effect. J Exp Psychol Appl 2009; 15:243–257.
Cull WL. Untangling the benefits of multiple study opportunities and repeated testing for cued recall. Appl Cogn Psychol 2000; 14:215–235.