Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment.
Assessment
Competency-based medical education
Feedback
Postgraduate medical education
Journal
Perspectives on medical education
ISSN: 2212-277X
Titre abrégé: Perspect Med Educ
Pays: Netherlands
ID NLM: 101590643
Informations de publication
Date de publication:
08 2020
08 2020
Historique:
pubmed:
7
6
2020
medline:
5
5
2021
entrez:
7
6
2020
Statut:
ppublish
Résumé
Competency-based medical education (CBME) hinges on robust assessment. However, integrating regular workplace-based assessment within demanding and sometimes chaotic clinical environments remains challenging. Many faculty lack assessment expertise, and some programs lack the infrastructure and faculty numbers to fulfill CBME's mandate. Recognizing this, we designed and implemented an assessment innovation that trains and deploys a cadre of faculty to assess in specialties outside their own. Specifically, we explored trainees' perceptions of and receptiveness to this novel assessment approach. Within Western University's Surgical Foundations program, 27 PGY‑1 trainees were formatively assessed by trained non-surgeons on a basic laparoscopic surgical skill. These assessments did not impact trainees' progression. Four focus groups were conducted to gauge residents' sentiments about the experience of cross-specialty assessment. Data were then analyzed using a thematic analysis approach. While a few trainees found the experience motivating, more often trainees questioned the feedback they received and the practicality of this assessment approach to advance their procedural skill acquisition. What trainees wanted were strategies for improvement, not merely an assessment of performance. Trainees' trepidation at the idea of using outside assessors to meet increased assessment demands appeared grounded in their expectations for assessment. What trainees appeared to desire was a coach-someone who could break their performance into its critical individual components-as opposed to an assessor whose role was limited to scoring their performance. Understanding trainees' receptivity to new assessment approaches is crucial; otherwise training programs run the risk of generating more assessments without added value.
Identifiants
pubmed: 32504448
doi: 10.1007/s40037-020-00594-0
pii: 10.1007/s40037-020-00594-0
pmc: PMC7459015
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
201-209Commentaires et corrections
Type : CommentIn
Références
Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676–82.
doi: 10.3109/0142159X.2010.500704
Frank JR, Snell LS, Ten Cate O, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45.
doi: 10.3109/0142159X.2010.501190
Gruppen LD, Ten Cate O, Lingard LA, Teunissen PW, Kogan JR. Enhanced requirements for assessment in a competency-based, time-variable medical education system. Acad Med. 2018;93(3S):S17–S21.
doi: 10.1097/ACM.0000000000002066
Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: when theory meets practice. BMC Med Educ. 2013;13(1):123.
doi: 10.1186/1472-6920-13-123
Driessen E, Scheele F. What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research. Med Teach. 2013;35(7):569–74.
doi: 10.3109/0142159X.2013.798403
Caverzagie KJ, Nousiainen MT, Ferguson PC, et al. Overarching challenges to the implementation of competency-based medical education. Med Teach. 2017;39(6):588–93.
doi: 10.1080/0142159X.2017.1315075
Watling C, LaDonna KA, Lingard L, Voyer S, Hatala R. “Sometimes the work just needs to be done”: socio-cultural influences on direct observation in medical training. Med Educ. 2016;50(10):1054–64.
doi: 10.1111/medu.13062
Pulito AR, Donnelly MB, Plymale M, Mentzer JRM. What do faculty observe of medical students’ clinical performance? Teach Learn Med. 2006;18(2):99–104.
doi: 10.1207/s15328015tlm1802_2
Howley LD, Wilson WG. Direct observation of students during clerkship rotations: a multiyear descriptive study. Acad Med. 2004;79(3):276–80.
doi: 10.1097/00001888-200403000-00017
Holmboe ES. Faculty and the observation of trainees’ clinical skills: problems and opportunities. Acad Med. 2004;79(1):16–22.
doi: 10.1097/00001888-200401000-00006
Burdick WP, Schoffstall J. Observation of emergency medicine residents at the bedside: How often does it happen? Acad Emerg Med. 1995;2(10):909–13.
doi: 10.1111/j.1553-2712.1995.tb03108.x
Holmboe ES, Ward DS, Reznick RK, et al. Faculty development in assessment: the missing link in competency-based medical education. Acad Med. 2011;86(4):460–7.
doi: 10.1097/ACM.0b013e31820cb2a7
Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Theory Pract. 2016;21(2):455–73.
doi: 10.1007/s10459-015-9614-0
Watling C, Driessen E, van der Vleuten CPM, Lingard L. Learning culture and feedback: an international study of medical athletes and musicians. Med Educ. 2014;48(7):713–23.
doi: 10.1111/medu.12407
Harrison CJ, Könings KD, Dannefer EF, Schuwirth LWT, Wass V, van der Vleuten CPM. Factors influencing students’ receptivity to formative feedback emerging from different assessment cultures. Perspect Med Educ. 2016;5(5):276–84.
doi: 10.1007/s40037-016-0297-x
Voyer S, Cuncic C, Butler DL, MacNeil K, Watling C, Hatala R. Investigating conditions for meaningful feedback in the context of an evidence-based feedback programme. Med Educ. 2016;50(9):943–54.
doi: 10.1111/medu.13067
MacNeil K, Cuncic C, Voyer S, Butler D, Hatala R. Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents’ clinical education. Adv Health Sci Educ Theory Pract. 2019; https://doi.org/10.1007/s10459-019-09948-8 .
doi: 10.1007/s10459-019-09948-8
Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the ‘black box’ differently: assessor cognition from three research perspectives. Med Educ. 2014;48(11):1055–68.
doi: 10.1111/medu.12546
Hoang NS, Lau JN. A call for mixed methods in competency-based medical education: how we can prevent the overfitting of curriculum and assessment. Acad Med. 2018;93(7):996–1001.
doi: 10.1097/ACM.0000000000002205
Lockyer J, Carraccio C, Chan M‑K, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609–16.
doi: 10.1080/0142159X.2017.1315082
Burm S, Sebok-Syer SS, Lingard L, et al. “You want me to assess what?”: Faculty perceptions of assessing residents from outside their specialty. Acad Med. 2019;94(10):1478–82.
doi: 10.1097/ACM.0000000000002771
Mahmood O, Dagnæs J, Bube S, Rohrsted M, Konge L. Nonspecialist raters can provide reliable assessments of procedural skills. J Surg Educ. 2018;75(2):370–6.
doi: 10.1016/j.jsurg.2017.07.003
Kim MJ, Boehler ML, Ketchum JK, Bueno R Jr, Williams RG, Dunnington GL. Skills coaches as part of the educational team: a randomized controlled trial of teaching of a basic surgical skill in the laboratory setting. Am J Surg. 2010;199(1):94–8.
doi: 10.1016/j.amjsurg.2009.08.016
Musits AN, Phrampus PE, Lutz JW, et al. Physician versus nonphysician instruction: evaluating an expert curriculum-competent facilitator model for simulation-based central venous catheter training. Simul Healthc. 2019;14(4):228–34.
doi: 10.1097/SIH.0000000000000374
Creswell JW. Qualitative inquiry and research design: choosing among five approaches. Thousand Oaks: SAGE; 2007.
Peters JH, Fried GM, Swanstrom LL, et al. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004;135(1):21–7.
doi: 10.1016/S0039-6060(03)00156-9
Chang OH, King LP, Modest AM, Hur HC. Developing an objective structured assessment of technical skills for laparoscopic suturing and intracorporeal knot tying. J Surg Educ. 2016;73(2):258–63.
doi: 10.1016/j.jsurg.2015.10.006
Kitzinger J. The methodology of focus groups: the importance of interaction between research participants. Sociol Health Illn. 1994;16(1):103–21.
doi: 10.1111/1467-9566.ep11347023
Patton MQ. Qualitative evaluation and research methods. Newbury Park: SAGE; 1990.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
doi: 10.1191/1478088706qp063oa
Bindal T, Wall D, Goodyear HM. Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach. 2011;33(11):919–27.
doi: 10.3109/0142159X.2011.558140
Nesbitt A, Baird F, Canning B, Griffin A, Sturrock A. Student perception of workplace-based assessment. Clin Teach. 2013;10(6):399–404.
doi: 10.1111/tct.12057
Cleland J, Reeve J, Rosenthal J, Johnston P. Resisting the tick box culture: refocusing medical education and training. Br J Gen Pract. 2014;64(625):422–3.
doi: 10.3399/bjgp14X681169
Ross S, Poth C‑A, Donoff MG, et al. Involving users in the refinement of the competency-based achievement system: an innovative approach to competency-based assessment. Med Teach. 2012;34(2):e143–e7.
doi: 10.3109/0142159X.2012.644828
Torre DM, Schuwirth LWT, Van der Vleuten CPM. Theoretical considerations on programmatic assessment. Med Teach. 2020;42(2):213–20.
doi: 10.1080/0142159X.2019.1672863
van der Vleuten CPM, Schuwirth LWT, Driessen EW, Dijkstra J, Tigelaar D, Baartman LKJ, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205–14.
doi: 10.3109/0142159X.2012.652239
Hoff TJ, Pohl H, Bartfield J. Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004;79(6):532–9.
doi: 10.1097/00001888-200406000-00007
Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76–85.
doi: 10.1111/medu.13645
Scott IM. Beyond “driving”: the relationship between assessment, performance and learning. Med Educ. 2020;54(1):54–9.
doi: 10.1111/medu.13935