Medical school grades may predict future clinical competence.
Journal
Journal of the Chinese Medical Association : JCMA
ISSN: 1728-7731
Titre abrégé: J Chin Med Assoc
Pays: Netherlands
ID NLM: 101174817
Informations de publication
Date de publication:
01 09 2022
01 09 2022
Historique:
entrez:
23
9
2022
pubmed:
24
9
2022
medline:
28
9
2022
Statut:
ppublish
Résumé
In real-world medical education, there is a lack of reliable predictors of future clinical competencies. Hence, we aim to identify the factors associated with clinical competencies and construct a prediction model to identify "improvement required" trainees. We analyzed data from medical students who graduated from National Yang-Ming University with clerkship training and participated in the postgraduate year (PGY) interview at Taipei Veterans General Hospital. Clinical competencies were evaluated using grades of national objective structured clinical examination (OSCEs). This study used data from medical students who graduated in July 2018 as the derivation cohort (N = 50) and those who graduated in July 2020 (n = 56) for validation. Medical school grades were associated with the performance of national OSCEs (Pearson r = 0.34, p = 0.017), but the grades of the structured PGY interviews were marginally associated with the national OSCE (Pearson r = 0.268, p = 0.06). A prediction model was constructed to identify "improvement required" trainees, defined: trainees with the lowest 25% of scores in the national OSCEs. According to this model, trainees with the lowest 25% medical school grades predicted a higher risk of the "improvement required" clinical performance (Q1-Q3 vs Q4 = 15% vs 60%, odds ratio = 8.5 [95% confidence interval = 1.8-39.4], p = 0.029). In the validation cohort, our prediction model could accurately classify 76.7% "improvement required" and "nonimprovement required" students. Our study suggests that interventions for students with unsatisfactory medical school grades are warranted to improve their clinical competencies.
Sections du résumé
BACKGROUND
In real-world medical education, there is a lack of reliable predictors of future clinical competencies. Hence, we aim to identify the factors associated with clinical competencies and construct a prediction model to identify "improvement required" trainees.
METHODS
We analyzed data from medical students who graduated from National Yang-Ming University with clerkship training and participated in the postgraduate year (PGY) interview at Taipei Veterans General Hospital. Clinical competencies were evaluated using grades of national objective structured clinical examination (OSCEs). This study used data from medical students who graduated in July 2018 as the derivation cohort (N = 50) and those who graduated in July 2020 (n = 56) for validation.
RESULTS
Medical school grades were associated with the performance of national OSCEs (Pearson r = 0.34, p = 0.017), but the grades of the structured PGY interviews were marginally associated with the national OSCE (Pearson r = 0.268, p = 0.06). A prediction model was constructed to identify "improvement required" trainees, defined: trainees with the lowest 25% of scores in the national OSCEs. According to this model, trainees with the lowest 25% medical school grades predicted a higher risk of the "improvement required" clinical performance (Q1-Q3 vs Q4 = 15% vs 60%, odds ratio = 8.5 [95% confidence interval = 1.8-39.4], p = 0.029). In the validation cohort, our prediction model could accurately classify 76.7% "improvement required" and "nonimprovement required" students.
CONCLUSION
Our study suggests that interventions for students with unsatisfactory medical school grades are warranted to improve their clinical competencies.
Identifiants
pubmed: 36150103
doi: 10.1097/JCMA.0000000000000782
pii: 02118582-202209000-00006
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
909-914Informations de copyright
Copyright © 2022, the Chinese Medical Association.
Déclaration de conflit d'intérêts
Conflicts of interest: Dr. Ming-Chih Hou and Dr. Wayne Huey-Herng Sheu, editorial board members at Journal of the Chinese Medical Association, have no roles in the peer review process of or decision to publish this article. The other authors declare that they have no conflicts of interest related to the subject matter or materials discussed in this article.
Références
Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the interview in resident candidate selection: a review of the literature. J Grad Med Educ. 2015;7:539–48.
Fleming AE, Smith S. Mistreatment of medical trainees: time for a new approach. JAMA Netw Open. 2018;1:e180869.
Andreassen P, Malling B. How are formative assessment methods used in the clinical setting? A qualitative study. Int J Med Educ. 2019;10:208–15.
Gonnella JS, Hojat M. Relationship between performance in medical school and postgraduate competence. J Med Educ. 1983;58:679–85.
Wingard JR, Williamson JW. Grades as predictors of physicians’ career performance: an evaluative literature review. J Med Educ. 1973;48:311–22.
Taylor CW, Albo D Jr. Measuring and predicting the performances of practicing physicians: an overview of two decades of research at the University of Utah. Acad Med. 1993;68(2 Suppl):S65–7.
Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctor performance: a meta-analysis. Med Educ. 2013;47:790–800.
Krupat E, Pelletier SR, Dienstag JL. Academic performance on first-year medical school exams: how well des it predict later performance on knowledge-based and clinical assessments? Teach Learn Med. 2017;29:181–7.
Brothers TE, Wetherholt S. Importance of the faculty interview during the resident application process. J Surg Educ. 2007;64:378–85.
Huang CC, Chan CY, Wu CL, Chen YL, Yang HW, Huang CC, et al. Assessment of clinical competence of medical students using the objective structured clinical examination: first 2 years’ experience in Taipei Veterans General Hospital. J Chin Med Assoc. 2010;73:589–95.
Chong L, Taylor S, Haywood M, Adelstein BA, Shulruf B. The sights and insights of examiners in objective structured clinical examinations. J Educ Eval Health Prof. 2017;14:34.
Chang CC, Lirng JF, Wang PN, Wang SJ, Chen CH, Yang LY, et al. A pilot study of integrating standardized patients in problem-based learning tutorial in Taiwan. J Chin Med Assoc. 2019;82:464–8.
Wood TJ, Humphrey-Murto SM, Norman GR. Standard setting in a small scale OSCE: a comparison of the Modified Borderline-Group Method and the Borderline Regression Method. Adv Health Sci Educ Theory Pract. 2006;11:115–22.
Hejri SM, Jalili M, Muijtjens AM, Van Der Vleuten CP. Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination. J Res Med Sci. 2013;18:887–91.
Homer M, Pell G. The impact of the inclusion of simulated patient ratings on the reliability of OSCE assessments under the borderline regression method. Med Teach. 2009;31:420–5.
Yousuf N, Violato C, Zuberi RW. Standard setting methods for pass/fail decisions on high-stakes objective structured clinical examinations: a validity study. Teach Learn Med. 2015;27:280–91.
Norcini JJ. Setting standards on educational tests. Med Educ. 2003;37:464–9.
Dwivedi NR, Vijayashankar NP, Hansda M, Dubey AK, Nwachukwu F, Curran V, et al. Comparing standard setting methods for objective structured clinical examinations in a caribbean medical school. J Med Educ Curric Dev. 2020;7:2382120520981992.
Kramer A, Muijtjens A, Jansen K, Düsman H, Tan L, Van Der Vleuten C. Comparison of a rational and an empirical standard setting procedure for an OSCE. Med Educ. 2003;37:132–9.
Shulruf B, Turner R, Poole P, Wilkinson T. The Objective Borderline method (OBM): a probability-based model for setting up an objective pass/fail cut-off score in medical programme assessments. Adv Health Sci Educ Theory Pract. 2013;18:231–44.
Kass GV. An exploratory technique for investigating large quantities of categorical data. J R Stat Soc Ser C Appl Stat. 1980;29:119–27.
Ozuah PO. Predicting residents’ performance: a prospective study. BMC Med Educ. 2002;2:7.
Shiroma PR, Alarcon RD. Selection factors among international medical graduates and psychiatric residency performance. Acad Psychiatry. 2010;34:128–31.
Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009;43:767–75.
Grewal SG, Yeung LS, Brandes SB. Predictors of success in a urology residency program. J Surg Educ. 2013;70:138–43.
Olawaiye A, Yeh J, Withiam-Leitch M. Resident selection process and prediction of clinical performance in an obstetrics and gynecology program. Teach Learn Med. 2006;18:310–5.
Wood PS, Smith WL, Altmaier EM, Tarico VS, Franken EA Jr. A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol. 1990;25:855–9.
Fryer JP, Corcoran N, George B, Wang E, Darosa D. Does resident ranking during recruitment accurately predict subsequent performance as a surgical resident? J Surg Educ. 2012;69:724–30.
George JM, Young D, Metz EN. Evaluating selected internship candidates and their subsequent performances. Acad Med. 1989;64:480–2.
Oldfield Z, Beasley SW, Smith J, Anthony A, Watt A. Correlation of selection scores with subsequent assessment scores during surgical training. ANZ J Surg. 2013;83:412–6.
LaGrasso JR, Kennedy DA, Hoehn JG, Ashruf S, Przybyla AM. Selection criteria for the integrated model of plastic surgery residency. Plast Reconstr Surg. 2008;121:121e–5e.
Wagoner NE, Suriano JR, Stoner JA. Factors used by program directors to select residents. J Med Educ. 1986;61:10–21.
Al Khalili K, Chalouhi N, Tjoumakaris S, Gonzalez LF, Starke RM, Rosenwasser R, et al. Programs selection criteria for neurological surgery applicants in the United States: a national survey for neurological surgery program directors. World Neurosurg. 2014;81:473–7.e2.
Parker AM, Petroze RT, Schirmer BD, Calland JF. Surgical residency market research-what are applicants looking for? J Surg Educ. 2013;70:232–6.
Otero HJ, Erturk SM, Ondategui-Parra S, Ros PR. Key criteria for selection of radiology residents: results of a national survey. Acad Radiol. 2006;13:1155–64.
Agahi F, Speicher MR, Cisek G. Association between undergraduate performance predictors and academic and clinical performance of osteopathic medical students. J Am Osteopath Assoc. 2018;118:106–14.
Gadbury-Amyot CC, Bray KK, Branson BS, Holt L, Keselyak N, Mitchell TV, et al. Predictive validity of dental hygiene competency assessment measures on one-shot clinical licensure examinations. J Dent Educ. 2005;69:363–70.
Vink SC, Van Tartwijk J, Bolk J, Verloop N. Integration of clinical and basic sciences in concept maps: a mixed-method study on teacher learning. BMC Med Educ. 2015;15:20.
Hale JF, Cahan MA, Zanetti ML. Integration of basic clinical skills training in medical education: an interprofessional simulated teaching experience. Teach Learn Med. 2011;23:278–84.
Jünger J, Schäfer S, Roth C, Schellberg D, Friedman Ben-David M, Nikendei C. Effects of basic clinical skills training on objective structured clinical examination performance. Med Educ. 2005;39:1015–20.
Remmen R, Scherpbier A, van der Vleuten C, Denekens J, Derese A, Hermann I, et al. Effectiveness of basic clinical skills training programmes: a cross-sectional comparison of four medical schools. Med Educ. 2001;35:121–8.
Yang YY, Wang SJ, Yang LY, Lirng JF, Huang CC, Liang JF, et al. Effects of a new parallel primary healthcare centre and on-campus training programme on history taking, physical examination skills and medical students’ preparedness: a prospective comparative study in Taiwan. BMJ Open. 2017;7:e016294.
Kumar N, Bhardwaj S, Rahman E. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom: a cohort study. Adv Med Educ Pract. 2018;9:239–45.