Utility of digitally supported surgical competency assessments in a work-based setting: a systematic review of the literature.
competency-based education
curriculum
education
operative procedures
surgery
Journal
ANZ journal of surgery
ISSN: 1445-2197
Titre abrégé: ANZ J Surg
Pays: Australia
ID NLM: 101086634
Informations de publication
Date de publication:
06 2020
06 2020
Historique:
received:
17
03
2019
revised:
15
07
2019
accepted:
18
08
2019
pubmed:
8
12
2019
medline:
30
4
2021
entrez:
8
12
2019
Statut:
ppublish
Résumé
Adequate training and assessment of competency in surgical trainees are important in minimizing adverse events and achieving good patient outcomes. The purpose was to investigate the current literature on digitally supported methods of facilitating work-based competency assessment of surgical trainees. Two concurrent literature searches were performed by two independent researchers using three databases: MEDLINE, Embase and Education Resources Information Center. Eligible studies reported on digitally supported tools for assessing surgical competency in a work-based setting for interns, residents or trainees of a post-graduate surgical programme. Studies focusing primarily on training courses, simulations or other tools for assessing surgical skills in a setting outside of work were excluded. Articles published as abstracts only, articles not published in English and review articles that did not contain original data were excluded. A total of 11 eligible studies were included, with six of these implementing smartphone application-based programmes, and five utilizing web-based programmes. Five studies implemented the 'System for Improving and Measuring Procedural Learning' smartphone application. Studies were based predominantly on general surgery residents, and were limited to the North American context. There was significant variability between studies regarding methodology, including the scoring system used to assess competency. This review confirms that digitally supported competency assessment of surgical trainees in a work-based setting is both feasible and effective. Digital platforms allow evaluations to be performed in a timely fashion without significant disturbance to workflow.
Sections du résumé
BACKGROUND
Adequate training and assessment of competency in surgical trainees are important in minimizing adverse events and achieving good patient outcomes. The purpose was to investigate the current literature on digitally supported methods of facilitating work-based competency assessment of surgical trainees.
METHODS
Two concurrent literature searches were performed by two independent researchers using three databases: MEDLINE, Embase and Education Resources Information Center. Eligible studies reported on digitally supported tools for assessing surgical competency in a work-based setting for interns, residents or trainees of a post-graduate surgical programme. Studies focusing primarily on training courses, simulations or other tools for assessing surgical skills in a setting outside of work were excluded. Articles published as abstracts only, articles not published in English and review articles that did not contain original data were excluded.
RESULTS
A total of 11 eligible studies were included, with six of these implementing smartphone application-based programmes, and five utilizing web-based programmes. Five studies implemented the 'System for Improving and Measuring Procedural Learning' smartphone application. Studies were based predominantly on general surgery residents, and were limited to the North American context. There was significant variability between studies regarding methodology, including the scoring system used to assess competency.
CONCLUSION
This review confirms that digitally supported competency assessment of surgical trainees in a work-based setting is both feasible and effective. Digital platforms allow evaluations to be performed in a timely fashion without significant disturbance to workflow.
Types de publication
Journal Article
Review
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
970-977Informations de copyright
© 2019 Royal Australasian College of Surgeons.
Références
Australian Institute of Health and Welfare. Australian Hospitals 2015-2016 at a Glance. Health services series no. 77. Canberra: Australian Institute of Health and Welfare, 2017.
Vedula SS, Ishii M, Hager GD. Objective assessment of surgical technical skill and competency in the operating room. Annu. Rev. Biomed. Eng. 2017; 19: 301-25.
Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian health care study. Med. J. Aust. 1995; 163: 458-71.
Kable AK, Gibberd RW, Spigelman AD. Adverse events in surgical patients in Australia. International J. Qual. Health Care 2002; 14: 269-76.
van der Leeuw RM, Lombarts KM, Arah OA, Heineman MJ. A systematic review of the effects of residency training on patient outcomes. BMC Med. 2012; 10: 65.
Lutz K, Yazdani A, Ross D. From time-based to competency-based standards: core transitional competencies in plastic surgery. J. Surg. Educ. 2015; 72: 228-34.
De Siqueira JR, Gough MJ. Correlation between experience targets and competence for general surgery certification. Br. J. Surg. 2016; 103: 921-7.
Hoffmann H, Oertli D, Mechera R et al. Comparison of Canadian and Swiss surgical training curricula: moving on toward competency-based surgical education. J. Surg. Educ. 2017; 74: 37-46.
Szasz P, Louridas M, Harris KA, Aggarwal R, Grantcharov TP. Assessing technical competence in surgical trainees: a systematic review. Ann. Surg. 2015; 261: 1046-55.
Frank JR, Snell LS, Cate OT et al. Competency-based medical education: theory to practice. Med. Teach. 2010; 32: 638-45.
Brightwell A, Grant J. Competency-based training: who benefits? Postgrad. Med. J. 2012; 89: 107-10.
Leung W-C. Competency based medical training: review. Br. Med. J. 2002; 325: 693-6.
Martin JA, Regehr G, Reznick R et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br. J. Surg. 1997; 84: 273-8.
Larson JL, Williams RG, Ketchum J, Boehler ML, Dunnington GL. Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents. Surgery 2005; 138: 640-9.
Liberati A, Altman DG, Tetzlaff J et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. Br. Med. J. 2009; 339: b2700.
George BC, Bohnen JD, Williams RG et al. Readiness of US general surgery residents for independent practice. Ann. Surg. 2017; 266: 582-94.
Hartranft TH, Yandle K, Graham T, Holden C, Chambers LW. Evaluating surgical residents quickly and easily against the milestones using electronic formative feedback. J. Surg. Educ. 2017; 74: 237-42.
Green JM. An innovative, no-cost, evidence-based smartphone platform for resident evaluation. J. Surg. Educ. 2016; 73: e14-e8.
Kozin ED, Bohnen JD, George BC et al. Novel mobile app allows for fast and validated intraoperative assessment of otolaryngology residents. OTO Open 2017; 1: 2473974X1668570.
Wohaibi EM, Earle DB, Ansanitis FE, Wait RB, Fernandez G, Seymour NE. A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance. J. Surg. Educ. 2007; 64: 333-41.
Kaban LB, Cappetta A, George BC, Lahey ET, Bohnen JD, Troulis MJ. Evaluation of oral and maxillofacial surgery residents' operative skills: feasibility and engagement study using SIMPL software for a mobile phone. J. Oral Maxillofac. Surg. 2017; 75: 2041-7.
Kobraei EM, Bohnen JD, George BC et al. Uniting evidence-based evaluation with the ACGME plastic surgery milestones: a simple and reliable assessment of resident operative performance. Plast. Reconstr. Surg. 2016; 138: 349e-57e.
Wagner JP, Chen DC, Donahue TR et al. Assessment of resident operative performance using a real-time mobile web system: preparing for the milestone age. J. Surg. Educ. 2014; 71: e41-e6.
Bohnen JD, George BC, Williams RG et al. The feasibility of real-time intraoperative performance assessment with SIMPL (system for improving and measuring procedural learning): early experience from a multi-institutional trial. J. Surg. Educ. 2016; 73: e118-e30.
Nousiainen MT, Mironova P, Hynes M et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med. Teach. 2018; 40: 1042-54.
George BC, Teitelbaum EN, Meyerson SL et al. Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance. J. Surg. Educ. 2014; 71: e90-e6.
Wozniak H, Ellaway RH, de Jong PG. What have we learnt about using digital technologies in health professional education? Med. J. Aust. 2018; 209: 431-3.e1.
Nathwani JN, Glarner CE, Law KE et al. Integrating postoperative feedback into workflow: perceived practices and barriers. J. Surg. Educ. 2017; 74: 406-14.
Swanwick T. Understanding Medical Education: Evidence, Theory and Practice, 2nd edn. Hoboken: Wiley-Blackwell, 2013.
Wozniak H. Conjecture mapping to optimize the design-based research process. Australas. J. Educ. Technol. 2015; 31: 597-612.
Gaunt A, Patel A, Royle J et al. What do surgeons and trainees think of WBAs and how do they use them? Bull. Roy. Coll. Surg. Engl. 2016; 98: 408-14.
Gaunt A, Patel A, Fallis S et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad. Med. 2017; 92: 827-34.
Szasz P, Louridas M, Harris KA, Grantcharov TP. Strategies for increasing the feasibility of performance assessments during competency-based education: subjective and objective evaluations correlate in the operating room. Am. J. Surg. 2017; 214: 365-72.
Williams RG, Verhulst S, Colliver JA, Sanfey H, Chen X, Dunnington GL. A template for reliable assessment of resident operative performance: assessment intervals, numbers of cases and raters. Surgery 2012; 152: 517-24.
Niitsu H, Hirabayashi N, Yoshimitsu M et al. Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg. Today 2013; 43: 271-5.