How can surgical skills in laparoscopic colon surgery be objectively assessed?-a scoping review.
Assessment tool
Colon surgery
Competency
Laparoscopy
Surgical education
Technical skills
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
03 2022
03 2022
Historique:
received:
31
07
2021
accepted:
21
11
2021
pubmed:
8
12
2021
medline:
4
3
2022
entrez:
7
12
2021
Statut:
ppublish
Résumé
In laparoscopic colorectal surgery, higher technical skills have been associated with improved patient outcome. With the growing interest in laparoscopic techniques, pressure on surgeons and certifying bodies is mounting to ensure that operative procedures are performed safely and efficiently. The aim of the present review was to comprehensively identify tools for skill assessment in laparoscopic colon surgery and to assess their validity as reported in the literature. A systematic search was conducted in EMBASE and PubMed/MEDLINE in May 2021 to identify studies examining technical skills assessment tools in laparoscopic colon surgery. Available information on validity evidence (content, response process, internal structure, relation to other variables, and consequences) was evaluated for all included tools. Fourteen assessment tools were identified, of which most were procedure-specific and video-based. Most tools reported moderate validity evidence. Commonly not reported were rater training, assessment correlation with variables other than training level, and validity reproducibility and reliability in external educational settings. The results of this review show that several tools are available for evaluation of laparoscopic colon cancer surgery, but few authors present substantial validity for tool development and use. As we move towards the implementation of new techniques in laparoscopic colon surgery, it is imperative to establish validity before surgical skill assessment tools can be applied to new procedures and settings. Therefore, future studies ought to examine different aspects of tool validity, especially correlation with other variables, such as patient morbidity and pathological reports, which impact patient survival.
Sections du résumé
BACKGROUND
In laparoscopic colorectal surgery, higher technical skills have been associated with improved patient outcome. With the growing interest in laparoscopic techniques, pressure on surgeons and certifying bodies is mounting to ensure that operative procedures are performed safely and efficiently. The aim of the present review was to comprehensively identify tools for skill assessment in laparoscopic colon surgery and to assess their validity as reported in the literature.
METHODS
A systematic search was conducted in EMBASE and PubMed/MEDLINE in May 2021 to identify studies examining technical skills assessment tools in laparoscopic colon surgery. Available information on validity evidence (content, response process, internal structure, relation to other variables, and consequences) was evaluated for all included tools.
RESULTS
Fourteen assessment tools were identified, of which most were procedure-specific and video-based. Most tools reported moderate validity evidence. Commonly not reported were rater training, assessment correlation with variables other than training level, and validity reproducibility and reliability in external educational settings.
CONCLUSION
The results of this review show that several tools are available for evaluation of laparoscopic colon cancer surgery, but few authors present substantial validity for tool development and use. As we move towards the implementation of new techniques in laparoscopic colon surgery, it is imperative to establish validity before surgical skill assessment tools can be applied to new procedures and settings. Therefore, future studies ought to examine different aspects of tool validity, especially correlation with other variables, such as patient morbidity and pathological reports, which impact patient survival.
Identifiants
pubmed: 34873653
doi: 10.1007/s00464-021-08914-z
pii: 10.1007/s00464-021-08914-z
pmc: PMC8847271
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
1761-1774Informations de copyright
© 2021. The Author(s).
Références
Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon M, Khan Z, Longman R, Coleman MG, Hanna GB (2015) Clinical validity of consultant technical skills assessment in the english national training programme for laparoscopic colorectal surgery. Br J Surg 102:991–997
pubmed: 25994456
Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SE, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg 155:960–968
pubmed: 32838425
pmcid: 7439214
Curtis NJ, Foster JD, Miskovic D, Brown CSB, Hewett PJ, Abbott S, Hanna GB, Stevenson ARL, Francis NK (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155:590–598
pubmed: 32374371
Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987
pubmed: 20632260
Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113
pubmed: 15972181
Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278
pubmed: 9052454
Palter VN, Grantcharov TP (2012) A prospective study demonstrating the reliability and validity of two procedure-specific evaluation tools to assess operative competence in laparoscopic colorectal surgery. Surg Endosc 26:2489–2503
pubmed: 22476826
Miskovic D, Ni M, Wyles SM, Parvaiz A, Hanna GB (2012) Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surg Endosc 26:796–803
pubmed: 22042584
Miskovic D, Ni M, Wyles SM, Tekkis P, Hanna GB (2012) Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum 55:1300–1310
pubmed: 23135590
Quirke P, Steele R, Monson J, Grieve R, Khanna S, Couture J, O’Callaghan C, Myint AS, Bessell E, Thompson LC, Parmar M, Stephens RJ, Sebag-Montefiore D (2009) Effect of the plane of surgery achieved on local recurrence in patients with operable rectal cancer: a prospective study using data from the MRC CR07 and NCIC-CTG CO16 randomised clinical trial. Lancet (London, England) 373:821–828
Kitz J, Fokas E, Beissbarth T, Ströbel P, Wittekind C, Hartmann A, Rüschoff J, Papadopoulos T, Rösler E, Ortloff-Kittredge P, Kania U, Schlitt H, Link KH, Bechstein W, Raab HR, Staib L, Germer CT, Liersch T, Sauer R, Rödel C, Ghadimi M, Hohenberger W (2018) Association of Plane of Total Mesorectal Excision With Prognosis of Rectal Cancer: Secondary Analysis of the CAO/ARO/AIO-04 Phase 3 Randomized Clinical Trial. JAMA Surg 153:e181607
pubmed: 29874375
pmcid: 29874375
West NP, Morris EJ, Rotimi O, Cairns A, Finan PJ, Quirke P (2008) Pathology grading of colon cancer surgical resection and its association with survival: a retrospective observational study. Lancet Oncol 9:857–865
pubmed: 18667357
Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell TM (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262
pubmed: 24424150
American Educational Research Association APA, National Council on Measurement in Education (2002) Standards for educational and psychological testing, 1.0, American Educational Research Association, Washington, DC
Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, Macdonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, Tunçalp Ö, Straus SE PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation.
Beckman TJ, Cook DA, Mandrekar JN (2005) What is the validity evidence for assessments of clinical teaching? J Gen Intern Med 20:1159–1164
pubmed: 16423109
pmcid: 1490286
Sidhu RS, Vikis E, Cheifetz R, Phang T (2006) Self-assessment during a 2-day laparoscopic colectomy course: can surgeons judge how well they are learning new skills? Am J Surg 191:677–681
pubmed: 16647359
Wohaibi EM, Earle DB, Ansanitis FE, Wait RB, Fernandez G, Seymour NE (2007) A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance. J Surg Educ 64:333–341
pubmed: 18063265
Jenkins JT, Currie A, Sala S, Kennedy RH (2016) A multi-modal approach to training in laparoscopic colorectal surgery accelerates proficiency gain. Surg Endosc 30:3007–3013
pubmed: 26487223
Watanabe Y, Madani A, Ito YM, Bilgic E, McKendy KM, Feldman LS, Fried GM, Vassiliou MC (2017) Psychometric properties of the global operative assessment of laparoscopic skills (GOALS) using item response theory. Am J Surg 213:273–276
pubmed: 27776756
Sarker SK, Delaney C (2011) Feasibility of self-appraisal in assessing operative performance in advanced laparoscopic colorectal surgery. Colorectal Disease : the official J Association of Coloproctol of Great Britain and Ireland 13:805–810
Sarker SK, Kumar I, Delaney C (2010) Assessing operative performance in advanced laparoscopic colorectal surgery. World J Surg 34:1594–1603
pubmed: 20182721
Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259
pubmed: 20832048
Miskovic D, Wyles SM, Carter F, Coleman MG, Hanna GB (2011) Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the english national training program. Surg Endosc 25:1136–1142
pubmed: 20835723
Mackenzie H, Cuming T, Miskovic D, Wyles SM, Langsford L, Anderson J, Thomas-Gibson S, Valori R, Hanna GB, Coleman MG, Francis N (2015) Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England. Ann Surg 261:149–156
pubmed: 24374538
Mackenzie H, Miskovic D, Ni M, Parvaiz A, Acheson AG, Jenkins JT, Griffith J, Coleman MG, Hanna GB (2013) Clinical and educational proficiency gain of supervised laparoscopic colorectal surgical trainees. Surg Endosc 27:2704–2711
pubmed: 23392980
Miskovic D, Ni M, Wyles SM, Kennedy RH, Francis NK, Parvaiz A, Cunningham C, Rockall TA, Gudgeon AM, Coleman MG, Hanna GB (2013) Is competency assessment at the specialist level achievable? A study for the national training programme in laparoscopic colorectal surgery in England. Ann Surg 257:476–482
pubmed: 23386240
Celentano V, Flashman KG (2019) Stepwise training in laparoscopic surgery for complex ileocolonic crohn’s disease: analysis of 127 training episodes. J Surg Educ 76:1364–1369
pubmed: 30930068
Glarner CE, McDonald RJ, Smith AB, Leverson GE, Peyre S, Pugh CM, Greenberg CC, Greenberg JA, Foley EF (2013) Utilizing a novel tool for the comprehensive assessment of resident operative performance. J Surg Educ 70:813–820
pubmed: 24209661
Champagne BJ, Steele SR, Hendren SK, Bakaki PM, Roberts PL, Delaney CP, Brady JT, MacRae HM (2017) The American society of colon and rectal surgeons assessment tool for performance of laparoscopic colectomy. Dis Colon Rectum 60:738–744
pubmed: 28594724
Nakayama Y, Kawamura H, Kobayashi H, Todate Y, Matsunaga R, Miyakawa T, Honda M (2020) Development of an assessment tool for laparoscopic sigmoidectomy using the delphi method. Annals of Cancer Research and Therapy 28:137–142
Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Taomoto J, Sugiyama Y, Murakami S, Saeki S, Mukaida H, Takiyama W (2013) Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today 43:271–275
pubmed: 22941345
Dath D, Regehr G, Birch D, Schlachta C, Poulin E, Mamazza J, Reznick R, MacRae HM (2004) Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills. Surg Endosc 18:1800–1804
pubmed: 15809794
Crawshaw BP, Steele SR, Lee EC, Delaney CP, Mustain WC, Russ AJ, Shanmugan S, Champagne BJ (2016) Failing to prepare is preparing to fail: a single-blinded, randomized controlled trial to determine the impact of a preoperative instructional video on the ability of residents to perform laparoscopic right colectomy. Dis Colon Rectum 59:28–34
pubmed: 26651109
Kitaguchi D, Takeshita N, Matsuzaki H, Oda T, Watanabe M, Mori K, Kobayashi E, Ito M (2020) Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg 79:88–94
pubmed: 32413503
Huettl F, Lang H, Paschold M, Watzka F, Wachter N, Hensel B, Kneist W, Huber T (2020) Rating of camera navigation skills in colorectal surgery. Int J Colorectal Dis 35:1111–1115
pubmed: 32222935
pmcid: 7245595
Celentano V, Smart N, Cahill RA, Spinelli A, Giglio MC, McGrath J, Obermair A, Pellino G, Hasegawa H, Lal P, Lorenzon L, De Angelis N, Boni L, Gupta S, Griffith JP, Acheson AG, Cecil TD, Coleman MG (2021) Development and validation of a recommended checklist for assessment of surgical videos quality: the LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS) video assessment tool. Surg Endosc 35:1362–1369
pubmed: 32253556
Holmboe ES, Hawkins RE, Huot SJ (2004) Effects of training in direct observation of medical residents’ clinical competence: a randomized trial. Ann Intern Med 140:874–881
pubmed: 15172901
George BC, Teitelbaum EN, Darosa DA, Hungness ES, Meyerson SL, Fryer JP, Schuller M, Zwischenberger JB (2013) Duration of faculty training needed to ensure reliable or performance ratings. J Surg Educ 70:703–708
pubmed: 24209644
Robertson RL, Park J, Gillman L, Vergis A (2020) The impact of rater training on the psychometric properties of standardized surgical skill assessment tools. Am J Surg 220:610–615
pubmed: 31982095
Robertson RL, Vergis A, Gillman LM, Park J (2018) Effect of rater training on the reliability of technical skill assessments: a randomized controlled trial. Can J Surg 61:15917
pubmed: 30265636
Benz S, Tannapfel A, Tam Y, Grunenwald A, Vollmer S, Stricker I (2019) Proposal of a new classification system for complete mesocolic excison in right-sided colon cancer. Tech Coloproctol 23:251–257
pubmed: 30838463
Foster JD, Miskovic D, Allison AS, Conti JA, Ockrim J, Cooper EJ, Hanna GB, Francis NK (2016) Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery. Tech Coloproctol 20:361–367
pubmed: 27154295
Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 119:166.e167–116
Yule S, Flin R, Paterson-Brown S, Maran N (2006) Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 139:140–149
pubmed: 16455321
Yule S, Paterson-Brown S (2012) Surgeons’ non-technical skills. Surg Clin North Am 92:37–50
pubmed: 22269259
Gjeraa K, Spanager L, Konge L, Petersen RH, Østergaard D (2016) Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc 30:5185–5199
pubmed: 27066972
Shanmugan S, Leblanc F, Senagore AJ, Ellis CN, Stein SL, Khan S, Delaney CP, Champagne BJ (2014) Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity? Dis Colon Rectum 57:210–214
pubmed: 24401883
Bilgic E, Endo S, Lebedeva E, Takao M, McKendy KM, Watanabe Y, Feldman LS, Vassiliou MC (2018) A scoping review of assessment tools for laparoscopic suturing. Surg Endosc 32:3009–3023
pubmed: 29721749
Watanabe Y, Bilgic E, Lebedeva E, McKendy KM, Feldman LS, Fried GM, Vassiliou MC (2016) A systematic review of performance assessment tools for laparoscopic cholecystectomy. Surg Endosc 30:832–844
pubmed: 26092014