Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy.
Assessment scale
Gastrectomy
Laparoscopy
Outcome prediction
Skill evaluation
Surgical education
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
12 2022
12 2022
Historique:
received:
10
01
2022
accepted:
23
04
2022
pubmed:
17
5
2022
medline:
16
11
2022
entrez:
16
5
2022
Statut:
ppublish
Résumé
The Japanese operative-rating scale for laparoscopic distal gastrectomy (JORS-LDG) was developed through cognitive task analysis together with the Delphi method to measure intraoperative performance during laparoscopic distal gastrectomy. This study aimed to investigate the value of this rating scale as an educational tool and a surgical outcome predictor in laparoscopic distal gastrectomy. The surgical performance of laparoscopic distal gastrectomy was assessed by the first assistant, through self-evaluation in the operating room and by video raters blind to the case. We evaluated inter-rater reliability, internal consistency, and correlations between the JORS-LDG scores and the evaluation methods, patient characteristics, and surgical outcomes. Fifty-four laparoscopic distal gastrectomy procedures performed by 40 surgeons at 16 institutions were evaluated in the operating room and with video recordings using the proposed rating scale. The video inter-rater reliability was > 0.8. Participating surgeons were divided into the low, intermediate, and high groups based on their total scores. The number of laparoscopic surgeries and laparoscopic gastrectomy procedures performed differed significantly among the groups according to laparoscopic distal gastrectomy skill levels. The low, intermediate, and high groups also differed in terms of median operating times (311, 266, and 229 min, respectively, P < 0.001), intraoperative complication rates (27.8, 11.8, and 0%, respectively, P = 0.01), and postoperative complication rates (22.2, 0, and 0%, respectively, P = 0.002). The JORS-LDG is a reliable and valid measure for laparoscopic distal gastrectomy training and could be useful in predicting surgical outcomes.
Sections du résumé
BACKGROUND
The Japanese operative-rating scale for laparoscopic distal gastrectomy (JORS-LDG) was developed through cognitive task analysis together with the Delphi method to measure intraoperative performance during laparoscopic distal gastrectomy. This study aimed to investigate the value of this rating scale as an educational tool and a surgical outcome predictor in laparoscopic distal gastrectomy.
METHODS
The surgical performance of laparoscopic distal gastrectomy was assessed by the first assistant, through self-evaluation in the operating room and by video raters blind to the case. We evaluated inter-rater reliability, internal consistency, and correlations between the JORS-LDG scores and the evaluation methods, patient characteristics, and surgical outcomes.
RESULTS
Fifty-four laparoscopic distal gastrectomy procedures performed by 40 surgeons at 16 institutions were evaluated in the operating room and with video recordings using the proposed rating scale. The video inter-rater reliability was > 0.8. Participating surgeons were divided into the low, intermediate, and high groups based on their total scores. The number of laparoscopic surgeries and laparoscopic gastrectomy procedures performed differed significantly among the groups according to laparoscopic distal gastrectomy skill levels. The low, intermediate, and high groups also differed in terms of median operating times (311, 266, and 229 min, respectively, P < 0.001), intraoperative complication rates (27.8, 11.8, and 0%, respectively, P = 0.01), and postoperative complication rates (22.2, 0, and 0%, respectively, P = 0.002).
CONCLUSIONS
The JORS-LDG is a reliable and valid measure for laparoscopic distal gastrectomy training and could be useful in predicting surgical outcomes.
Identifiants
pubmed: 35578050
doi: 10.1007/s00464-022-09305-8
pii: 10.1007/s00464-022-09305-8
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
8807-8816Informations de copyright
© 2022. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
Références
Martin JA, Regehr G, Reznick R, Macdrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278
pubmed: 9052454
Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555 (discussion 555)
doi: 10.1016/j.amjsurg.2007.02.003
pubmed: 17434353
Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ (2012) The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med 87:1401–1407
doi: 10.1097/ACM.0b013e3182677805
pubmed: 22914526
Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113
doi: 10.1016/j.amjsurg.2005.04.004
pubmed: 15972181
Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the global operative assessment of laparoscopic skills-groin hernia (GOALS-GH). Am J Surg 201:54–61
doi: 10.1016/j.amjsurg.2010.09.006
pubmed: 21167366
Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259
doi: 10.1016/j.amjsurg.2010.01.031
pubmed: 20832048
Kurashima Y, Watanabe Y, Hiki N, Poudel S, Kitagami H, Ebihara Y, Murakami S, Shichinohe T, Hirano S (2019) Development of a novel tool to assess skills in laparoscopic gastrectomy using the Delphi method: the Japanese operative rating scale for laparoscopic distal gastrectomy (JORS-LDG). Surg Endosc 33:3945–3952
doi: 10.1007/s00464-019-06681-6
pubmed: 30756172
Poudel S, Kurashima Y, Kawarada Y, Watanabe Y, Murakami Y, Matsumura Y, Kato H, Miyazaki K, Shichinohe T, Hirano S (2016) Development and validation of a checklist for assessing recorded performance of laparoscopic inguinal hernia repair. Am J Surg 212:468–474
doi: 10.1016/j.amjsurg.2015.09.014
pubmed: 26750607
Fecso AB, Szasz P, Kerezov G, Grantcharov TP (2017) The effect of technical performance on patient outcomes in surgery: a systematic review. Ann Surg 265:492–501
doi: 10.1097/SLA.0000000000001959
pubmed: 27537534
Arvidsson D, Berndsen FH, Larsson LG, Leijonmarck CE, Rimbäck G, Rudberg C, Smedberg S, Spangen L, Montgomery A (2005) Randomized clinical trial comparing 5-year recurrence rate after laparoscopic versus Shouldice repair of primary inguinal hernia. Br J Surg 92:1085–1091
doi: 10.1002/bjs.5137
pubmed: 16106480
Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442
doi: 10.1056/NEJMsa1300625
pubmed: 24106936
Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon M, Khan Z, Longman R, Coleman MG, Hanna GB (2015) Clinical validity of consultant technical skills assessment in the english national training programme for laparoscopic colorectal surgery. Br J Surg 102:991–997
doi: 10.1002/bjs.9828
pubmed: 25994456
Hogg ME, Zenati M, Novak S, Chen Y, Jun Y, Steve J, Kowalsky SJ, Bartlett DL, Zureikat AH, Zeh HJ (2016) Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg 264:482–491
doi: 10.1097/SLA.0000000000001862
pubmed: 27433897
Fecso AB, Bhatti JA, Stotland PK, Quereshy FA, Grantcharov TP (2019) Technical performance as a predictor of clinical outcomes in laparoscopic gastric cancer surgery. Ann Surg 270:115–120
doi: 10.1097/SLA.0000000000002741
pubmed: 29578907
Curtis NJ, Foster JD, Miskovic D, Brown CSB, Hewett PJ, Abbott S, Hanna GB, Stevenson ARL, Francis NK (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155:590–598
doi: 10.1001/jamasurg.2020.1004
pubmed: 32374371
Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SR, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg 155:960–968
doi: 10.1001/jamasurg.2020.3007
pubmed: 32838425
pmcid: 7439214
Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg 273:766–771
doi: 10.1097/SLA.0000000000003385
pubmed: 31188214
Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 67:401–409
doi: 10.1016/j.jclinepi.2013.12.002
pubmed: 24581294
Walter SD, Eliasziw M, Donner A (1998) Sample size and optimal designs for reliability studies. Stat Med 17:101–110
doi: 10.1002/(SICI)1097-0258(19980115)17:1<101::AID-SIM727>3.0.CO;2-E
pubmed: 9463853
Mori T, Kimura T, Kitajima M (2010) Skill accreditation system for laparoscopic gastroenterologic surgeons in Japan. Minim Invasive Ther Allied Technol 19:18–23
doi: 10.3109/13645700903492969
pubmed: 20095893
Szasz P, Louridas M, Harris KA, Aggarwal R, Grantcharov TP (2015) Assessing technical competence in surgical trainees: a systematic review. Ann Surg 261:1046–1055
doi: 10.1097/SLA.0000000000000866
pubmed: 25119118
Groenier M, Brummer L, Bunting BP, Gallagher AG (2020) Reliability of observational assessment methods for outcome-based assessment of surgical skill: systematic review and meta-analyses. J Surg Educ 77:189–201
doi: 10.1016/j.jsurg.2019.07.007
pubmed: 31444148
Shibasaki S, Suda K, Nakauchi M, Nakamura T, Kadoya S, Kikuchi K, Inaba K, Uyama I (2018) Outermost layer-oriented medial approach for infrapyloric nodal dissection in laparoscopic distal gastrectomy. Surg Endosc 32:2137–2148
doi: 10.1007/s00464-018-6111-6
pubmed: 29450630
Kitano S (2009) What technique is suitable for laparoscopic suprapancreatic lymph node dissection? Gastric Cancer 12:67–68
doi: 10.1007/s10120-009-0512-0
pubmed: 19562459
Fukunaga T, Hiki N, Tokunaga M, Nohara K, Akashi Y, Katayama H, Yoshiba H, Yamada K, Ohyama S, Yamaguchi T (2009) Left-sided approach for suprapancreatic lymph node dissection in laparoscopy-assisted distal gastrectomy without duodenal transection. Gastric Cancer 12:106–112
doi: 10.1007/s10120-009-0508-9
pubmed: 19562465
Guerra F, Giuliani G, Iacobone M, Bianchi PP, Coratti A (2017) Pancreas-related complications following gastrectomy: systematic review and meta-analysis of open versus minimally invasive surgery. Surg Endosc 31:4346–4356
doi: 10.1007/s00464-017-5507-z
pubmed: 28378074
Washio M, Yamashita K, Niihara M, Hosoda K, Hiki N (2020) Postoperative pancreatic fistula after gastrectomy for gastric cancer. Ann Gastroenterol Surg 4:618–627
doi: 10.1002/ags3.12398
pubmed: 33319151
pmcid: 7726690
Hiki N, Honda M, Etoh T, Yoshida K, Kodera Y, Kakeji Y, Kumamaru H, Miyata H, Yamashita Y, Inomata M, Konno H, Seto Y, Kitano S (2018) Higher incidence of pancreatic fistula in laparoscopic gastrectomy. Real-world evidence from a nationwide prospective cohort study. Gastric Cancer 21:162–170
doi: 10.1007/s10120-017-0764-z
pubmed: 28887712
Bonrath EM, Zevin B, Dedy NJ, Grantcharov TP (2013) Error rating tool to identify and analyse technical errors and events in laparoscopic surgery. Br J Surg 100:1080–1088
doi: 10.1002/bjs.9168
pubmed: 23754649
Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell T (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262
doi: 10.1097/SLA.0000000000000520
pubmed: 24424150