Development and validity evidence of an objective structured assessment of technical skills score for minimally invasive linear-stapled, hand-sewn intestinal anastomoses: the A-OSATS score.
Anastomosis
Delphi method
Minimally invasive surgery
OSATS
Skill assessment
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
06 2022
06 2022
Historique:
received:
04
04
2021
accepted:
17
10
2021
pubmed:
11
11
2021
medline:
14
5
2022
entrez:
10
11
2021
Statut:
ppublish
Résumé
The aim of this study was to develop a reliable objective structured assessment of technical skills (OSATS) score for linear-stapled, hand-sewn closure of enterotomy intestinal anastomoses (A-OSATS). The Delphi methodology was used to create a traditional and weighted A-OSATS score highlighting the more important steps for patient outcomes according to an international expert consensus. Minimally invasive novices, intermediates, and experts were asked to perform a minimally invasive linear-stapled intestinal anastomosis with hand-sewn closure of the enterotomy in a live animal model either laparoscopically or robot-assisted. Video recordings were scored by two blinded raters assessing intrarater and interrater reliability and discriminative abilities between novices (n = 8), intermediates (n = 24), and experts (n = 8). The Delphi process included 18 international experts and was successfully completed after 4 rounds. A total of 4 relevant main steps as well as 15 substeps were identified and a definition of each substep was provided. A maximum of 75 points could be reached in the unweighted A-OSATS score and 170 points in the weighted A-OSATS score respectively. A total of 41 anastomoses were evaluated. Excellent intrarater (r = 0.807-0.988, p < 0.001) and interrater (intraclass correlation coefficient = 0.923-0.924, p < 0.001) reliability was demonstrated. Both versions of the A-OSATS correlated well with the general OSATS and discriminated between novices, intermediates, and experts defined by their OSATS global rating scale. With the weighted and unweighted A-OSATS score, we propose a new reliable standard to assess the creation of minimally invasive linear-stapled, hand-sewn anastomoses based on an international expert consensus. Validity evidence in live animal models is provided in this study. Future research should focus on assessing whether the weighted A-OSATS exceeds the predictive capabilities of patient outcomes of the unweighted A-OSATS and provide further validity evidence on using the score on different anastomotic techniques in humans.
Identifiants
pubmed: 34755235
doi: 10.1007/s00464-021-08806-2
pii: 10.1007/s00464-021-08806-2
pmc: PMC9085690
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
4529-4541Informations de copyright
© 2021. The Author(s).
Références
Antoniou SA et al (2015) Past, present, and future of minimally invasive abdominal surgery. JSLS. https://doi.org/10.4293/JSLS.2015.00052
doi: 10.4293/JSLS.2015.00052
pubmed: 26508823
pmcid: 4589904
Nickel F et al (2021) Minimally Invasive Versus open AbdominoThoracic Esophagectomy for esophageal carcinoma (MIVATE)—study protocol for a randomized controlled trial DRKS00016773. Trials 22(1):1–12
doi: 10.1186/s13063-020-04966-z
Beyer-Berjot L et al (2014) Advanced training in laparoscopic abdominal surgery: a systematic review. Surgery 156(3):676–688
doi: 10.1016/j.surg.2014.04.044
Park IJ et al (2009) Multidimensional analysis of the learning curve for laparoscopic colorectal surgery: lessons from 1,000 cases of laparoscopic colorectal surgery. Surg Endosc 23(4):839–846
doi: 10.1007/s00464-008-0259-4
Wehrtmann F et al (2019) Learning curves of laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy in bariatric surgery: a systematic review and introduction of a standardization. Obes Surg 30:640–656
doi: 10.1007/s11695-019-04230-7
Haney CM et al (2020) Training and learning curves in minimally invasive pancreatic surgery: from simulation to mastery. J Pancreatol 3(2):101–110
doi: 10.1097/JP9.0000000000000050
Birkmeyer JD et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442
doi: 10.1056/NEJMsa1300625
Curtis NJ et al (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155(7):590–598
doi: 10.1001/jamasurg.2020.1004
Johnston MJ et al (2016) An overview of research priorities in surgical simulation: what the literature shows has been achieved during the 21st century and what remains. Am J Surg 211(1):214–225
doi: 10.1016/j.amjsurg.2015.06.014
Kowalewski KF et al (2018) LapTrain: multi-modality training curriculum for laparoscopic cholecystectomy-results of a randomized controlled trial. Surg Endosc 32(9):3830–3838
doi: 10.1007/s00464-018-6110-7
Harrington DT et al (2007) A time-cost analysis of teaching a laparoscopic entero-enterostomy. J Surg Educ 64(6):342–345
doi: 10.1016/j.jsurg.2007.06.009
Manuel-Palazuelos JC et al (2016) Learning curve patterns generated by a training method for laparoscopic small bowel anastomosis. Adv Simul (Lond) 1:16
doi: 10.1186/s41077-016-0017-y
Barreira MA et al (2017) Model for simulated training of laparoscopic gastroenterostomy. Acta Cir Bras 32(1):81–89
doi: 10.1590/s0102-865020170110
Vassiliou MC et al (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190(1):107–113
doi: 10.1016/j.amjsurg.2005.04.004
Martin J et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278
pubmed: 9052454
Peyre SE et al (2009) Laparoscopic Nissen fundoplication assessment: task analysis as a model for the development of a procedural checklist. Surg Endosc 23(6):1227–1232
doi: 10.1007/s00464-008-0214-4
Zevin B et al (2013) Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg 216(5):955-965.e8
doi: 10.1016/j.jamcollsurg.2013.01.003
Banegas MP et al (2016) Toward greater adoption of minimally invasive and nephron-sparing surgical techniques for renal cell cancer in the United States. Urol Oncol 34(10):433.e9-433.e17
doi: 10.1016/j.urolonc.2016.05.021
Watanabe Y et al (2016) A systematic review of performance assessment tools for laparoscopic cholecystectomy. Surg Endosc 30(3):832–844
doi: 10.1007/s00464-015-4285-8
de Montbrun S, Satterthwaite L, Grantcharov TP (2016) Setting pass scores for assessment of technical performance by surgical trainees. Br J Surg 103(3):300–306
doi: 10.1002/bjs.10047
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420
doi: 10.1037/0033-2909.86.2.420
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163
doi: 10.1016/j.jcm.2016.02.012
Hinkle DE, Wiersma W, Jurs SG (2003) Applied statistics for the behavioral sciences, vol 663. Houghton Mifflin College Division, Boston
Javier R et al (2019) Does rating with a checklist improve the effect of E-learning for cognitive and practical skills in bariatric surgery? A rater-blinded, randomized-controlled trial. Surg Endosc 33(5):1532–1543
doi: 10.1007/s00464-018-6441-4
Sullivan GM (2011) A primer on the validity of assessment instruments. In: The Accreditation Council for Graduate Medical Education Suite 2000, 515 ….
Messick S (1989) Validity. In: Linn R (ed) Educational Measurement (3rd Edn) American Council on Education. Macmillan, Washington
Kane MT (2006) Validation. In: Brennen RL (ed) Educational measurement. Praeger Publishers, Westport, pp 17–64
Cook DA et al (2015) A contemporary approach to validity arguments: a practical guide to K ane’s framework. Med Educ 49(6):560–575
doi: 10.1111/medu.12678
Dalkey NC (1969) The Delphi method: An experimental study of group opinion. RAND Corp Santa Monica Calif, Santa Monica
Schmidt MW et al (2018) The Heidelberg VR score: development and validation of a composite score for laparoscopic virtual reality training. Surg Endosc 33:2093
doi: 10.1007/s00464-018-6480-x
Chowriappa AJ et al (2013) Development and validation of a composite scoring system for robot-assisted surgical training–the Robotic Skills Assessment Score. J Surg Res 185(2):561–569
doi: 10.1016/j.jss.2013.06.054
Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201(2):251–259
doi: 10.1016/j.amjsurg.2010.01.031
Diamond IR et al (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 67(4):401–409
doi: 10.1016/j.jclinepi.2013.12.002
Hsu C-C, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assess Res Eval 12(1):1–8
Trevelyan EG, Robinson PN (2015) Delphi methodology in health research: how to do it? Eur J Integr Med 7(4):423–428
doi: 10.1016/j.eujim.2015.07.002
Skulmoski GJ, Hartman FT, Krahn J (2007) The Delphi method for graduate research. J Inform Technol Educ 6:001
Dawe SR et al (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101(9):1063–1076
doi: 10.1002/bjs.9482