Proposal and multicentric validation of a laparoscopic Roux-en-Y gastric bypass surgery ontology.
Inter-rater reliability
Intra-rater reliability
Laparoscopic Roux-en-Y gastric bypass
Ontology
Surgical data science
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
03 2023
03 2023
Historique:
received:
30
07
2022
accepted:
14
10
2022
pubmed:
27
10
2022
medline:
21
3
2023
entrez:
26
10
2022
Statut:
ppublish
Résumé
Phase and step annotation in surgical videos is a prerequisite for surgical scene understanding and for downstream tasks like intraoperative feedback or assistance. However, most ontologies are applied on small monocentric datasets and lack external validation. To overcome these limitations an ontology for phases and steps of laparoscopic Roux-en-Y gastric bypass (LRYGB) is proposed and validated on a multicentric dataset in terms of inter- and intra-rater reliability (inter-/intra-RR). The proposed LRYGB ontology consists of 12 phase and 46 step definitions that are hierarchically structured. Two board certified surgeons (raters) with > 10 years of clinical experience applied the proposed ontology on two datasets: (1) StraBypass40 consists of 40 LRYGB videos from Nouvel Hôpital Civil, Strasbourg, France and (2) BernBypass70 consists of 70 LRYGB videos from Inselspital, Bern University Hospital, Bern, Switzerland. To assess inter-RR the two raters' annotations of ten randomly chosen videos from StraBypass40 and BernBypass70 each, were compared. To assess intra-RR ten randomly chosen videos were annotated twice by the same rater and annotations were compared. Inter-RR was calculated using Cohen's kappa. Additionally, for inter- and intra-RR accuracy, precision, recall, F1-score, and application dependent metrics were applied. The mean ± SD video duration was 108 ± 33 min and 75 ± 21 min in StraBypass40 and BernBypass70, respectively. The proposed ontology shows an inter-RR of 96.8 ± 2.7% for phases and 85.4 ± 6.0% for steps on StraBypass40 and 94.9 ± 5.8% for phases and 76.1 ± 13.9% for steps on BernBypass70. The overall Cohen's kappa of inter-RR was 95.9 ± 4.3% for phases and 80.8 ± 10.0% for steps. Intra-RR showed an accuracy of 98.4 ± 1.1% for phases and 88.1 ± 8.1% for steps. The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB.
Sections du résumé
BACKGROUND
Phase and step annotation in surgical videos is a prerequisite for surgical scene understanding and for downstream tasks like intraoperative feedback or assistance. However, most ontologies are applied on small monocentric datasets and lack external validation. To overcome these limitations an ontology for phases and steps of laparoscopic Roux-en-Y gastric bypass (LRYGB) is proposed and validated on a multicentric dataset in terms of inter- and intra-rater reliability (inter-/intra-RR).
METHODS
The proposed LRYGB ontology consists of 12 phase and 46 step definitions that are hierarchically structured. Two board certified surgeons (raters) with > 10 years of clinical experience applied the proposed ontology on two datasets: (1) StraBypass40 consists of 40 LRYGB videos from Nouvel Hôpital Civil, Strasbourg, France and (2) BernBypass70 consists of 70 LRYGB videos from Inselspital, Bern University Hospital, Bern, Switzerland. To assess inter-RR the two raters' annotations of ten randomly chosen videos from StraBypass40 and BernBypass70 each, were compared. To assess intra-RR ten randomly chosen videos were annotated twice by the same rater and annotations were compared. Inter-RR was calculated using Cohen's kappa. Additionally, for inter- and intra-RR accuracy, precision, recall, F1-score, and application dependent metrics were applied.
RESULTS
The mean ± SD video duration was 108 ± 33 min and 75 ± 21 min in StraBypass40 and BernBypass70, respectively. The proposed ontology shows an inter-RR of 96.8 ± 2.7% for phases and 85.4 ± 6.0% for steps on StraBypass40 and 94.9 ± 5.8% for phases and 76.1 ± 13.9% for steps on BernBypass70. The overall Cohen's kappa of inter-RR was 95.9 ± 4.3% for phases and 80.8 ± 10.0% for steps. Intra-RR showed an accuracy of 98.4 ± 1.1% for phases and 88.1 ± 8.1% for steps.
CONCLUSION
The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB.
Identifiants
pubmed: 36289088
doi: 10.1007/s00464-022-09745-2
pii: 10.1007/s00464-022-09745-2
pmc: PMC10017621
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
2070-2077Informations de copyright
© 2022. The Author(s).
Références
Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S, Hashizume M, Katic D, Kenngott H, Kranzfelder M, Malpani A, März K, Neumuth T, Padoy N, Pugh C, Schoch N, Stoyanov D, Taylor R, Wagner M, Hager GD, Jannin P (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1:691–696. https://doi.org/10.1038/s41551-017-0132-7
doi: 10.1038/s41551-017-0132-7
pubmed: 31015666
Ward TM, Fer DM, Ban Y, Rosman G, Meireles OR, Hashimoto DA (2021) Challenges in surgical video annotation. Comput Assist Surg 26:58–68. https://doi.org/10.1080/24699322.2021.1937320
doi: 10.1080/24699322.2021.1937320
Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36:86–97. https://doi.org/10.1109/TMI.2016.2593957
doi: 10.1109/TMI.2016.2593957
pubmed: 27455522
Aspart F, Bolmgren JL, Lavanchy JL, Beldi G, Woods MS, Padoy N, Hosgor E (2022) ClipAssistNet: bringing real-time safety feedback to operating rooms. Int J Comput Assist Radiol Surg 17:5–13. https://doi.org/10.1007/s11548-021-02441-x
doi: 10.1007/s11548-021-02441-x
pubmed: 34297269
Mascagni P, Vardazaryan A, Alapatt D, Urade T, Emre T, Fiorillo C, Pessaux P, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2022) Artificial intelligence for surgical safety. Ann Surg 275:955–961. https://doi.org/10.1097/SLA.0000000000004351
doi: 10.1097/SLA.0000000000004351
pubmed: 33201104
Lavanchy JL, Zindel J, Kirtac K, Twick I, Hosgor E, Candinas D, Beldi G (2021) Automation of surgical skill assessment using a three-stage machine learning algorithm. Sci Rep. https://doi.org/10.1038/s41598-021-84295-6
doi: 10.1038/s41598-021-84295-6
pubmed: 33879815
pmcid: 8058074
Maier-Hein L, Wagner M, Ross T, Reinke A, Bodenstedt S, Full PM, Hempe H, Mindroc-Filimon D, Scholz P, Tran TN, Bruno P, Kisilenko A, Müller B, Davitashvili T, Capek M, Tizabi MD, Eisenmann M, Adler TJ, Gröhl J, Schellenberg M, Seidlitz S, Lai TYE, Pekdemir B, Roethlingshoefer V, Both F, Bittel S, Mengler M, Mündermann L, Apitz M, Kopp-Schneider A, Speidel S, Nickel F, Probst P, Kenngott HG, Müller-Stich BP (2021) Heidelberg colorectal data set for surgical data science in the sensor operating room. Sci Data. https://doi.org/10.1038/s41597-021-00882-2
doi: 10.1038/s41597-021-00882-2
pubmed: 33846356
pmcid: 8042116
Kitaguchi D, Takeshita N, Matsuzaki H, Oda T, Watanabe M, Mori K, Kobayashi E, Ito M (2020) Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg 79:88–94. https://doi.org/10.1016/j.ijsu.2020.05.015
doi: 10.1016/j.ijsu.2020.05.015
pubmed: 32413503
Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 270:414–421. https://doi.org/10.1097/SLA.0000000000003460
doi: 10.1097/SLA.0000000000003460
pubmed: 31274652
Ramesh S, Dall’Alba D, Gonzalez C, Yu T, Mascagni P, Mutter D, Marescaux J, Fiorini P, Padoy N (2021) Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures. Int J Comput Assist Radiol Surg 16(7):1111–1119. https://doi.org/10.1007/s11548-021-02388-z
doi: 10.1007/s11548-021-02388-z
pubmed: 34013464
pmcid: 8260406
Welbourn R, Hollyman M, Kinsman R, Dixon J, Liem R, Ottosson J, Ramos A, Våge V, Al-Sabah S, Brown W, Cohen R, Walton P, Himpens J (2019) Bariatric surgery worldwide: baseline demographic description and one-year outcomes from the fourth IFSO global registry report 2018. Obes Surg 29:782–795. https://doi.org/10.1007/s11695-018-3593-1
doi: 10.1007/s11695-018-3593-1
pubmed: 30421326
Angrisani L, Santonicola A, Iovino P, Ramos A, Shikora S, Kow L (2021) Bariatric surgery survey 2018: similarities and disparities among the 5 IFSO chapters. Obes Surg 31:1937–1948. https://doi.org/10.1007/s11695-020-05207-7/Published
doi: 10.1007/s11695-020-05207-7/Published
pubmed: 33432483
pmcid: 7800839
Flum DR, Belle SH, King WC, Wahed AS, Berk P, Chapman W, Pories W, Courcoulas A, McCloskey C, Mitchell J, Patterson E (2009) Perioperative safety in the longitudinal assessment of bariatric surgery. N Engl J Med 361:445–454. https://doi.org/10.1056/NEJMoa0901836
doi: 10.1056/NEJMoa0901836
pubmed: 19641201
Topart P, Becouarn G, Ritz P (2012) Comparative early outcomes of three laparoscopic bariatric procedures: sleeve gastrectomy, Roux-en-Y gastric bypass, and biliopancreatic diversion with duodenal switch. Surg Obes Relat Dis 8:250–254. https://doi.org/10.1016/j.soard.2011.05.012
doi: 10.1016/j.soard.2011.05.012
pubmed: 21803660
Kehagias I, Karamanakos SN, Argentou M, Kalfarentzos F (2011) Randomized clinical trial of laparoscopic Roux-en-Y gastric bypass versus laparoscopic sleeve gastrectomy for the management of patients with BMI < 50 kg/m2. Obes Surg 21:1650–1656. https://doi.org/10.1007/s11695-011-0479-x
doi: 10.1007/s11695-011-0479-x
pubmed: 21818647
Meireles OR, Rosman G, Altieri MS, Carin L, Hager G, Madani A, Padoy N, Pugh CM, Sylla P, Ward TM, Hashimoto DA, Ban Y, Filicori F, Mascagni P, Mellinger J, Schlacta C, Speidel S, Juergens T, Garcia-Kilroy P, Asselman D, Bohnen J, Draelos RB, Fuchs H, Henao R, Sarikaya D, Boyle C, Fer D, Li Z, Ramadorai A, Stoyanov D, Yoo A, Gonzalez C, Oleynikov D, Pratt J, Scott D, Vedula S, Witkowski E, Shimizu T, Tousignant M, Azagury D, Bridault F, Dunkin B, Grantcharov T, Jannin P, Malpani A, Perretta S, Schwaitzberg S, Jarc A, Landfors K, Mahadik A, Nguyen H (2021) SAGES consensus recommendations on an annotation framework for surgical video. Surg Endosc 35:4918–4929. https://doi.org/10.1007/s00464-021-08578-9
doi: 10.1007/s00464-021-08578-9
pubmed: 34231065
Gibaud B, Forestier G, Feldmann C, Ferrigno G, Gonçalves P, Haidegger T, Julliard C, Katić D, Kenngott H, Maier-Hein L, März K, de Momi E, Nagy DÁ, Nakawala H, Neumann J, Neumuth T, Rojas Balderrama J, Speidel S, Wagner M, Jannin P (2018) Toward a standard ontology of surgical process models. Int J Comput Assist Radiol Surg 13:1397–1408. https://doi.org/10.1007/s11548-018-1824-5
doi: 10.1007/s11548-018-1824-5
pubmed: 30006820
Garrow CR, Kowalewski KF, Li L, Wagner M, Schmidt MW, Engelhardt S, Hashimoto DA, Kenngott HG, Bodenstedt S, Speidel S, Müller-Stich BP, Nickel F (2021) Machine learning for surgical phase recognition: a systematic review. Ann Surg 273:684–693. https://doi.org/10.1097/SLA.0000000000004425
doi: 10.1097/SLA.0000000000004425
pubmed: 33201088
Dergachyova O, Bouget D, Huaulmé A, Morandi X, Jannin P, Dergachyova O, Bouget D, Huaulmé A, Morandi X, Jannin P, Rennes MCHU, X (2016) Automatic data-driven real-time segmentation and recognition of surgical workflow. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-016
doi: 10.1007/s11548-016
pubmed: 26995598
Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Measur 20:37–46. https://doi.org/10.1177/001316446002000104
doi: 10.1177/001316446002000104