An artificial intelligence model that automatically labels roux-en-Y gastric bypasses, a comparison to trained surgeon annotators.
Artificial intelligence
Intelligent video annotator
Machine learning
Procedure segmentation
Roux-en-Y gastric bypass
Surgical assessment
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
07 2023
07 2023
Historique:
received:
21
03
2022
accepted:
04
01
2023
medline:
7
7
2023
pubmed:
20
1
2023
entrez:
19
1
2023
Statut:
ppublish
Résumé
Artificial intelligence (AI) can automate certain tasks to improve data collection. Models have been created to annotate the steps of Roux-en-Y Gastric Bypass (RYGB). However, model performance has not been compared with individual surgeon annotator performance. We developed a model that automatically labels RYGB steps and compares its performance to surgeons. 545 videos (17 surgeons) of laparoscopic RYGB procedures were collected. An annotation guide (12 steps, 52 tasks) was developed. Steps were annotated by 11 surgeons. Each video was annotated by two surgeons and a third reconciled the differences. A convolutional AI model was trained to identify steps and compared with manual annotation. For modeling, we used 390 videos for training, 95 for validation, and 60 for testing. The performance comparison between AI model versus manual annotation was performed using ANOVA (Analysis of Variance) in a subset of 60 testing videos. We assessed the performance of the model at each step and poor performance was defined (F1-score < 80%). The convolutional model identified 12 steps in the RYGB architecture. Model performance varied at each step [F1 > 90% for 7, and > 80% for 2]. The reconciled manual annotation data (F1 > 80% for > 5 steps) performed better than trainee's (F1 > 80% for 2-5 steps for 4 annotators, and < 2 steps for 4 annotators). In testing subset, certain steps had low performance, indicating potential ambiguities in surgical landmarks. Additionally, some videos were easier to annotate than others, suggesting variability. After controlling for variability, the AI algorithm was comparable to the manual (p < 0.0001). AI can be used to identify surgical landmarks in RYGB comparable to the manual process. AI was more accurate to recognize some landmarks more accurately than surgeons. This technology has the potential to improve surgical training by assessing the learning curves of surgeons at scale.
Identifiants
pubmed: 36658282
doi: 10.1007/s00464-023-09870-6
pii: 10.1007/s00464-023-09870-6
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
5665-5672Informations de copyright
© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
Références
ASMBS. Estimate of Bariatric Surgery Numbers, 2011–2019. Retrieved from https://asmbs.org/resources/estimate-of-bariatric-surgery-numbers 2021
Hopper AN, Jamison MH, Lewis WG (2007) Learning curves in surgical practice. Postgrad Med J 83(986):777–779. https://doi.org/10.1136/pgmj.2007.057190
doi: 10.1136/pgmj.2007.057190
pubmed: 18057179
pmcid: 2750931
Kersebaum JN, Möller T, von Schönfels W et al (2020) Robotic roux-en-Y gastric bypass procedure guide. J Soc Laparosc Robot Surg. https://doi.org/10.4293/JSLS.2020.00062
doi: 10.4293/JSLS.2020.00062
Major P, Wysocki M, Dworak J, Pędziwiatr M, Małczak P, Budzyński A (2017) Are bariatric operations performed by residents safe and efficient? Surg Obes Relat Dis 13(4):614–621
doi: 10.1016/j.soard.2016.11.017
pubmed: 28159560
Loukas C (2018) Video content analysis of surgical procedures. Surg Endosc 32(2):553–568
doi: 10.1007/s00464-017-5878-1
pubmed: 29075965
Ward TM, Fer DM, Ban Y et al (2021) Challenges in surgical video annotation. Comput Assist Surg 26(1):58–68
doi: 10.1080/24699322.2021.1937320
Jin Y, Dou Q, Chen H et al (2017) SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network. IEEE Trans Med Imaging 37(5):1114–1126
doi: 10.1109/TMI.2017.2787657
Zisimopoulos O, Flouty E, Luengo I et al (2018) Deepphase: surgical phase recognition in cataracts videos. In: Frangi Alejandro F, Schnabel Julia A, Davatzikos Christos, Alberola-López Carlos, Fichtinger Gabor (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018: 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings. Part IV Springer, Cham
Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S et al (2015) Beyond short snippets: deep networks for video classification. arXiv:1503.08909
Zhang B, Ghanem A, Simes A et al (2021) Surgical workflow recognition with 3DCNN for sleeve gastrectomy. Int J Comput Assist Radiol Surg 16(11):2029–2036
doi: 10.1007/s11548-021-02473-3
pubmed: 34415503
pmcid: 8589754
Farha YA, Gall J (2019) MS-TCN: Multi-stage temporal convolutional network for action segmentation. arXiv:1903.01945
Czempiel T, Paschali M, Keicher M et al (2020) TeCNO: Surgical phase recognition with multi-stage temporal convolutional networks. In: Martel AL, Abolmaesumi P, Stoyanov D et al (eds) Medical Image Computing and Computer Assisted Intervention — MICCAI 2020. Lecture Notes in Computer Science, vol 12263. Springer, Cham
Zhang B, Ghanem A, Simes A et al (2021) SWNet: Surgical workflow recognition with deep convolutional network. PMLR 143:855–869
Kadkhodamohammadi A, Luengo I, Stoyanov D (2022) PATG: position-aware temporal graph networks for surgical phase recognition on laparoscopic videos. Int J Comput Assist Radiol Surg 17(5):849–856
doi: 10.1007/s11548-022-02600-8
pubmed: 35353299
Zhang B, Abbing J, Ghanem A et al (2022) Towards accurate surgical workflow recognition with convolutional networks and transformers. Comput Methods Biomech Biomed Eng Imaging Visual 10(4):349–356
doi: 10.1080/21681163.2021.2002191
Jin Y, Li H, Dou Q et al (2020) Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Medl Image Anal. https://doi.org/10.1016/j.media.2019.101572
doi: 10.1016/j.media.2019.101572
Ramesh S, Dall’Alba D, Gonzalez C et al (2021) Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-021-02388-z
doi: 10.1007/s11548-021-02388-z
pubmed: 34013464
pmcid: 8260406
Tran D, Wang H, Torresani L et al (2018) A closer look at spatiotemporal convolutions for action recognition. arXiv:1711.11248
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV. pp 770–778
Ghadiyaram D, Tran D, Mahajan D (2019) Large-scale weakly-supervised pre-training for video action recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA. pp 12038–12047
Derczynski L (2016) Complementarity, F-score, and NLP evaluation. In: International Conference on Language Resources and Evaluation
Goutte C, Gaussier E (2005) A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Losada David E, Fernández-Luna Juan M (eds) Advances in Information Retrieval. Springer, Berlin
Lecuyer G, Ragot M, Martin N et al (2019) Assisted annotation of surgical videos using deep learning. In: CARS 2019, Computer Assisted Radiology and Surgery, 33rd International Congress and Exhibition. Couvent des jacobins, Rennes, France
Kitaguchi D, Takeshita N, Matsuzaki H et al (2020) Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg. https://doi.org/10.1016/j.ijsu.2020.05.015
doi: 10.1016/j.ijsu.2020.05.015
pubmed: 32413503
Nwoye CI, Alapatt D, Yu T et al (2022) CholecTriplet2021: a benchmark challenge for surgical action triplet recognition. arXiv:2204.04746
Krippendorff K (2011) Computing Krippendorff’s alpha-reliability. Annenberg School for Communication Departmental Papers: Philadelphia, PA
McHugh ML (2012) Interrater reliability: the kappa statistic. Biochemia medica 22(3):276–282
doi: 10.11613/BM.2012.031
pubmed: 23092060
pmcid: 3900052