Challenges in multi-centric generalization: phase and step recognition in Roux-en-Y gastric bypass surgery.
Gastric bypass
Multi-centric validation
Multi-task temporal convolutional network
Phase recognition
Step recognition
Surgical data science
Journal
International journal of computer assisted radiology and surgery
ISSN: 1861-6429
Titre abrégé: Int J Comput Assist Radiol Surg
Pays: Germany
ID NLM: 101499225
Informations de publication
Date de publication:
18 May 2024
18 May 2024
Historique:
received:
16
12
2023
accepted:
02
04
2024
medline:
18
5
2024
pubmed:
18
5
2024
entrez:
18
5
2024
Statut:
aheadofprint
Résumé
Most studies on surgical activity recognition utilizing artificial intelligence (AI) have focused mainly on recognizing one type of activity from small and mono-centric surgical video datasets. It remains speculative whether those models would generalize to other centers. In this work, we introduce a large multi-centric multi-activity dataset consisting of 140 surgical videos (MultiBypass140) of laparoscopic Roux-en-Y gastric bypass (LRYGB) surgeries performed at two medical centers, i.e., the University Hospital of Strasbourg, France (StrasBypass70) and Inselspital, Bern University Hospital, Switzerland (BernBypass70). The dataset has been fully annotated with phases and steps by two board-certified surgeons. Furthermore, we assess the generalizability and benchmark different deep learning models for the task of phase and step recognition in 7 experimental studies: (1) Training and evaluation on BernBypass70; (2) Training and evaluation on StrasBypass70; (3) Training and evaluation on the joint MultiBypass140 dataset; (4) Training on BernBypass70, evaluation on StrasBypass70; (5) Training on StrasBypass70, evaluation on BernBypass70; Training on MultiBypass140, (6) evaluation on BernBypass70 and (7) evaluation on StrasBypass70. The model's performance is markedly influenced by the training data. The worst results were obtained in experiments (4) and (5) confirming the limited generalization capabilities of models trained on mono-centric data. The use of multi-centric training data, experiments (6) and (7), improves the generalization capabilities of the models, bringing them beyond the level of independent mono-centric training and validation (experiments (1) and (2)). MultiBypass140 shows considerable variation in surgical technique and workflow of LRYGB procedures between centers. Therefore, generalization experiments demonstrate a remarkable difference in model performance. These results highlight the importance of multi-centric datasets for AI model generalization to account for variance in surgical technique and workflows. The dataset and code are publicly available at https://github.com/CAMMA-public/MultiBypass140.
Identifiants
pubmed: 38761319
doi: 10.1007/s11548-024-03166-3
pii: 10.1007/s11548-024-03166-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
ID : P500PM 206724
Organisme : Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
ID : P5R5PM 217663
Organisme : Novartis Stiftung für Medizinisch-Biologische Forschung
ID : #23C162
Organisme : Horizon 2020 Framework Programme
ID : 813782
Organisme : Academie Nationale de la Recherche
ID : ANR-20-CHIA-0029-01
Organisme : Academie Nationale de la Recherche
ID : ANR-10-IAHU-02
Informations de copyright
© 2024. The Author(s).
Références
Maier-Hein L, Eisenmann M, Sarikaya D et al (2022) Surgical data science - from concepts toward clinical translation. Med Image Anal 76:102306
doi: 10.1016/j.media.2021.102306
pubmed: 34879287
Pedrett R, Mascagni P, Beldi G, Padoy N, Lavanchy JL (2023) Technical skill assessment in minimally invasive surgery using artificial intelligence: A systematic review. Surg Endosc 37:7412–424
doi: 10.1007/s00464-023-10335-z
pubmed: 37584774
pmcid: 10520175
Meireles OR, Rosman G, Altieri MS, Carin L, Hager G, Madani A, Padoy N, Pugh CM, Sylla P, Ward TM, H DA, (2021) SAGES consensus recommendations on an annotation framework for surgical video. Surg Endosc 35(9):4918–4929
Garrow CR, Kowalewski K-F, Li L, Wagner M, Schmidt MW, Engelhardt S, Hashimoto DA, Kenngott HG, Bodenstedt S, Speidel S, Müller-Stich BP, Nickel F (2020) Machine learning for surgical phase recognition. Ann Surg 273(4):684–693
doi: 10.1097/SLA.0000000000004425
Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: A deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86–97
doi: 10.1109/TMI.2016.2593957
pubmed: 27455522
Ramesh S, Dall’Alba D, Gonzalez C, Yu T, Mascagni P, Mutter D, Marescaux J, Fiorini P, Padoy N (2021) Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures. Int J Comput Assist Radiol Surg
Charriere K, Quellec G, Lamard M, Coatrieux G, Cochener B, Cazuguel G (2014) Automated surgical step recognition in normalized cataract surgery videos. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 4647–4650
Nwoye CI, Yu T, Gonzalez C, Seeliger B, Mascagni P, Mutter D, Marescaux J, Padoy N (2022) Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Med Image Anal 78:102433
doi: 10.1016/j.media.2022.102433
pubmed: 35398658
Hajj HA, Lamard M, Conze P-H, Cochener B, Quellec G (2018) Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks. Med Image Anal 47:203–218
doi: 10.1016/j.media.2018.05.001
pubmed: 29778931
Vardazaryan A, Mutter D, Marescaux J, Padoy N (2018) Weakly-supervised learning for tool localization in laparoscopic videos. In: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, 169–179
Lavanchy JL, Gonzalez C, Kassem H, Nett PC, Mutter D, Padoy N (2023) Proposal and multicentric validation of a laparoscopic roux-en-y gastric bypass surgery ontology. Surg Endosc 37(3):2070–2077
doi: 10.1007/s00464-022-09745-2
pubmed: 36289088
Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video. Ann Surg 270(3):414–421
doi: 10.1097/SLA.0000000000003460
pubmed: 31274652
Kitaguchi D, Takeshita N, Matsuzaki H, Hasegawa H, Igaki T, Oda T, Ito M (2021) Deep learning-based automatic surgical step recognition in intraoperative videos for transanal total mesorectal excision. Surg Endosc 36(2):1143–1151
doi: 10.1007/s00464-021-08381-6
pubmed: 33825016
pmcid: 8758657
Takeuchi M, Collins T, Ndagijimana A, Kawakubo H, Kitagawa Y, Marescaux J, Mutter D, Perretta S, Hostettler A, Dallemagne B (2022) Automatic surgical phase recognition in laparoscopic inguinal hernia repair with artificial intelligence. Hernia 26(6):1669–1678
doi: 10.1007/s10029-022-02621-x
pubmed: 35536371
Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH, Navarrete-Welton A, Sankaranarayanan G, Brunt LM, Okrainec A, Alseidi A (2020) Artificial intelligence for intraoperative guidance. Ann Surg 276(2):363–369
doi: 10.1097/SLA.0000000000004594
pubmed: 33196488
Mascagni P, Alapatt D, Laracca GG, Guerriero L, Spota A, Fiorillo C, Vardazaryan A, Quero G, Alfieri S, Baldari L, Cassinotti E, Boni L, Cuccurullo D, Costamagna G, Dallemagne B, Padoy N (2022) Multicentric validation of EndoDigest: a computer vision platform for video documentation of the critical view of safety in laparoscopic cholecystectomy. Surgical Endoscopy
Kassem H, Alapatt D, Mascagni P, Karargyris A, Padoy N (2023) Federated Cycling (FedCy): Semi-Supervised Federated Learning of Surgical Phases. IEEE Trans Med Imaging 42(7):1920–1931
doi: 10.1109/TMI.2022.3222126
pubmed: 36374877
Wagner M, Müller-Stich B-P, Kisilenko A, et al (2023) Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark. Medical Image Analysis 86, 102770
Valderrama N, Puentes PR, Hernández I, Ayobi N, Verlyck M, Santander J, Caicedo J, Fernández N, Arbeláez P (2022) Towards holistic surgical scene understanding. In: Lecture Notes in Computer Science, 442–452
Lavanchy, J.L., Vardazaryan, A., Mascagni, P., Consortium, A., Mutter, D., Padoy, N. (2023) Preserving privacy in surgical video analysis using a deep learning classifier to identify out-of-body scenes in endoscopic videos. Sci Rep 13(1):9235
doi: 10.1038/s41598-023-36453-1
Mazellier J-P, Boujon A, Bour-Lang M, Erharhd M, Waechter J, Wernert E, Mascagni P, Padoy N (2023) MOSaiC: a Web-based Platform for Collaborative Medical Video Assessment and Annotation. arXiv
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Hochreiter S, Schmidhuber J (1996) Lstm can solve hard long time lag problems. In:Mozer MC, Jordan M, Petsche T (eds.) Advances in Neural Information Processing Systems, 473–479
Czempiel T, Paschali M, Keicher M, Simson W, Feussner H, Kim ST, Navab N (2020) TeCNO: Surgical phase recognition with multi-stage temporal convolutional networks. In: International Conference on Medical Image Computing and Computer-assisted Intervention, 343–352