Real-time detection of active bleeding in laparoscopic colectomy using artificial intelligence.
Artificial intelligence
Bleeding
Object detection
Surgery
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
17 May 2024
17 May 2024
Historique:
received:
12
02
2024
accepted:
20
04
2024
medline:
18
5
2024
pubmed:
18
5
2024
entrez:
17
5
2024
Statut:
aheadofprint
Résumé
Most intraoperative adverse events (iAEs) result from surgeons' errors, and bleeding is the majority of iAEs. Recognizing active bleeding timely is important to ensure safe surgery, and artificial intelligence (AI) has great potential for detecting active bleeding and providing real-time surgical support. This study aimed to develop a real-time AI model to detect active intraoperative bleeding. We extracted 27 surgical videos from a nationwide multi-institutional surgical video database in Japan and divided them at the patient level into three sets: training (n = 21), validation (n = 3), and testing (n = 3). We subsequently extracted the bleeding scenes and labeled distinctively active bleeding and blood pooling frame by frame. We used pre-trained YOLOv7_6w and developed a model to learn both active bleeding and blood pooling. The Average Precision at an Intersection over Union threshold of 0.5 (AP.50) for active bleeding and frames per second (FPS) were quantified. In addition, we conducted two 5-point Likert scales (5 = Excellent, 4 = Good, 3 = Fair, 2 = Poor, and 1 = Fail) questionnaires about sensitivity (the sensitivity score) and number of overdetection areas (the overdetection score) to investigate the surgeons' assessment. We annotated 34,117 images of 254 bleeding events. The AP.50 for active bleeding in the developed model was 0.574 and the FPS was 48.5. Twenty surgeons answered two questionnaires, indicating a sensitivity score of 4.92 and an overdetection score of 4.62 for the model. We developed an AI model to detect active bleeding, achieving real-time processing speed. Our AI model can be used to provide real-time surgical support.
Sections du résumé
BACKGROUND
BACKGROUND
Most intraoperative adverse events (iAEs) result from surgeons' errors, and bleeding is the majority of iAEs. Recognizing active bleeding timely is important to ensure safe surgery, and artificial intelligence (AI) has great potential for detecting active bleeding and providing real-time surgical support. This study aimed to develop a real-time AI model to detect active intraoperative bleeding.
METHODS
METHODS
We extracted 27 surgical videos from a nationwide multi-institutional surgical video database in Japan and divided them at the patient level into three sets: training (n = 21), validation (n = 3), and testing (n = 3). We subsequently extracted the bleeding scenes and labeled distinctively active bleeding and blood pooling frame by frame. We used pre-trained YOLOv7_6w and developed a model to learn both active bleeding and blood pooling. The Average Precision at an Intersection over Union threshold of 0.5 (AP.50) for active bleeding and frames per second (FPS) were quantified. In addition, we conducted two 5-point Likert scales (5 = Excellent, 4 = Good, 3 = Fair, 2 = Poor, and 1 = Fail) questionnaires about sensitivity (the sensitivity score) and number of overdetection areas (the overdetection score) to investigate the surgeons' assessment.
RESULTS
RESULTS
We annotated 34,117 images of 254 bleeding events. The AP.50 for active bleeding in the developed model was 0.574 and the FPS was 48.5. Twenty surgeons answered two questionnaires, indicating a sensitivity score of 4.92 and an overdetection score of 4.62 for the model.
CONCLUSIONS
CONCLUSIONS
We developed an AI model to detect active bleeding, achieving real-time processing speed. Our AI model can be used to provide real-time surgical support.
Identifiants
pubmed: 38760565
doi: 10.1007/s00464-024-10874-z
pii: 10.1007/s00464-024-10874-z
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Japan Agency for Medical Research and Development
ID : JP21he2102001h0003
Informations de copyright
© 2024. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
Références
Bohnen JD, Mavros MN, Ramly EP, Chang Y, Yeh DD, Lee J, de Moya M, King DR, Fagenholz PJ, Butler K, Velmahos GC, Kaafarani HMA (2017) Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room. Ann Surg 265:1119–1125
doi: 10.1097/SLA.0000000000001906
pubmed: 27805961
Garbens A, Goldenberg M, Wallis CJD, Tricco A, Grantcharov TP (2018) The cost of intraoperative adverse events in abdominal and pelvic surgery: a systematic review. Am J Surg 215:163–170
doi: 10.1016/j.amjsurg.2017.06.025
pubmed: 28709625
Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113
doi: 10.1016/j.amjsurg.2005.04.004
pubmed: 15972181
Champagne BJ, Steele SR, Hendren SK, Bakaki PM, Roberts PL, Delaney CP, Brady JT, MacRae HM (2017) The American Society of colon and rectal surgeons assessment tool for performance of laparoscopic colectomy. Dis Colon Rectum 60:738–744
doi: 10.1097/DCR.0000000000000817
pubmed: 28594724
Miskovic D, Ni M, Wyles SM, Parvaiz A, Hanna GB (2012) Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surg Endosc 26:796–803
doi: 10.1007/s00464-011-1955-z
pubmed: 22042584
Foster JD, Miskovic D, Allison AS, Conti JA, Ockrim J, Cooper EJ, Hanna GB, Francis NK (2016) Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery. Tech Coloproctol 20:361–367
doi: 10.1007/s10151-016-1444-4
pubmed: 27154295
Van Rutte P, Nienhuijs S, Jakimowicz J, Van Montfort G (2017) Identification of technical errors and hazard zones in sleeve gastrectomy using OCHRA. Surg Endosc 31:561–566
doi: 10.1007/s00464-016-4997-4
pubmed: 27287912
Jung JJ, Jüni P, Gee DW, Zak Y, Cheverie J, Yoo JS, Morton JM, Grantcharov T (2020) Development and evaluation of a novel instrument to measure severity of intraoperative events using video data. Ann Surg 272:220–226
doi: 10.1097/SLA.0000000000003897
pubmed: 32675485
Bonrath EM, Zevin B, Dedy NJ, Grantcharov TP (2013) Error rating tool to identify and analyse technical errors and events in laparoscopic surgery. Br J Surg 100:1080–1088
doi: 10.1002/bjs.9168
pubmed: 23754649
Wu WC, Smith TS, Henderson WG, Eaton CB, Poses RM, Uttley G, Mor V, Sharma SC, Vezeridis M, Khuri SF, Friedmann PD (2010) Operative blood loss, blood transfusion, and 30-day mortality in older patients after major noncardiac surgery. Ann Surg 252:11–17
doi: 10.1097/SLA.0b013e3181e3e43f
pubmed: 20505504
Roshanov PS, Eikelboom JW, Sessler DI, Kearon C, Guyatt GH, Crowther M, Tandon V, Borges FK, Lamy A, Whitlock R, Biccard BM, Szczeklik W, Panju M, Spence J, Garg AX, McGillion M, VanHelder T, Kavsak PA, de Beer J, Winemaker M, Le Manach Y, Sheth T, Pinthus JH, Siegal D, Thabane L, Simunovic MRI, Mizera R, Ribas S, Devereaux PJ (2021) Bleeding Independently associated with Mortality after noncardiac Surgery (BIMS): an international prospective cohort study establishing diagnostic criteria and prognostic importance. Br J Anaesth 126:163–171
doi: 10.1016/j.bja.2020.06.051
pubmed: 32768179
McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A, Etemadi M, Garcia-Vicente F, Gilbert FJ, Halling-Brown M, Hassabis D, Jansen S, Karthikesalingam A, Kelly CJ, King D, Ledsam JR, Melnick D, Mostofi H, Peng L, Reicher JJ, Romera-Paredes B, Sidebottom R, Suleyman M, Tse D, Young KC, De Fauw J, Shetty S (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94
doi: 10.1038/s41586-019-1799-6
pubmed: 31894144
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118
doi: 10.1038/nature21056
pubmed: 28117445
pmcid: 8382232
Misawa M, Kudo SE, Mori Y, Cho T, Kataoka S, Yamauchi A, Ogawa Y, Maeda Y, Takeda K, Ichimasa K, Nakamura H, Yagawa Y, Toyoshima N, Ogata N, Kudo T, Hisayuki T, Hayashi T, Wakamura K, Baba T, Ishida F, Itoh H, Roth H, Oda M, Mori K (2018) Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 154:2027-2029.e2023
doi: 10.1053/j.gastro.2018.04.003
pubmed: 29653147
Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T, Ohnishi T, Fujishiro M, Matsuo K, Fujisaki J, Tada T (2018) Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21:653–660
doi: 10.1007/s10120-018-0793-2
pubmed: 29335825
Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH, Navarrete-Welton A, Sankaranarayanan G, Brunt LM, Okrainec A, Alseidi A (2020) Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg. https://doi.org/10.1097/SLA.0000000000004594
doi: 10.1097/SLA.0000000000004594
pubmed: 33196488
Kumazu Y, Kobayashi N, Kitamura N, Rayan E, Neculoiu P, Misumi T, Hojo Y, Nakamura T, Kumamoto T, Kurahashi Y, Ishida Y, Masuda M, Shinohara H (2021) Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy. Sci Rep 11:21198
doi: 10.1038/s41598-021-00557-3
pubmed: 34707141
pmcid: 8551298
Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S, Ebe K, Tada K, Endo Y, Etoh T, Nakashima M, Inomata M (2021) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 35:1651–1658
doi: 10.1007/s00464-020-07548-x
pubmed: 32306111
Kitaguchi D, Takeshita N, Matsuzaki H, Igaki T, Hasegawa H, Kojima S, Mori K, Ito M (2022) Real-time vascular anatomical image navigation for laparoscopic surgery: experimental study. Surg Endosc 36:6105–6112
doi: 10.1007/s00464-022-09384-7
pubmed: 35764837
Sato Y, Sese J, Matsuyama T, Onuki M, Mase S, Okuno K, Saito K, Fujiwara N, Hoshino A, Kawada K, Tokunaga M, Kinugasa Y (2022) Preliminary study for developing a navigation system for gastric cancer surgery using artificial intelligence. Surg Today 52:1753–1758
doi: 10.1007/s00595-022-02508-5
pubmed: 35511359
Sato K, Fujita T, Matsuzaki H, Takeshita N, Fujiwara H, Mitsunaga S, Kojima T, Mori K, Daiko H (2022) Real-time detection of the recurrent laryngeal nerve in thoracoscopic esophagectomy using artificial intelligence. Surg Endosc 36:5531–5539
doi: 10.1007/s00464-022-09268-w
pubmed: 35476155
Takeuchi M, Collins T, Lipps C, Haller M, Uwineza J, Okamoto N, Nkusi R, Marescaux J, Kawakubo H, Kitagawa Y, Gonzalez C, Mutter D, Perretta S, Hostettler A, Dallemagne B (2023) Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence. Surg Endosc 37:4525–4534
doi: 10.1007/s00464-023-09934-7
pubmed: 36828887
Yamazaki Y, Kanaji S, Matsuda T, Oshikiri T, Nakamura T, Suzuki S, Hiasa Y, Otake Y, Sato Y, Kakeji Y (2020) Automated surgical instrument detection from laparoscopic gastrectomy video images using an open source convolutional neural network platform. J Am Coll Surg 230:725-732.e721
doi: 10.1016/j.jamcollsurg.2020.01.037
pubmed: 32156655
Kitaguchi D, Lee Y, Hayashi K, Nakajima K, Kojima S, Hasegawa H, Takeshita N, Mori K, Ito M (2022) Development and validation of a model for laparoscopic colorectal surgical instrument recognition using convolutional neural network-based instance segmentation and videos of laparoscopic procedures. JAMA Netw Open 5:e2226265
doi: 10.1001/jamanetworkopen.2022.26265
pubmed: 35984660
pmcid: 9391983
Kitaguchi D, Takeshita N, Matsuzaki H, Takano H, Owada Y, Enomoto T, Oda T, Miura H, Yamanashi T, Watanabe M, Sato D, Sugomori Y, Hara S, Ito M (2020) Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg Endosc 34:4924–4931
doi: 10.1007/s00464-019-07281-0
pubmed: 31797047
Shinozuka K, Turuda S, Fujinaga A, Nakanuma H, Kawamura M, Matsunobu Y, Tanaka Y, Kamiyama T, Ebe K, Endo Y, Etoh T, Inomata M, Tokuyasu T (2022) Artificial intelligence software available for medical devices: surgical phase recognition in laparoscopic cholecystectomy. Surg Endosc 36:7444–7452
doi: 10.1007/s00464-022-09160-7
pubmed: 35266049
pmcid: 9485170
Funke I, Mees ST, Weitz J, Speidel S (2019) Video-based surgical skill assessment using 3D convolutional neural networks. Int J Comput Assist Radiol Surg 14:1217–1225
doi: 10.1007/s11548-019-01995-1
pubmed: 31104257
Kitaguchi D, Takeshita N, Matsuzaki H, Igaki T, Hasegawa H, Ito M (2021) Development and validation of a 3-dimensional convolutional neural network for automatic surgical skill assessment based on spatiotemporal video analysis. JAMA Netw Open 4:e2120786
doi: 10.1001/jamanetworkopen.2021.20786
pubmed: 34387676
pmcid: 8363914
Igaki T, Kitaguchi D, Matsuzaki H, Nakajima K, Kojima S, Hasegawa H, Takeshita N, Kinugasa Y, Ito M (2023) Automatic surgical skill assessment system based on concordance of standardized surgical field development using artificial intelligence. JAMA Surg 158:e231131
doi: 10.1001/jamasurg.2023.1131
pubmed: 37285142
Kiyasseh D, Ma R, Haque TF, Miles BJ, Wagner C, Donoho DA, Anandkumar A, Hung AJ (2023) A vision transformer for decoding surgeon activity from surgical videos. Nat Biomed Eng 7:780–796
doi: 10.1038/s41551-023-01010-8
pubmed: 36997732
pmcid: 10307635
Kyungmin J, Bareum C, Songe C, Youngjin M, Jaesoon C (2016) Automatic detection of hemorrhage and surgical instrument in laparoscopic surgery image. Annu Int Conf IEEE Eng Med Biol Soc 2016:1260–1263
Garcia-Martinez A, Vicente-Samper JM, Sabater-Navarro JM (2017) Automatic detection of surgical haemorrhage using computer vision. Artif Intell Med 78:55–60
doi: 10.1016/j.artmed.2017.06.002
pubmed: 28764873
Okamoto T, Ohnishi T, Kawahira H, Dergachyava O, Jannin P, Haneishi H (2019) Real-time identification of blood regions for hemostasis support in laparoscopic surgery. SIViP 13:405–412
doi: 10.1007/s11760-018-1369-7
Mongan J, Moy L, Kahn CE Jr (2020) Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2:e200029
doi: 10.1148/ryai.2020200029
pubmed: 33937821
pmcid: 8017414
World Medical Association Declaration of Helsinki (2013) ethical principles for medical research involving human subjects. JAMA 310:2191–2194
doi: 10.1001/jama.2013.281053
Wang C-Y, Bochkovskiy A, Liao H-YM (2023) YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7464–7475
MMYOLO Contributors (2022) MMYOLO: OpenMMLab YOLO series toolbox and benchmark. https://github.com/open-mmlab/mmyolo
Microsoft Common Objects in Context; detection evaluation metrics. https://cocodataset.org/#detection-eval
Namikawa K, Hirasawa T, Nakano K, Ikenoyama Y, Ishioka M, Shiroma S, Tokai Y, Yoshimizu S, Horiuchi Y, Ishiyama A, Yoshio T, Tsuchida T, Fujisaki J, Tada T (2020) Artificial intelligence-based diagnostic system classifying gastric cancers and ulcers: comparison between the original and newly developed systems. Endoscopy 52:1077–1083
doi: 10.1055/a-1194-8771
pubmed: 32503056
Gong D, Wu L, Zhang J, Mu G, Shen L, Liu J, Wang Z, Zhou W, An P, Huang X, Jiang X, Li Y, Wan X, Hu S, Chen Y, Hu X, Xu Y, Zhu X, Li S, Yao L, He X, Chen D, Huang L, Wei X, Wang X, Yu H (2020) Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study. Lancet Gastroenterol Hepatol 5:352–361
doi: 10.1016/S2468-1253(19)30413-3
pubmed: 31981518
Wang P, Liu X, Berzin TM, Glissen Brown JR, Liu P, Zhou C, Lei L, Li L, Guo Z, Lei S, Xiong F, Wang H, Song Y, Pan Y, Zhou G (2020) Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 5:343–351
doi: 10.1016/S2468-1253(19)30411-X
pubmed: 31981517
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
Zhu X, Su W, Lu L, Li B, Wang X, Dai J (2020) Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint arXiv:201004159