Artificial Intelligence for Surgical Safety: Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy Using Deep Learning.
Journal
Annals of surgery
ISSN: 1528-1140
Titre abrégé: Ann Surg
Pays: United States
ID NLM: 0372354
Informations de publication
Date de publication:
01 05 2022
01 05 2022
Historique:
pubmed:
18
11
2020
medline:
14
5
2022
entrez:
17
11
2020
Statut:
ppublish
Résumé
To develop a deep learning model to automatically segment hepatocystic anatomy and assess the criteria defining the critical view of safety (CVS) in laparoscopic cholecystectomy (LC). Poor implementation and subjective interpretation of CVS contributes to the stable rates of bile duct injuries in LC. As CVS is assessed visually, this task can be automated by using computer vision, an area of artificial intelligence aimed at interpreting images. Still images from LC videos were annotated with CVS criteria and hepatocystic anatomy segmentation. A deep neural network comprising a segmentation model to highlight hepatocystic anatomy and a classification model to predict CVS criteria achievement was trained and tested using 5-fold cross validation. Intersection over union, average precision, and balanced accuracy were computed to evaluate the model performance versus the annotated ground truth. A total of 2854 images from 201 LC videos were annotated and 402 images were further segmented. Mean intersection over union for segmentation was 66.6%. The model assessed the achievement of CVS criteria with a mean average precision and balanced accuracy of 71.9% and 71.4%, respectively. Deep learning algorithms can be trained to reliably segment hepatocystic anatomy and assess CVS criteria in still laparoscopic images. Surgical-technical partnerships should be encouraged to develop and evaluate deep learning models to improve surgical safety.
Sections du résumé
OBJECTIVE
To develop a deep learning model to automatically segment hepatocystic anatomy and assess the criteria defining the critical view of safety (CVS) in laparoscopic cholecystectomy (LC).
BACKGROUND
Poor implementation and subjective interpretation of CVS contributes to the stable rates of bile duct injuries in LC. As CVS is assessed visually, this task can be automated by using computer vision, an area of artificial intelligence aimed at interpreting images.
METHODS
Still images from LC videos were annotated with CVS criteria and hepatocystic anatomy segmentation. A deep neural network comprising a segmentation model to highlight hepatocystic anatomy and a classification model to predict CVS criteria achievement was trained and tested using 5-fold cross validation. Intersection over union, average precision, and balanced accuracy were computed to evaluate the model performance versus the annotated ground truth.
RESULTS
A total of 2854 images from 201 LC videos were annotated and 402 images were further segmented. Mean intersection over union for segmentation was 66.6%. The model assessed the achievement of CVS criteria with a mean average precision and balanced accuracy of 71.9% and 71.4%, respectively.
CONCLUSIONS
Deep learning algorithms can be trained to reliably segment hepatocystic anatomy and assess CVS criteria in still laparoscopic images. Surgical-technical partnerships should be encouraged to develop and evaluate deep learning models to improve surgical safety.
Identifiants
pubmed: 33201104
pii: 00000658-202205000-00020
doi: 10.1097/SLA.0000000000004351
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
955-961Informations de copyright
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
The authors declare no conflict of interest.
Références
Tornqvist B, Stromberg C, Persson G, et al. Effect of intended intraoperative cholangiography and early detection of bile duct injury on survival after cholecystectomy: population based cohort study. BMJ 2012; 345:e6457.
Alkhaffaf B, Decadt B. 15 years of litigation following laparoscopic cholecystectomy in England. Ann Surg 2010; 251:682–685.
Berci G, Hunter J, Morgenstern L, et al. Laparoscopic cholecystectomy: first, do no harm; second, take care of bile duct stones. Surg Endosc 2013; 27:1051–1054.
Strasberg SM, Hertl M, Soper NJ. An analysis of the problem of biliary injury during laparoscopic cholecystectomy. J Am Coll Surg 1995; 180:101–125.
Way LW, Stewart L, Gantert W, et al. Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg 2003; 237:460–469.
Pucher PH, Brunt LM, Fanelli RD, et al. SAGES expert Delphi consensus: critical factors for safe surgical practice in laparoscopic cholecystectomy. Surg Endose 2015; 29:3074–3085.
Wakabayashi G, Iwashita Y, Hibi T, et al. Tokyo guidelines 2018: surgical management of acute cholecystitis: safe steps in laparoscopic cholecystectomy for acute cholecystitis (with videos). J Hepatobiliary Pancreat Sci 2018; 25:73–86.
Conrad C, Wakabayashi G, Asbun HJ, et al. IRCAD recommendation on safe laparoscopic cholecystectomy. J Hepatobiliary Pancreat Sci 2017; 24:603–615.
Pucher PH, Brunt LM, Davies N, et al. Outcome trends and safety measures after 30 years of laparoscopic cholecystectomy: a systematic review and pooled data analysis. Surg Endosc 2018; 32:2175–2183.
Massarweh NN, Devlin A, Symons RG, et al. Risk tolerance and bile duct injury: surgeon characteristics, risk-taking preference, and common bile duct injuries. J Am Coll Surg 2009; 209:17–24.
Nijssen MA, Schreinemakers JM, Meyer Z, et al. Complications after laparoscopic cholecystectomy: a video evaluation study of whether the critical view of safety was reached. World J Surg 2015; 39:1798–1803.
Rawlings A, Hodgett SE, Matthews BD, et al. Single-incision laparoscopic cholecystectomy: initial experience with critical view of safety dissection and routine intraoperative cholangiography. J Am Coll Surg 2010; 211:1–7.
Stefanidis D, Chintalapudi N, Anderson-Montoya B, et al. How often do surgeons obtain the critical view of safety during laparoscopic cholecystectomy? Surg Endosc 2017; 31:142–146.
Maier-Hein L, Vedula SS, Speidel S, et al. Surgical data science for next-generation interventions. Nat Biomed Eng 2017; 1:691–696.
Vercauteren T, Unberath M, Padoy N, et al. CAI4CAI: the rise of contextual artificial intelligence in computer assisted interventions. Proc IEEE Inst Electr Electron Eng 2020; 108:198–214.
Mascagni P, Fiorillo C, Urade T, et al. Formalizing video documentation of the Critical View of Safety in laparoscopic cholecystectomy: a step towards artificial intelligence assistance to improve surgical safety. Surg Endosc 2019; 34:2709–2714.
Breheret A. Pixel Annotation Tool 2017. Available at: https://github.com/abreheret/PixelAnnotationTool . Accessed April 6, 2020.
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42:60–88.
Chen L-C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on ComputerVision (ECCV); 2018:801–818.
Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition: IEEE; 2009: 248–255.
Lin T-Y, Maire M, Belongie S, et al. Microsoft coco: Common Objects in Context. European Conference on Computer Vision . Springer; 2014, 740–755.
Everingham M, Eslami SA, Van Gool L, et al. The pascal visual object classes challenge: a retrospective. Int J Comput Vis 2015; 111:98–136.
Kapishnikov A, Bolukbasi T, Viegas F, et al. XRAI: better attributions through regions. Proceedings of the IEEE International Conference on Computer Vision; 2019:4948–4957.
Jung JJ, Juni P, Lebovic G, et al. First-year analysis of the operating room black box study. Ann Surg 2020; 271:122–127.
Padoy N. Machine and deep learning for workflow recognition during surgery. Minim Invasive Ther Allied Technol 2019; 28:82–90.
Hashimoto DA, Rosman G, Witkowski ER, et al. Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 2019; 270:414–421.
Adams-McGavin RC, Jung JJ, van Dalen A, et al. System factors affecting patient safety in the OR: an analysis of safety threats and resiliency. Ann Surg 2021; 274:114–119.
Yu F, Silva Croso G, Kim TS, et al. Assessment of automated identification of phases in videos of cataract surgery using machine learning and deep learning techniques. JAMANetw Open 2019; 2:e191860.
Twinanda AP, Yengera G, Mutter D, et al. RSDNet: learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Trans Med Imaging 2019; 38:1069–1078.
Wahr JA, Prager RL, Abernathy JH 3rd, et al. Patient safety in the cardiac operating room: human factors and teamwork: a scientific statement from the American Heart Association. Circulation 2013; 128:1139–1169.
Peponis T, Baekgaard JS, Bohnen JD, et al. Are surgeons reluctant to accurately report intraoperative adverse events? A prospective study of 1,989 patients. Surgery 2018; 164:525–529.
Bohnen JD, Mavros MN, Ramly EP, et al. Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room. Ann Surg 2017; 265:1119–1125.
Ramly EP, Bohnen JD, Farhat MR, et al. The nature, patterns, clinical outcomes, and financial impact of intraoperative adverse events in emergency surgery. Am J Surg 2016; 212:16–23.
Fletcher R, Cortina CS, Kornfield H, et al. Bile duct injuries: a contemporary survey of surgeon attitudes and experiences. Surg Endosc 2019; 34:3079–3084.
Niemann AC, Matusko N, Sandhu G, et al. Assessing the effect of the critical view of safety criteria on simulated operative decision-making: a pilot study. Surg Endosc 2019; 33:911–916.
Chen CB, Palazzo F, Doane SM, et al. Increasing resident utilization and recognition of the critical view of safety during laparoscopic cholecystectomy: a pilot study from an academic medical center. Surg Endosc 2017; 31:1627–1635.
Springer, Pakhomov D, Premachandran V, Allan M, et al. Deep Residual Learning for Instrument Segmentation in Robotic Surgery. International Workshop on Machine Learning in Medical Imaging. 2019; 566–573.
Sanford DE, Strasberg SM. A simple effective method for generation of a permanent record of the Critical View of Safety during laparoscopic cholecystectomy by intraoperative “doublet” photography. J Am Coll Surg 2014; 218:170–178.
Strasberg SM, Brunt LM. Rationale and use of the critical view of safety in laparoscopic cholecystectomy. J Am Coll Surg 2010; 211:132–138.
Emous M, Westerterp M, Wind J, et al. Registering the critical view of safety: photo or video? Surg Endosc 2010; 24:2527–2530.