Artificial intelligence for phase recognition in complex laparoscopic cholecystectomy.
Artificial Intelligence
Cholecystectomy
Computer Vision
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
12 2022
12 2022
Historique:
received:
01
12
2021
accepted:
19
06
2022
pubmed:
9
8
2022
medline:
16
11
2022
entrez:
8
8
2022
Statut:
ppublish
Résumé
The potential role and benefits of AI in surgery has yet to be determined. This study is a first step in developing an AI system for minimizing adverse events and improving patient's safety. We developed an Artificial Intelligence (AI) algorithm and evaluated its performance in recognizing surgical phases of laparoscopic cholecystectomy (LC) videos spanning a range of complexities. A set of 371 LC videos with various complexity levels and containing adverse events was collected from five hospitals. Two expert surgeons segmented each video into 10 phases including Calot's triangle dissection and clipping and cutting. For each video, adverse events were also annotated when present (major bleeding; gallbladder perforation; major bile leakage; and incidental finding) and complexity level (on a scale of 1-5) was also recorded. The dataset was then split in an 80:20 ratio (294 and 77 videos), stratified by complexity, hospital, and adverse events to train and test the AI model, respectively. The AI-surgeon agreement was then compared to the agreement between surgeons. The mean accuracy of the AI model for surgical phase recognition was 89% [95% CI 87.1%, 90.6%], comparable to the mean inter-annotator agreement of 90% [95% CI 89.4%, 90.5%]. The model's accuracy was inversely associated with procedure complexity, decreasing from 92% (complexity level 1) to 88% (complexity level 3) to 81% (complexity level 5). The AI model successfully identified surgical phases in both simple and complex LC procedures. Further validation and system training is warranted to evaluate its potential applications such as to increase patient safety during surgery.
Sections du résumé
BACKGROUND
The potential role and benefits of AI in surgery has yet to be determined. This study is a first step in developing an AI system for minimizing adverse events and improving patient's safety. We developed an Artificial Intelligence (AI) algorithm and evaluated its performance in recognizing surgical phases of laparoscopic cholecystectomy (LC) videos spanning a range of complexities.
METHODS
A set of 371 LC videos with various complexity levels and containing adverse events was collected from five hospitals. Two expert surgeons segmented each video into 10 phases including Calot's triangle dissection and clipping and cutting. For each video, adverse events were also annotated when present (major bleeding; gallbladder perforation; major bile leakage; and incidental finding) and complexity level (on a scale of 1-5) was also recorded. The dataset was then split in an 80:20 ratio (294 and 77 videos), stratified by complexity, hospital, and adverse events to train and test the AI model, respectively. The AI-surgeon agreement was then compared to the agreement between surgeons.
RESULTS
The mean accuracy of the AI model for surgical phase recognition was 89% [95% CI 87.1%, 90.6%], comparable to the mean inter-annotator agreement of 90% [95% CI 89.4%, 90.5%]. The model's accuracy was inversely associated with procedure complexity, decreasing from 92% (complexity level 1) to 88% (complexity level 3) to 81% (complexity level 5).
CONCLUSION
The AI model successfully identified surgical phases in both simple and complex LC procedures. Further validation and system training is warranted to evaluate its potential applications such as to increase patient safety during surgery.
Identifiants
pubmed: 35941306
doi: 10.1007/s00464-022-09405-5
pii: 10.1007/s00464-022-09405-5
pmc: PMC9652206
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
9215-9223Informations de copyright
© 2022. The Author(s).
Références
Fletcher DR, Hobbs MST, Tan P et al (1999) Complications of cholecystectomy: risks of the laparoscopic approach and protective effects of operative cholangiography. Ann Surg 229:449–457
doi: 10.1097/00000658-199904000-00001
pubmed: 10203075
pmcid: 1191728
Deziel DJ (1994) Complications of cholecystectomy: incidence, clinical manifestations, and diagnosis. Surg Clin North Am 74:809–823
doi: 10.1016/S0039-6109(16)46382-6
pubmed: 8047943
Brunt LM et al (2020) Safe cholecystectomy multi-society practice guideline and state-of-the-art consensus conference on prevention of bile duct injury during cholecystectomy. Surg Endosc 34:2827–2855
doi: 10.1007/s00464-020-07568-7
Schwaitzberg SD, Scott DJ, Jones DB et al (2014) Threefold increased bile duct injury rate is associated with less surgeon experience in an insurance claims database. Surg Endosc 28:3068–3073
doi: 10.1007/s00464-014-3580-0
pubmed: 24902815
Törnqvist B, Strömberg C, Akre O et al (2015) Selective intraoperative cholangiography and risk of bile duct injury during cholecystectomy. Br J Surg 102:952–958
doi: 10.1002/bjs.9832
pubmed: 25919401
Lilley EJ, Scott JW, Jiang W et al (2017) Intraoperative cholangiography during cholecystectomy among hospitalized medicare beneficiaries with non-neoplastic biliary disease. Am J Surg 214:682–686
doi: 10.1016/j.amjsurg.2017.06.021
pubmed: 28669532
Barrett M, Asbun HJ, Chien H-L et al (2018) Bile duct injury and morbidity following cholecystectomy: a need for improvement. Surg Endosc 32:1683–1688
doi: 10.1007/s00464-017-5847-8
pubmed: 28916877
Pucher PH et al (2018) Outcome trends and safety measures after 30 years of laparoscopic cholecystectomy: a systematic review and pooled data analysis. Surg Endosc 32:2175–2183
doi: 10.1007/s00464-017-5974-2
pubmed: 29556977
pmcid: 5897463
Fong ZV, Pitt HA, Strasberg SM et al (2018) Diminished survival in patients with bile leak and ductal injury: management strategy and outcomes. J Am Coll Surg 226:568-576.e1
doi: 10.1016/j.jamcollsurg.2017.12.023
pubmed: 29307612
pmcid: 6053912
Strasberg SM, Brunt LM (2010) Rationale and use of the critical view of safety in laparoscopic cholecystectomy. J Am Coll Surg 211:132–138
doi: 10.1016/j.jamcollsurg.2010.02.053
pubmed: 20610259
Way LW, Stewart L, Gantert W et al (2003) Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg 237:460–469
doi: 10.1097/01.SLA.0000060680.92690.E9
pubmed: 12677139
pmcid: 1514483
Twinanda AP, Shehata S, Mutter D et al (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36:86–97
doi: 10.1109/TMI.2016.2593957
pubmed: 27455522
Bar O, Neimark D, Zohar M et al (2020) Impact of data on generalization of AI for surgical intelligence applications. Sci Rep 10:22208
doi: 10.1038/s41598-020-79173-6
pubmed: 33335191
pmcid: 7747564
Mascagni P, Vardazaryan A, Alapatt D et al (2020) Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg. https://doi.org/10.1097/SLA.0000000000004351
doi: 10.1097/SLA.0000000000004351
pubmed: 33201104
Madni TD, Leshikar DE, Minshall CT et al (2018) The Parkland grading scale for cholecystitis. Am J Surg 215:625–630
doi: 10.1016/j.amjsurg.2017.05.017
pubmed: 28619262
Madni TD, Nakonezny PA, Barrios E et al (2019) Prospective validation of the Parkland grading scale for cholecystitis. Am J Surg 217:90–97
doi: 10.1016/j.amjsurg.2018.08.005
pubmed: 30190078
Rastegari M, Ordonez V, Redmon J et al (2016) XNOR-Net: ImageNet classification using binary convolutional neural networks. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision—ECCV 2016. Springer International Publishing, pp 525–542
doi: 10.1007/978-3-319-46493-0_32
Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29:2352–2449
doi: 10.1162/neco_a_00990
pubmed: 28599112
Ciresan DC, Meier U, Masci J et al (2011) Flexible, high performance convolutional neural networks for image classification. IJCAI. https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-210
doi: 10.5591/978-1-57735-516-8/IJCAI11-210
Li Q, Cai W, Wang X, et al. (2014) Medical image classification with convolutional neural network. In 2014 13th International Conference on Control Automation Robotics Vision (ICARCV), pp. 844–848.
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. IEEE Conf Comput Vision Pattern Recognition (CVPR) 2016:770–778
Farha YA, Gall J (2019) MS-TCN: multi-stage temporal convolutional network for action segmentation. IEEE/CVF Conf Comput Vision Pattern Recognition (CVPR) 2019:3575–3584
van den Oord A, Dieleman S, Zen H, et al. (2016) WaveNet: a generative model for raw audio. arXiv [cs.SD]. http://arxiv.org/abs/1609.03499
Stein S, McKenna SJ (2013) Combining embedded accelerometers with computer vision for recognizing food preparation activities. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 729–738.
Kuehne H, Arslan A, Serre T (2014) The language of actions: recovering the syntax and semantics of goal-directed human activities. 2014 IEEE Conf Comput Vision Pattern Recognition. https://doi.org/10.1109/cvpr.2014.105
doi: 10.1109/cvpr.2014.105
Czempiel T, Paschali M, Keicher M, et al. (2020) TeCNO: surgical phase recognition with multi-stage temporal convolutional networks. arXiv [eess.IV]. http://arxiv.org/abs/2003.10751
Tanwani AK, Sermanet P, Yan A, et al. (2020) Motion2Vec: semi-supervised representation learning from surgical videos. IEEE International Conference on Robotics and Automation (ICRA), pp. 2174–2181.
Strömblad CT, Baxter-King RG, Meisami A et al (2021) Effect of a predictive model on planned surgical duration accuracy, patient wait time, and use of presurgical resources: a randomized clinical trial. JAMA Surg. https://doi.org/10.1001/jamasurg.2020.6361
doi: 10.1001/jamasurg.2020.6361
pubmed: 34259825
pmcid: 7841577
Strasberg SM, Hertl M, Soper NJ (1995) An analysis of the problem of biliary injury during laparoscopic cholecystectomy. J Am Coll Surg 180:101–125
pubmed: 8000648
McBee MP, Awan OA, Colucci AT et al (2018) Deep learning in radiology. Acad Radiol 25:1472–1480
doi: 10.1016/j.acra.2018.02.018
pubmed: 29606338
Choy G, Khalilzadeh O, Michalski M et al (2018) Current applications and future impact of machine learning in radiology. Radiology 288:318–328
doi: 10.1148/radiol.2018171820
pubmed: 29944078
Madabhushi A, Lee G (2016) Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal 33:170–175
doi: 10.1016/j.media.2016.06.037
pubmed: 27423409
pmcid: 5556681
Freedman D, Blau Y, Katzir L et al (2020) Detecting deficient coverage in colonoscopies. IEEE Trans Med Imaging 39:3451–3462
doi: 10.1109/TMI.2020.2994221
pubmed: 32746092
Jin A, Yeung S, Jopling J, et al. (2018) Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. arXiv [cs.CV]. http://arxiv.org/abs/1802.08774
Al Hajj H, Lamard M, Charriere K et al (2017) Surgical tool detection in cataract surgery videos through multi-image fusion inside a convolutional neural network. IEEE Eng Med Biol Soc 2017:2002–2005
Jin Y, Dou Q, Chen H et al (2018) SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network. IEEE Trans Med Imaging 37:1114–1126
doi: 10.1109/TMI.2017.2787657
pubmed: 29727275
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90
doi: 10.1145/3065386