A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos: Documenting Safety in Laparoscopic Cholecystectomy.
Journal
Annals of surgery
ISSN: 1528-1140
Titre abrégé: Ann Surg
Pays: United States
ID NLM: 0372354
Informations de publication
Date de publication:
01 07 2021
01 07 2021
Historique:
pubmed:
9
1
2021
medline:
6
8
2021
entrez:
8
1
2021
Statut:
ppublish
Résumé
The aim of this study was to develop a computer vision platform to automatically locate critical events in surgical videos and provide short video clips documenting the critical view of safety (CVS) in laparoscopic cholecystectomy (LC). Intraoperative events are typically documented through operator-dictated reports that do not always translate the operative reality. Surgical videos provide complete information on surgical procedures, but the burden associated with storing and manually analyzing full-length videos has so far limited their effective use. A computer vision platform named EndoDigest was developed and used to analyze LC videos. The mean absolute error (MAE) of the platform in automatically locating the manually annotated time of the cystic duct division in full-length videos was assessed. The relevance of the automatically extracted short video clips was evaluated by calculating the percentage of video clips in which the CVS was assessable by surgeons. A total of 155 LC videos were analyzed: 55 of these videos were used to develop EndoDigest, whereas the remaining 100 were used to test it. The time of the cystic duct division was automatically located with a MAE of 62.8 ± 130.4 seconds (1.95% of full-length video duration). CVS was assessable in 91% of the 2.5 minutes long video clips automatically extracted from the considered test procedures. Deep learning models for workflow analysis can be used to reliably locate critical events in surgical videos and document CVS in LC. Further studies are needed to assess the clinical impact of surgical data science solutions for safer laparoscopic cholecystectomy.
Sections du résumé
OBJECTIVE
The aim of this study was to develop a computer vision platform to automatically locate critical events in surgical videos and provide short video clips documenting the critical view of safety (CVS) in laparoscopic cholecystectomy (LC).
BACKGROUND
Intraoperative events are typically documented through operator-dictated reports that do not always translate the operative reality. Surgical videos provide complete information on surgical procedures, but the burden associated with storing and manually analyzing full-length videos has so far limited their effective use.
METHODS
A computer vision platform named EndoDigest was developed and used to analyze LC videos. The mean absolute error (MAE) of the platform in automatically locating the manually annotated time of the cystic duct division in full-length videos was assessed. The relevance of the automatically extracted short video clips was evaluated by calculating the percentage of video clips in which the CVS was assessable by surgeons.
RESULTS
A total of 155 LC videos were analyzed: 55 of these videos were used to develop EndoDigest, whereas the remaining 100 were used to test it. The time of the cystic duct division was automatically located with a MAE of 62.8 ± 130.4 seconds (1.95% of full-length video duration). CVS was assessable in 91% of the 2.5 minutes long video clips automatically extracted from the considered test procedures.
CONCLUSIONS
Deep learning models for workflow analysis can be used to reliably locate critical events in surgical videos and document CVS in LC. Further studies are needed to assess the clinical impact of surgical data science solutions for safer laparoscopic cholecystectomy.
Identifiants
pubmed: 33417329
doi: 10.1097/SLA.0000000000004736
pii: 00000658-202107000-00048
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Video-Audio Media
Langues
eng
Sous-ensembles de citation
IM
Pagination
e93-e95Commentaires et corrections
Type : CommentIn
Type : CommentIn
Type : CommentIn
Informations de copyright
Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
The authors report no conflicts of interest.
Références
Curtis NJ, Foster JD, Miskovic D, et al. Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 2020; 155:590–598.
Pugh CM, Hashimoto DA, Korndirffer JR, et al. The what? How? And who? Of video based assessment. Am J Surg 2020; 221:13–18.
Brunt LM, Deziel DJ, Telem DA, et al. Safe cholecystectomy multi-society practice guideline and state of the art consensus conference on prevention of bile duct injury during cholecystectomy. Ann Surg 2020; 272:3–23.
Mascagni P, Fiorillo C, Urade T, et al. Formalizing video documentation of the Critical View of Safety in laparoscopic cholecystectomy: a step towards artificial intelligence assistance to improve surgical safety. Surg Endosc 2019; 34:2709–2714.
Yu T, Mutter D, Marescaux J, et al. Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition. arXiv:181200033 [cs, stat]. Available at: http://arxiv.org/abs/1812.00033 . 2019. Accessed May 5, 2020.
Nwoye CI, Mutter D, Marescaux J, et al. Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg 2019; 14:1059–1067.
Nijssen MAJ, Schreinemakers JMJ, Meyer Z, et al. Complications after laparoscopic cholecystectomy: a video evaluation study of whether the critical view of safety was reached. World J Surg 2015; 39:1798–1803.
Hashimoto DA, Rosman G, Witkowski ER, et al. Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 2019; 270:414–421.
Kitaguchi D, Takeshita N, Matsuzaki H, et al. Automated Laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg 2020; 79:88–94.
Mascagni D, Vardazaryan A, Alapatt D, et al. Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg 2020; doi: 10.1097/SLA.0000000000004351. [Ahead of Print].
doi: 10.1097/SLA.0000000000004351.