Automatic detection and segmentation of chorda tympani under microscopic vision in otosclerosis patients via convolutional neural networks.
artificial intelligence
chorda tympani
deep learning
microscopic surgery
otosclerosis
Journal
The international journal of medical robotics + computer assisted surgery : MRCAS
ISSN: 1478-596X
Titre abrégé: Int J Med Robot
Pays: England
ID NLM: 101250764
Informations de publication
Date de publication:
Dec 2023
Dec 2023
Historique:
revised:
08
08
2023
received:
15
10
2022
accepted:
14
08
2023
medline:
6
11
2023
pubmed:
27
8
2023
entrez:
27
8
2023
Statut:
ppublish
Résumé
Artificial intelligence (AI) techniques, especially deep learning (DL) techniques, have shown promising results for various computer vision tasks in the field of surgery. However, AI-guided navigation during microscopic surgery for real-time surgical guidance and decision support is much more complex, and its efficacy has yet to be demonstrated. We propose a model dedicated to the evaluation of DL-based semantic segmentation of chorda tympani (CT) during microscopic surgery. Various convolutional neural networks were constructed, trained, and validated for semantic segmentation of CT. Our dataset has 5817 images annotated from 36 patients, which were further randomly split into the training set (90%, 5236 images) and validation set (10%, 581 images). In addition, 1500 raw images from 3 patients (500 images randomly selected per patient) were used to evaluate the network performance. When evaluated on a validation set (581 images), our proposed CT detection networks achieved great performance, and the modified U-net performed best (mIOU = 0.892, mPA = 0.9427). Moreover, when applying U-net to predict the test set (1500 raw images from 3 patients), our methods also showed great overall performance (Accuracy = 0.976, Precision = 0.996, Sensitivity = 0.979, Specificity = 0.902). This study suggests that DL can be used for the automated detection and segmentation of CT in patients with otosclerosis during microscopic surgery with a high degree of performance. Our research validated the potential feasibility for future vision-based navigation surgical assistance and autonomous surgery using AI.
Sections du résumé
BACKGROUND
BACKGROUND
Artificial intelligence (AI) techniques, especially deep learning (DL) techniques, have shown promising results for various computer vision tasks in the field of surgery. However, AI-guided navigation during microscopic surgery for real-time surgical guidance and decision support is much more complex, and its efficacy has yet to be demonstrated. We propose a model dedicated to the evaluation of DL-based semantic segmentation of chorda tympani (CT) during microscopic surgery.
METHODS
METHODS
Various convolutional neural networks were constructed, trained, and validated for semantic segmentation of CT. Our dataset has 5817 images annotated from 36 patients, which were further randomly split into the training set (90%, 5236 images) and validation set (10%, 581 images). In addition, 1500 raw images from 3 patients (500 images randomly selected per patient) were used to evaluate the network performance.
RESULTS
RESULTS
When evaluated on a validation set (581 images), our proposed CT detection networks achieved great performance, and the modified U-net performed best (mIOU = 0.892, mPA = 0.9427). Moreover, when applying U-net to predict the test set (1500 raw images from 3 patients), our methods also showed great overall performance (Accuracy = 0.976, Precision = 0.996, Sensitivity = 0.979, Specificity = 0.902).
CONCLUSIONS
CONCLUSIONS
This study suggests that DL can be used for the automated detection and segmentation of CT in patients with otosclerosis during microscopic surgery with a high degree of performance. Our research validated the potential feasibility for future vision-based navigation surgical assistance and autonomous surgery using AI.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
e2567Subventions
Organisme : National Key Research and Development Program of China
ID : 2019YFB1311801
Organisme : National High Level Hospital Clinical Research Funding
ID : 2022-PUMCH-B-095
Organisme : National High Level Hospital Clinical Research Funding
ID : 2022-PUMCH-A-029
Informations de copyright
© 2023 John Wiley & Sons Ltd.
Références
Mavrogenis AF, Markatos K, Saranteas T, et al. The history of microsurgery. Eur J Orthop Surg Traumatol. 2019;29(2):247-254. https://doi.org/10.1007/s00590-019-02378-7
McManus LJ, Stringer MD, Dawes PJ. Iatrogenic injury of the chorda tympani: a systematic review. J Laryngol Otol. 2012;126(1):8-14. https://doi.org/10.1017/s0022215111002039
Saito T, Manabe Y, Shibamori Y, et al. Long-term follow-up results of electrogustometry and subjective taste disorder after middle ear surgery. Laryngoscope. 2001;111(11):2064-2070. https://doi.org/10.1097/00005537-200111000-00037
Miuchi S, Sakagami M, Tsuzuki K, Noguchi K, Mishiro Y, Katsura H. Taste disturbance after stapes surgery - clinical and experimental study. Acta Otolaryngol Suppl. 2009;562(Suppl 562):71-78. https://doi.org/10.1080/00016480902915731
Guder E, Böttcher A, Pau HW, Just T. Taste function after stapes surgery. Auris Nasus Larynx. 2012;39:562-566. https://doi.org/10.1016/j.anl.2011.10.020
Ziylan F, Smeeing DPJ, Bezdjian A, Stegeman I, Thomeer HGXM. Feasibility of preservation of chorda tympani nerve during noninflammatory ear surgery: a systematic review. Laryngoscope. 2018;128(8):1904-1913. https://doi.org/10.1002/lary.26970
Choi N, Ahn J, Cho YS. Taste changes in patients with middle ear surgery by intraoperative manipulation of chorda tympani nerve. Otol Neurotol. 2018;39(5):591-596. https://doi.org/10.1097/mao.0000000000001780
Molinari G, Reale M, Bonali M, et al. Taste impairment after endoscopic stapes surgery: do anatomic variability of chorda tympani and surgical technique matter?: post-operative dysgeusia after EStS. Eur Arch Oto-Rhino-Laryngol. 2021;279(5):2269-2277. https://doi.org/10.1007/s00405-021-06908-0
Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc. 2020;92(4):807-812. https://doi.org/10.1016/j.gie.2020.06.040
Deo RC. Machine learning in medicine. Circulation. 2015;132(20):1920-1930. https://doi.org/10.1161/circulationaha.115.001593
Handelman GS, Kok HK, Chandra RV, Razavi AH, Lee MJ, Asadi H. eDoctor: machine learning and the future of medicine. J Intern Med. 2018;284(6):603-619. Epub 2018 Sep 3. PMID: 30102808. https://doi.org/10.1111/joim.12822
Sharghi A, Haugerud H, Oh D, Mohareri O. Automatic operating room surgical activity recognition for robot-assisted surgery. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2020 385-395. Springer International Publishing; 2020.
Kitaguchi D, Takeshita N, Matsuzaki H, et al. Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg Endosc. 2020;34(11):4924-4931. https://doi.org/10.1007/s00464-019-07281-0
Hashimoto DA, Rosman G, Witkowski ER, et al. Computer vision analysis of intraoperative video: auto-mated recognition of operative steps in laparoscopic sleeve gas-trectomy. Ann Surg. 2019;270(3):414-421. https://doi.org/10.1097/sla.0000000000003460
Sahu M, Mukhopadhyay A, Szengel A, Zachow S. Address-ing multi-label imbalance problem of surgical tool detection using CNN. Int J Comput Assist Radiol Surg. 2017;12(6):1013-1020. https://doi.org/10.1007/s11548-017-1565-x
Nwoye CI, Mutter D, Marescaux J, Padoy N. Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg. 2019;14(6):1059-1067. https://doi.org/10.1007/s11548-019-01958-6
Tokuyasu T, Iwashita Y, Matsunobu Y, et al. Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc. 2020;35(4):1651-1658. https://doi.org/10.1007/s00464-020-07548-x
Madad Zadeh S, Francois T, Calvet L, et al. SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology. Surg Endosc. 2020;34(12):5377-5383. https://doi.org/10.1007/s00464-019-07330-8
Madani A, Namazi B, Altieri MS, et al. Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg. 2020;276(2):363-369. https://doi.org/10.1097/SLA.0000000000004594
Koskinen J, Torkamani-Azar M, Hussein A, Huotarinen A, Bednarik R. Automated tool detection with deep learning for monitoring kinematics and eye-hand coordination in microsurgery. Comput Biol Med. 2022;141:105121. Epub 2021 Dec 11. PMID: 34968859. https://doi.org/10.1016/j.compbiomed.2021.105121
Wang G, Kikuchi Y, Yi J, Zou Q, Zhou R, Guo X. Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematurity; 2022. arxiv:2201.01250[cs.LG,cs.CV].
Marín R, Chang V. Impact of transfer learning for human sperm segmentation using deep learning. Comput Biol Med. 2021;136:104687. Epub 2021 Jul 29. PMID: 34364259. https://doi.org/10.1016/j.compbiomed.2021.104687
He K, Girshick R, Dollár P. Rethinking ImageNet Pre-training; 2018. arxiv:1811.08883[cs.CV].
Zeiler Matthew D, Fergus R. Visualizing and Understanding Convolutional Networks; 2013. arxiv:1311.2901[cs.CV].
Selvaraju Ramprasaath R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization; 2019. arxiv:1610.02391[cs.CV,cs.AI,cs.LG].
Chattopadhyay A, Sarkar A, Prantik H, Balasubramanian VN. Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks; 2018. arxiv:1710.11063[cs.CV].
Chen L.-C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation; 2018. arxiv:1802.02611[cs.CV].
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L.-C. MobileNetV2: Inverted Residuals and Linear Bottlenecks; 2019. arxiv:1801.04381[cs.CV].
Ronneberger O, Fischer P, Brox T. U-net: Convolutional Networks for Biomedical Image Segmentation; 2015. arxiv:1505.04597[cs.CV].
Esteva A, Kuprel B, Novoa R, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. https://doi.org/10.1038/nature21056
Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402-2410. https://doi.org/10.1001/jama.2016.17216
Yu L, Chen H, Dou Q, Qin J, Heng PA. Integrating online and offline three dimensional deep learning for automated polyp detection in colonoscopyvideos. IEEE J Biomed Health Inf. 2017;21(1):65-75. https://doi.org/10.1109/jbhi.2016.2637004
Eminaga O, Eminaga N, Semjonow A, Breil B. Diagnostic classification of cystoscopic images using deep convolutional neural networks. JCO Clin Cancer Inf. 2018;2:1-8. https://doi.org/10.1200/cci.17.00126
Bamba Y, Ogawa S, Itabashi M, et al. Object and anatomical feature recognition in surgical video images based on a convolutional neural network. Int J Comput Assist Radiol Surg. 2021;16(11):2045-2054. https://doi.org/10.1007/s11548-021-02434-w
Bano S, Casella A, Vasconcelos F, et al. FetReg: Placental Vessel Segmentation and Registration in Fetoscopy Challenge Dataset; 2021. arXiv preprint arXiv: 2106.05923v0592.
Adamian N, Naunheim MR, Jowett N. An open-source computer vision tool for automated vocal fold tracking from videoendoscopy. Laryngoscope. 2021;131(1):E219-E225. https://doi.org/10.1002/lary.28669
Jiang Z, Lei Y, Zhang L, et al. Automated quantitative analysis of blood flow in extracranial-intracranial arterial bypass based on indocyanine green angiography. Front Surg. 2021;8:649719. https://doi.org/10.3389/fsurg.2021.649719
Song Y, Hu J, Wu Q, et al. Automatic delineation of the clinical target volume and organs at risk by deep learning for rectal cancer postoperative radiotherapy. Radiother Oncol. 2020;145:186-192. https://doi.org/10.1016/j.radonc.2020.01.020
Wang Z, Zhong Y, Yao M, et al. Automated segmentation of macular edema for the diagnosis of ocular disease using deep learning method. Sci Rep. 2021;11(1):13392. https://doi.org/10.1038/s41598-021-92458-8
Russell BC, Torralba A, Murphy KP, Freeman WT. LabelMe: a database and web-based tool for image. Int J Comput Vis. 2005;77(1).
Ikeda A, Nosato H, Kochi Y, et al. Support system of cystoscopic diagnosis for bladder cancer based on artificial intelligence. J Endourol. 2020;34(3):352-358. https://doi.org/10.1089/end.2019.0509
Sarker IH. Deep learning: a comprehensive Overview on techniques, taxonomy, applications and research directions. SN Comput Sci. 2021;2(6):420. Epub 2021 Aug 18. PMID: 34426802; PMCID: PMC8372231. https://doi.org/10.1007/s42979-021-00815-1
Barata C, Celebi ME, Marques JS. Explainable skin lesion diagnosis using taxonomies. Pattern Recognit. 2021;110:107413. https://doi.org/10.1016/j.patcog.2020.107413
Yap MH, Hachiuma R, Alavi A, et al. Deep learning in diabetic foot ulcers detection: a comprehensive evaluation. Comput Biol Med. 2021;135:104596. https://doi.org/10.1016/j.compbiomed.2021.104596