A novel machine learning-based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images.
Artificial intelligence
Colon diseases
Colonoscopy
Computer-aided diagnosis system
Heat map
convolution neural network
Journal
Surgical endoscopy
ISSN: 1432-2218
Titre abrégé: Surg Endosc
Pays: Germany
ID NLM: 8806653
Informations de publication
Date de publication:
01 2022
01 2022
Historique:
received:
07
05
2020
accepted:
13
01
2021
pubmed:
17
2
2021
medline:
3
3
2022
entrez:
16
2
2021
Statut:
ppublish
Résumé
Computer-aided diagnosis (CAD)-based artificial intelligence (AI) has been shown to be highly accurate for detecting and characterizing colon polyps. However, the application of AI to identify normal colon landmarks and differentiate multiple colon diseases has not yet been established. We aimed to develop a convolutional neural network (CNN)-based algorithm (GUTAID) to recognize different colon lesions and anatomical landmarks. Colonoscopic images were obtained to train and validate the AI classifiers. An independent dataset was collected for verification. The architecture of GUTAID contains two major sub-models: the Normal, Polyp, Diverticulum, Cecum and CAncer (NPDCCA) and Narrow-Band Imaging for Adenomatous/Hyperplastic polyps (NBI-AH) models. The development of GUTAID was based on the 16-layer Visual Geometry Group (VGG16) architecture and implemented on Google Cloud Platform. In total, 7838 colonoscopy images were used for developing and validating the AI model. An additional 1273 images were independently applied to verify the GUTAID. The accuracy for GUTAID in detecting various colon lesions/landmarks is 93.3% for polyps, 93.9% for diverticula, 91.7% for cecum, 97.5% for cancer, and 83.5% for adenomatous/hyperplastic polyps. A CNN-based algorithm (GUTAID) to identify colonic abnormalities and landmarks was successfully established with high accuracy. This GUTAID system can further characterize polyps for optical diagnosis. We demonstrated that AI classification methodology is feasible to identify multiple and different colon diseases.
Identifiants
pubmed: 33591447
doi: 10.1007/s00464-021-08331-2
pii: 10.1007/s00464-021-08331-2
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
640-650Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature.
Références
Wolff WI, Shinya H (1974) Earlier diagnosis of cancer of the colon through colonic endoscopy (colonoscopy). Cancer 34(Suppl):912–931
doi: 10.1002/1097-0142(197409)34:3+<912::AID-CNCR2820340720>3.0.CO;2-P
Wolf AMD, Fontham ETH, Church TR, Flowers CR, Guerra CE, LaMonte SJ, Etzioni R, McKenna MT, Oeffinger KC, Shih YT, Walter LC, Andrews KS, Brawley OW, Brooks D, Fedewa SA, Manassaram-Baptiste D, Siegel RL, Wender RC, Smith RA (2018) Colorectal cancer screening for average-risk adults: 2018 guideline update from the American Cancer Society. CA Cancer J Clin 68:250–281
doi: 10.3322/caac.21457
Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56
doi: 10.1038/s41591-018-0300-7
Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H (2018) Artificial intelligence in radiology. Nat Rev Cancer 18:500–510
doi: 10.1038/s41568-018-0016-5
Hogarty DT, Su JC, Phan K, Attia M, Hossny M, Nahavandi S, Lenane P, Moloney FJ, Yazdabadi A (2020) Artificial intelligence in dermatology—where we are and the way to the future: a review. Am J Clin Dermatol 21:41–47
doi: 10.1007/s40257-019-00462-6
Kudo SE, Mori Y, Misawa M, Takeda K, Kudo T, Itoh H, Oda M, Mori K (2019) Artificial intelligence and colonoscopy: current status and future perspectives. Dig Endosc 31:363–371
doi: 10.1111/den.13340
Misawa M, Kudo SE, Mori Y, Cho T, Kataoka S, Yamauchi A, Ogawa Y, Maeda Y, Takeda K, Ichimasa K, Nakamura H, Yagawa Y, Toyoshima N, Ogata N, Kudo T, Hisayuki T, Hayashi T, Wakamura K, Baba T, Ishida F, Itoh H, Roth H, Oda M, Mori K (2018) Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 154:2027-2029.e2023
doi: 10.1053/j.gastro.2018.04.003
Becq A, Chandnani M, Bharadwaj S, Baran B, Ernest-Suarez K, Gabr M, Glissen-Brown J, Sawhney M, Pleskow DK, Berzin TM (2020) Effectiveness of a deep-learning polyp detection system in prospectively collected colonoscopy videos with variable bowel preparation quality. J Clin Gastroenterol 54:554–557
doi: 10.1097/MCG.0000000000001272
Wang P, Berzin TM, Glissen Brown JR, Bharadwaj S, Becq A, Xiao X, Liu P, Li L, Song Y, Zhang D, Li Y, Xu G, Tu M, Liu X (2019) Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 68:1813–1819
doi: 10.1136/gutjnl-2018-317500
Gong D, Wu L, Zhang J, Mu G, Shen L, Liu J, Wang Z, Zhou W, An P, Huang X, Jiang X, Li Y, Wan X, Hu S, Chen Y, Hu X, Xu Y, Zhu X, Li S, Yao L, He X, Chen D, Huang L, Wei X, Wang X, Yu H (2020) Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study. Lancet Gastroenterol Hepatol 5:352–361
doi: 10.1016/S2468-1253(19)30413-3
Berzin TM, Topol EJ (2020) Adding artificial intelligence to gastrointestinal endoscopy. Lancet 395:485
doi: 10.1016/S0140-6736(20)30294-4
Chen PJ, Lin MC, Lai MJ, Lin JC, Lu HH, Tseng VS (2018) Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology 154:568–575
doi: 10.1053/j.gastro.2017.10.010
Byrne MF, Chapados N, Soudan F, Oertel C, Linares Perez M, Kelly R, Iqbal N, Chandelier F, Rex DK (2019) Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 68:94–100
doi: 10.1136/gutjnl-2017-314547
Kudo SE, Misawa M, Mori Y, Hotta K, Ohtsuka K, Ikematsu H, Saito Y, Takeda K, Nakamura H, Ichimasa K, Ishigaki T, Toyoshima N, Kudo T, Hayashi T, Wakamura K, Baba T, Ishida F, Inoue H, Itoh H, Oda M, Mori K (2020) Artificial intelligence-assisted system improves endoscopic identification of colorectal neoplasms. Clin Gastroenterol Hepatol 18:1874-1881.e1872
doi: 10.1016/j.cgh.2019.09.009
Chiu H-M, Chou Y-H, Lu P-Y, Lin P-C, Hung S-H, Chang L-C, Hsu W-F, Lee Y-C, Chen C-P, Wu M-S (2018) Su1737—novel computer-aided recognition system applying deep learning technology for verification of cecal intubation. Gastroenterology 154:S-572
doi: 10.1016/S0016-5085(18)32100-0
Cho M, Kim JH, Kong HJ, Hong KS, Kim S (2018) A novel summary report of colonoscopy: timeline visualization providing meaningful colonoscopy video information. Int J Colorectal Dis 33:549–559
doi: 10.1007/s00384-018-2980-3
Takenaka K, Ohtsuka K, Fujii T, Negi M, Suzuki K, Shimizu H, Oshima S, Akiyama S, Motobayashi M, Nagahori M, Saito E, Matsuoka K, Watanabe M (2020) Development and validation of a deep neural network for accurate evaluation of endoscopic images from patients with ulcerative colitis. Gastroenterology 158:2150–2157
doi: 10.1053/j.gastro.2020.02.012
Ding Z, Shi H, Zhang H, Meng L, Fan M, Han C, Zhang K, Ming F, Xie X, Liu H, Liu J, Lin R, Hou X (2019) Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 157:1044-1054.e1045
doi: 10.1053/j.gastro.2019.06.025
Owais M, Arsalan M, Choi J, Mahmood T, Park KR (2019) Artificial intelligence-based classification of multiple gastrointestinal diseases using endoscopy videos for clinical diagnosis. J Clin Med. https://doi.org/10.3390/jcm8070986
doi: 10.3390/jcm8070986
pubmed: 31514466
pmcid: 6780110
Borgli H, Thambawita V, Smedsrud PH, Hicks S, Jha D, Eskeland SL, Randel KR, Pogorelov K, Lux M, Nguyen DTD, Johansen D, Griwodz C, Stensland HK, Ceja EG, Schmidt PT, Hammer HL, Riegler M, Halvorsen P, de Lange T (2019) Hyper-Kvasir: a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data. https://doi.org/10.1038/s41597-020-00622-y
doi: 10.1038/s41597-020-00622-y
Rezende E, Ruppert G, Carvalho T, Theophilo A, Ramos F, Geus PD (2018) Malicious software classification using VGG16 deep neural network’s bottleneck features. Springer, Cham, pp 51–59
Martineau M, Raveaux R, Chatelain C, Conte D, Venturini G (2018) Effective training of convolutional neural networks for insect image recognition. Springer, Cham, pp 426–437
Krishnaswamy Rangarajan A, Purushothaman R (2020) Disease classification in eggplant using pre-trained VGG16 and MSVM. Sci Rep 10:2322
doi: 10.1038/s41598-020-59108-x
Lui TKL, Guo CG, Leung WK (2020) Accuracy of artificial intelligence on histology prediction and detection of colorectal polyps: a systematic review and meta-analysis. Gastrointest Endosc 92:11-22.e16
doi: 10.1016/j.gie.2020.02.033
Li J, Nadel MR, Poppell CF, Dwyer DM, Lieberman DA, Steinberger EK (2010) Quality assessment of colonoscopy reporting: results from a statewide cancer screening program. Diagn Ther Endosc. https://doi.org/10.1155/2010/419796
doi: 10.1155/2010/419796
pubmed: 20936146
pmcid: 2948883
Technology Committee ASGE, Abu Dayyeh BK, Thosani N, Konda V, Wallace MB, Rex DK, Chauhan SS, Hwang JH, Komanduri S, Manfredi M, Maple JT, Murad FM, Siddiqui UD, Banerjee S (2015) ASGE Technology Committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc 81:502.e501-502.e516
Jin EH, Lee D, Bae JH, Kang HY, Kwak MS, Seo JY, Yang JI, Yang SY, Lim SH, Yim JY, Lim JH, Chung GE, Chung SJ, Choi JM, Han YM, Kang SJ, Lee J, Chan Kim H, Kim JS (2020) Improved accuracy in optical diagnosis of colorectal polyps using convolutional neural networks with visual explanations. Gastroenterology 158:2169-2179.e2168
doi: 10.1053/j.gastro.2020.02.036
Obuch JC, Pigott CM, Ahnen DJ (2015) Sessile serrated polyps: detection, eradication, and prevention of the evil twin. Curr Treat Options Gastroenterol 13:156–170
doi: 10.1007/s11938-015-0046-y
Mori Y, Kudo S, Ogawa Y, Misawa M, Takeda K, Kudo T, Wakamura K, Hayashi T, Ichimasa K, Maeda Y, Toyoshima N, Nakamura H, Katagiri A, Baba T, Ishida F, Oda M, Mori K, Inoue H (2017) Mo1991 can artificial intelligence correctly diagnose sessile serrated adenomas/polyps? Gastrointest Endosc 85:AB510
doi: 10.1016/j.gie.2017.03.1178
Ozawa T, Ishihara S, Fujishiro M, Kumagai Y, Shichijo S, Tada T (2020) Automated endoscopic detection and classification of colorectal polyps using convolutional neural networks. Ther Adv Gastroenterol 13:1756284820910659
doi: 10.1177/1756284820910659
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang ZH, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 115:211–252
doi: 10.1007/s11263-015-0816-y
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv 14091556
Ioffe S, Szegedy CJA (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv abs/1502.03167
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2818–2826
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Chan GCY, Muhammad A, Shah SAA, Tang TB, Lu C, Meriaudeau F (2017) Transfer learning for Diabetic Macular Edema (DME) detection on Optical Coherence Tomography (OCT) images. In: 2017 IEEE international conference on signal and image processing applications (ICSIPA), pp 493–496
Hwang DK, Hsu CC, Chang KJ, Chao D, Sun CH, Jheng YC, Yarmishyn AA, Wu JC, Tsai CY, Wang ML, Peng CH, Chien KH, Kao CL, Lin TC, Woung LC, Chen SJ, Chiou SH (2019) Artificial intelligence-based decision-making for age-related macular degeneration. Theranostics 9:232–245
doi: 10.7150/thno.28447
Wang S, Dong L, Wang X, Wang X (2020) Classification of pathological types of lung cancer from CT images by deep residual neural networks with transfer learning strategy. Open Med (Wars) 15:190–197
doi: 10.1515/med-2020-0028
Mori Y, Kudo SE, Misawa M, Saito Y, Ikematsu H, Hotta K, Ohtsuka K, Urushibara F, Kataoka S, Ogawa Y, Maeda Y, Takeda K, Nakamura H, Ichimasa K, Kudo T, Hayashi T, Wakamura K, Ishida F, Inoue H, Itoh H, Oda M, Mori K (2018) Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med 169:357–366
doi: 10.7326/M18-0249
Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-C (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. https://arxiv.org/abs/1506.04214v2
Ribeiro MT, Singh S, Guestrin C (2016) “Why Should I Trust You?”: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. Association for Computing Machinery, San Francisco, pp 1135–1144
Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz CP, Patel BN, Yeom KW, Shpanskaya K, Blankenberg FG, Seekins J, Amrhein TJ, Mong DA, Halabi SS, Zucker EJ, Ng AY, Lungren MP (2018) Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med 15:e1002686
doi: 10.1371/journal.pmed.1002686
Freedman MT, Osicka T (2008) Heat maps: an aid for data analysis and understanding of ROC CAD experiments. Acad Radiol 15:249–259
doi: 10.1016/j.acra.2007.07.010
Mori Y (2019) Artificial intelligence and colonoscopy: the time is ripe to begin clinical trials. Endoscopy 51:219–220
doi: 10.1055/a-0754-5556