Evaluating false-positive detection in a computer-aided detection system for colonoscopy.
Adenoma
Artificial intelligence
Colonoscopy
Colorectal cancer
Computer-aided detection
Journal
Journal of gastroenterology and hepatology
ISSN: 1440-1746
Titre abrégé: J Gastroenterol Hepatol
Pays: Australia
ID NLM: 8607909
Informations de publication
Date de publication:
25 Jan 2024
25 Jan 2024
Historique:
revised:
21
12
2023
received:
16
10
2023
accepted:
03
01
2024
medline:
26
1
2024
pubmed:
26
1
2024
entrez:
26
1
2024
Statut:
aheadofprint
Résumé
Computer-aided detection (CADe) systems can efficiently detect polyps during colonoscopy. However, false-positive (FP) activation is a major limitation of CADe. We aimed to compare the rate and causes of FP using CADe before and after an update designed to reduce FP. We analyzed CADe-assisted colonoscopy videos recorded between July 2022 and October 2022. The number and causes of FPs and excessive time spent by the endoscopist on FP (ET) were compared pre- and post-update using 1:1 propensity score matching. During the study period, 191 colonoscopy videos (94 and 97 in the pre- and post-update groups, respectively) were recorded. Propensity score matching resulted in 146 videos (73 in each group). The mean number of FPs and median ET per colonoscopy were significantly lower in the post-update group than those in the pre-update group (4.2 ± 3.7 vs 18.1 ± 11.1; P < 0.001 and 0 vs 16 s; P < 0.001, respectively). Mucosal tags, bubbles, and folds had the strongest association with decreased FP post-update (pre-update vs post-update: 4.3 ± 3.6 vs 0.4 ± 0.8, 0.32 ± 0.70 vs 0.04 ± 0.20, and 8.6 ± 6.7 vs 1.6 ± 1.7, respectively). There was no significant decrease in the true positive rate (post-update vs pre-update: 95.0% vs 99.2%; P = 0.09) or the adenoma detection rate (post-update vs pre-update: 52.1% vs 49.3%; P = 0.87). The updated CADe can reduce FP without impairing polyp detection. A reduction in FP may help relieve the burden on endoscopists.
Sections du résumé
BACKGROUND AND AIM
OBJECTIVE
Computer-aided detection (CADe) systems can efficiently detect polyps during colonoscopy. However, false-positive (FP) activation is a major limitation of CADe. We aimed to compare the rate and causes of FP using CADe before and after an update designed to reduce FP.
METHODS
METHODS
We analyzed CADe-assisted colonoscopy videos recorded between July 2022 and October 2022. The number and causes of FPs and excessive time spent by the endoscopist on FP (ET) were compared pre- and post-update using 1:1 propensity score matching.
RESULTS
RESULTS
During the study period, 191 colonoscopy videos (94 and 97 in the pre- and post-update groups, respectively) were recorded. Propensity score matching resulted in 146 videos (73 in each group). The mean number of FPs and median ET per colonoscopy were significantly lower in the post-update group than those in the pre-update group (4.2 ± 3.7 vs 18.1 ± 11.1; P < 0.001 and 0 vs 16 s; P < 0.001, respectively). Mucosal tags, bubbles, and folds had the strongest association with decreased FP post-update (pre-update vs post-update: 4.3 ± 3.6 vs 0.4 ± 0.8, 0.32 ± 0.70 vs 0.04 ± 0.20, and 8.6 ± 6.7 vs 1.6 ± 1.7, respectively). There was no significant decrease in the true positive rate (post-update vs pre-update: 95.0% vs 99.2%; P = 0.09) or the adenoma detection rate (post-update vs pre-update: 52.1% vs 49.3%; P = 0.87).
CONCLUSIONS
CONCLUSIONS
The updated CADe can reduce FP without impairing polyp detection. A reduction in FP may help relieve the burden on endoscopists.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024 Journal of Gastroenterology and Hepatology Foundation and John Wiley & Sons Australia, Ltd.
Références
Patel SG, May FP, Anderson JC et al. Updates on age to start and stop colorectal cancer screening: recommendations from the U.S. Multi-Society Task Force on Colorectal Cancer. Gastroenterology 2022; 162: 285-299.
Bretthauer M, Løberg M, Wieszczy P et al. Effect of colonoscopy screening on risks of colorectal cancer and related death. N. Engl. J. Med. 2022; 387: 1547-1556.
Kaminski MF, Regula J, Kraszewska E et al. Quality indicators for colonoscopy and the risk of interval cancer. N. Engl. J. Med. 2010; 362: 1795-1803.
Corley DA, Jensen CD, Marks AR et al. Adenoma detection rate and risk of colorectal cancer and death. N. Engl. J. Med. 2014; 370: 1298-1306.
Zhao S, Wang S, Pan P et al. Magnitude, risk factors, and factors associated with adenoma miss rate of tandem colonoscopy: a systematic review and meta-analysis. Gastroenterology 2019; 156: 1661-1674.e11.
le Clercq CM, Bouwens MW, Rondagh EJ et al. Postcolonoscopy colorectal cancers are preventable: a population-based study. Gut 2014; 63: 957-963.
Wieszczy P, Waldmann E, Løberg M et al. Colonoscopist performance and colorectal cancer risk after adenoma removal to stratify surveillance: two nationwide observational studies. Gastroenterology 2021; 160: 1067-1074.e6.
Kamba S, Tamai N, Saitoh I et al. Reducing adenoma miss rate of colonoscopy assisted by artificial intelligence: a multicenter randomized controlled trial. J. Gastroenterol. 2021; 56: 746-757.
Hassan C, Wallace MB, Sharma P, Maselli R, Craviotto V, Spadaccini M, Repici A. New artificial intelligence system: first validation study versus experienced endoscopists for colorectal polyp detection. Gut 2020; 69: 799-800.
Fernández-Esparrach G, Bernal J, López-Cerón M et al. Exploring the clinical potential of an automatic colonic polyp detection method based on the creation of energy maps. Endoscopy 2016; 48: 837-842.
Misawa M, Kudo SE, Mori Y et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 2018; 154: 2027-2029.e3.
Gong D, Wu L, Zhang J et al. Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study. Lancet Gastroenterol. Hepatol. 2020; 5: 352-361.
Repici A, Badalamenti M, Maselli R et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520.e7.
Wang P, Liu X, Berzin TM et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol. Hepatol. 2020; 5: 343-351.
Barua I, Vinsard DG, Jodal HC et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy 2021; 53: 277-284.
Misawa M, Kudo SE, Mori Y et al. Current status and future perspective on artificial intelligence for lower endoscopy. Dig. Endosc. 2021; 33: 273-284.
Xu H, Tang RSY, Lam TYT et al. Artificial intelligence-assisted colonoscopy for colorectal cancer screening: a multicenter randomized controlled trial. Clin. Gastroenterol. Hepatol. 2023; 21: 337-346.e3.
Messmann H, Bisschops R, Antonelli G et al. Expected value of artificial intelligence in gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2022; 54: 1211-1231.
Ahmad OF, Mori Y, Misawa M, Kudo SE, Anderson JT, Bernal J et al. Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method. Endoscopy 2021; 53: 893-901.
Hsieh YH, Tang CP, Tseng CW, Lin TL, Leung FW. Computer-aided detection false positives in colonoscopy. Diagnostics (Basel) 2021; 11: 1113.
Kato M, Uedo N, Hokimoto S et al. Guidelines for gastroenterological endoscopy in patients undergoing antithrombotic treatment: 2017 appendix on anticoagulants including direct oral anticoagulants. Dig. Endosc. 2018; 30: 433-440.
Japanese Society for Cancer of the Colon and Rectum. Japanese Classification of Colorectal, Appendiceal, and Anal Carcinoma: the 3rd English Edition [Secondary Publication]. J. Anus. Rectum. Colon 2019; 3: 175-195.
Nagtegaal ID, Odze RD, Klimstra D et al. The 2019 WHO classification of tumours of the digestive system. Histopathology 2020; 76: 182-188.
Fu KI, Sano Y, Kato S et al. Chromoendoscopy using indigo carmine dye spraying with magnifying observation is the most reliable method for differential diagnosis between non-neoplastic and neoplastic colorectal lesions: a prospective study. Endoscopy 2004; 36: 1089-1093.
Sano Y, Ikematsu H, Fu KI et al. Meshed capillary vessels by use of narrow-band imaging for differential diagnosis of small colorectal polyps. Gastrointest. Endosc. 2009; 69: 278-283.
Endoscopic Classification Review Group. Update on the Paris classification of superficial neoplastic lesions in the digestive tract. Endoscopy 2005; 37: 570-578.
Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm. Stat. 2011; 10: 150-161.
Parasa S, Repici A, Berzin T, Leggett C, Gross SA, Sharma P. Framework and metrics for the clinical use and implementation of artificial intelligence algorithms into endoscopy practice: recommendations from the American Society for Gastrointestinal Endoscopy Artificial Intelligence Task Force. Gastrointest. Endosc. 2023; 97: 815-824.e1.
Guo Z, Nemoto D, Zhu X et al. Polyp detection algorithm can detect small polyps: ex vivo reading test compared with endoscopists. Dig. Endosc. 2021; 33: 162-169.
Urban G, Tripathi P, Alkayali T, Mittal M, Jalali F, Karnes W, Baldi P. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078.e8.
Sánchez-Peralta LF, Bote-Curiel L, Picón A, Sánchez-Margallo FM, Pagador JB. Deep learning to find colorectal polyps in colonoscopy: a systematic literature review. Artif. Intell. Med. 2020; 108: 101923.
Holzwanger EA, Bilal M, Glissen Brown JR et al. Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy. Endoscopy 2021; 53: 937-940.
Becq A, Chandnani M, Bharadwaj S et al. Effectiveness of a deep-learning polyp detection system in prospectively collected colonoscopy videos with variable bowel preparation quality. J. Clin. Gastroenterol. 2020; 54: 554-557.
Hassan C, Badalamenti M, Maselli R et al. Computer-aided detection-assisted colonoscopy: classification and relevance of false positives. Gastrointest. Endosc. 2020; 92: 900-904.e4.
Spadaccini M, Hassan C, Alfarone L et al. Comparing the number and relevance of false activations between 2 artificial intelligence computer-aided detection systems: the NOISE study. Gastrointest. Endosc. 2022; 95: 975-981.e1.
Vinsard DG, Mori Y, Misawa M et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest. Endosc. 2019; 90: 55-63.
Tang CP, Lin TL, Hsieh YH, Hsieh CH, Tseng CW, Leung FW. Polyp detection and false-positive rates by computer-aided analysis of withdrawal-phase videos of colonoscopy of the right-sided colon segment in a randomized controlled trial comparing water exchange and air insufflation. Gastrointest. Endosc. 2022; 95: 1198-1206.e6.
Ishiyama M, Kudo SE, Misawa M et al. Impact of the clinical use of artificial intelligence-assisted neoplasia detection for colonoscopy: a large-scale prospective, propensity score-matched study (with video). Gastrointest. Endosc. 2022; 95: 155-163.
Repici A, Spadaccini M, Antonelli G et al. Artificial intelligence and colonoscopy experience: lessons from two randomised trials. Gut 2022; 71: 757-765.
Misawa M, Kudo SE, Mori Y et al. Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointest. Endosc. 2021; 93: 960-967.e3.