Deep Learning Models for Predicting Malignancy Risk in CT-Detected Pulmonary Nodules: A Systematic Review and Meta-analysis.
Chest CT
Diagnosis
Lung cancer
Pulmonary nodules
Screening
Journal
Lung
ISSN: 1432-1750
Titre abrégé: Lung
Pays: United States
ID NLM: 7701875
Informations de publication
Date de publication:
23 May 2024
23 May 2024
Historique:
received:
15
01
2024
accepted:
12
05
2024
medline:
24
5
2024
pubmed:
24
5
2024
entrez:
23
5
2024
Statut:
aheadofprint
Résumé
There has been growing interest in using artificial intelligence/deep learning (DL) to help diagnose prevalent diseases earlier. In this study we sought to survey the landscape of externally validated DL-based computer-aided diagnostic (CADx) models, and assess their diagnostic performance for predicting the risk of malignancy in computed tomography (CT)-detected pulmonary nodules. An electronic search was performed in four databases (from inception to 10 August 2023). Studies were eligible if they were peer-reviewed experimental or observational articles comparing the diagnostic performance of externally validated DL-based CADx models with models widely used in clinical practice to predict the risk of malignancy. A bivariate random-effect approach for the meta-analysis on the included studies was used. Seventeen studies were included, comprising 8553 participants and 9884 nodules. Pooled analyses showed DL-based CADx models were 11.6% more sensitive than physician judgement alone, and 14.5% more than clinical risk models alone. They had a similar pooled specificity to physician judgement alone [0.77 (95% CI 0.68-0.84) v 0.81 (95% CI 0.71-0.88)], and were 7.4% more specific than clinical risk models alone. They had superior pooled areas under the receiver operating curve (AUC), with relative pooled AUCs of 1.03 (95% CI 1.00-1.07) and 1.10 (95% CI 1.07-1.13) versus physician judgement and clinical risk models alone, respectively. DL-based models are already used in clinical practice in certain settings for nodule management. Our results show their diagnostic performance potentially justifies wider, more routine deployment alongside experienced physician readers to help inform multidisciplinary team decision-making.
Sections du résumé
BACKGROUND
BACKGROUND
There has been growing interest in using artificial intelligence/deep learning (DL) to help diagnose prevalent diseases earlier. In this study we sought to survey the landscape of externally validated DL-based computer-aided diagnostic (CADx) models, and assess their diagnostic performance for predicting the risk of malignancy in computed tomography (CT)-detected pulmonary nodules.
METHODS
METHODS
An electronic search was performed in four databases (from inception to 10 August 2023). Studies were eligible if they were peer-reviewed experimental or observational articles comparing the diagnostic performance of externally validated DL-based CADx models with models widely used in clinical practice to predict the risk of malignancy. A bivariate random-effect approach for the meta-analysis on the included studies was used.
RESULTS
RESULTS
Seventeen studies were included, comprising 8553 participants and 9884 nodules. Pooled analyses showed DL-based CADx models were 11.6% more sensitive than physician judgement alone, and 14.5% more than clinical risk models alone. They had a similar pooled specificity to physician judgement alone [0.77 (95% CI 0.68-0.84) v 0.81 (95% CI 0.71-0.88)], and were 7.4% more specific than clinical risk models alone. They had superior pooled areas under the receiver operating curve (AUC), with relative pooled AUCs of 1.03 (95% CI 1.00-1.07) and 1.10 (95% CI 1.07-1.13) versus physician judgement and clinical risk models alone, respectively.
CONCLUSION
CONCLUSIONS
DL-based models are already used in clinical practice in certain settings for nodule management. Our results show their diagnostic performance potentially justifies wider, more routine deployment alongside experienced physician readers to help inform multidisciplinary team decision-making.
Identifiants
pubmed: 38782779
doi: 10.1007/s00408-024-00706-1
pii: 10.1007/s00408-024-00706-1
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s).
Références
Woodard GA, Jones KD, Jablons DM (2016) Lung cancer staging and prognosis. Cancer Treat Res 170:47–75. https://doi.org/10.1007/978-3-319-40389-2_3
doi: 10.1007/978-3-319-40389-2_3
pubmed: 27535389
Loverdos K, Fotiadis A, Kontogianni C et al (2019) Lung nodules: a comprehensive review on current approach and management. Ann Thorac Med 14:226–238. https://doi.org/10.4103/atm.ATM_110_19
doi: 10.4103/atm.ATM_110_19
pubmed: 31620206
pmcid: 6784443
Gould MK, Tang T, Liu I-LA et al (2015) Recent trends in the identification of incidental pulmonary nodules. Am J Respir Crit Care Med 192:1208–1214. https://doi.org/10.1164/rccm.201505-0990OC
doi: 10.1164/rccm.201505-0990OC
pubmed: 26214244
Mahesh M, Ansari AJ, Mettler FA (2023) Patient exposure from radiologic and nuclear medicine procedures in the United States and worldwide: 2009–2018. Radiology. https://doi.org/10.1148/radiol.221263
doi: 10.1148/radiol.221263
pubmed: 37962507
Paez R, Kammer MN, Massion P (2021) Risk stratification of indeterminate pulmonary nodules. Curr Opin Pulm Med 27:240–248. https://doi.org/10.1097/MCP.0000000000000780
doi: 10.1097/MCP.0000000000000780
pubmed: 33973553
Swensen SJ, Silverstein MD, Ilstrup DM et al (1997) The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules. Arch Intern Med 157:849–855
doi: 10.1001/archinte.1997.00440290031002
pubmed: 9129544
McWilliams A, Tammemagi MC, Mayo JR et al (2013) Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 369:910–919. https://doi.org/10.1056/NEJMoa1214726
doi: 10.1056/NEJMoa1214726
pubmed: 24004118
pmcid: 3951177
Herder GJ, van Tinteren H, Golding RP et al (2005) Clinical prediction model to characterize pulmonary nodules. Chest 128:2490–2496. https://doi.org/10.1378/chest.128.4.2490
doi: 10.1378/chest.128.4.2490
pubmed: 16236914
Tsakok MT, Mashar M, Pickup L et al (2021) The utility of a convolutional neural network (CNN) model score for cancer risk in indeterminate small solid pulmonary nodules, compared to clinical practice according to British Thoracic Society guidelines. Eur J Radiol 137:109553. https://doi.org/10.1016/j.ejrad.2021.109553
doi: 10.1016/j.ejrad.2021.109553
pubmed: 33581913
Stroup DF, Berlin JA, Morton SC (2000) Meta-analysis of observational studies in epidemiology. JAMA 283:2008. https://doi.org/10.1001/jama.283.15.2008
doi: 10.1001/jama.283.15.2008
pubmed: 10789670
Page MJ, McKenzie JE, Bossuyt PM et al (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. https://doi.org/10.1136/bmj.n71
doi: 10.1136/bmj.n71
pubmed: 33782057
pmcid: 8479591
Whiting PF, Rutjes AWS, Westwood ME et al (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
doi: 10.7326/0003-4819-155-8-201110180-00009
pubmed: 22007046
The Cochrane Collaboration (2020) Review manager (RevMan). https://revman.cochrane.org/
Doebler P, Sousa-Pinto B (2022) Meta-analysis of diagnostic accuracy with mada. R Packages. https://r-forge.r-project.org/projects/mada
Chen K, Nie Y, Park S et al (2021) Development and validation of machine learning-based model for the prediction of malignancy in multiple pulmonary nodules: analysis from multicentric cohorts. Clin Cancer Res 27:2255–2265. https://doi.org/10.1158/1078-0432.CCR-20-4007
doi: 10.1158/1078-0432.CCR-20-4007
pubmed: 33627492
Massion PP, Antic S, Ather S et al (2020) Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules. Am J Respir Crit Care Med 202:241–249. https://doi.org/10.1164/rccm.201903-0505OC
doi: 10.1164/rccm.201903-0505OC
pubmed: 32326730
pmcid: 7365375
Kim RY, Oke JL, Pickup LC et al (2022) Artificial intelligence tool for assessment of indeterminate pulmonary nodules detected with CT. Radiology 304:683–691. https://doi.org/10.1148/radiol.212182
doi: 10.1148/radiol.212182
pubmed: 35608444
Choi HK, Ghobrial M, Mazzone PJ (2018) Models to estimate the probability of malignancy in patients with pulmonary nodules. Ann Am Thorac Soc 15:1117–1126. https://doi.org/10.1513/AnnalsATS.201803-173CME
doi: 10.1513/AnnalsATS.201803-173CME
pubmed: 30272500
González Maldonado S, Delorme S, Hüsing A et al (2020) Evaluation of prediction models for identifying malignancy in pulmonary nodules detected via low-dose computed tomography. JAMA Netw Open 3:e1921221. https://doi.org/10.1001/jamanetworkopen.2019.21221
doi: 10.1001/jamanetworkopen.2019.21221
pubmed: 32058555
White CS, Dharaiya E, Campbell E, Boroczky L (2017) The vancouver lung cancer risk prediction model: assessment by using a subset of the national lung screening trial cohort. Radiology 283:264–272. https://doi.org/10.1148/radiol.2016152627
doi: 10.1148/radiol.2016152627
pubmed: 27740906
Hunter B, Chen M, Ratnakumar P et al (2022) A radiomics-based decision support tool improves lung cancer diagnosis in combination with the Herder score in large lung nodules. EBioMedicine 86:104344. https://doi.org/10.1016/j.ebiom.2022.104344
doi: 10.1016/j.ebiom.2022.104344
pubmed: 36370635
pmcid: 9664396
Baldwin DR, Gustafson J, Pickup L et al (2020) External validation of a convolutional neural network artificial intelligence tool to predict malignancy in pulmonary nodules. Thorax 75:306–312. https://doi.org/10.1136/thoraxjnl-2019-214104
doi: 10.1136/thoraxjnl-2019-214104
pubmed: 32139611
Mazzone PJ, Lam L (2022) Evaluating the patient with a pulmonary nodule. JAMA 327:264. https://doi.org/10.1001/jama.2021.24287
doi: 10.1001/jama.2021.24287
pubmed: 35040882
Huang P, Lin CT, Li Y et al (2019) Prediction of lung cancer risk at follow-up screening with low-dose CT: a training and validation study of a deep learning method. Lancet Digit Health 1:e353–e362. https://doi.org/10.1016/S2589-7500(19)30159-1
doi: 10.1016/S2589-7500(19)30159-1
pubmed: 32864596
pmcid: 7450858
Adams SJ, Mondal P, Penz E et al (2021) Development and cost analysis of a lung nodule management strategy combining artificial intelligence and Lung-RADS for baseline lung cancer screening. J Am Coll Radiol 18:741–751. https://doi.org/10.1016/j.jacr.2020.11.014
doi: 10.1016/j.jacr.2020.11.014
pubmed: 33482120
Adams SJ, Madtes DK, Burbridge B et al (2023) Clinical impact and generalizability of a computer-assisted diagnostic tool to risk-stratify lung nodules with CT. J Am Coll Radiol 20:232–242. https://doi.org/10.1016/j.jacr.2022.08.006
doi: 10.1016/j.jacr.2022.08.006
pubmed: 36064040
Ardila D, Kiraly AP, Bharadwaj S et al (2019) End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med 25:954–961. https://doi.org/10.1038/s41591-019-0447-x
doi: 10.1038/s41591-019-0447-x
pubmed: 31110349
Chen Y, Tian X, Fan K et al (2022) The value of artificial intelligence film reading system based on deep learning in the diagnosis of non-small-cell lung cancer and the significance of efficacy monitoring: a retrospective, clinical, nonrandomized Controlled Study. Comput Math Methods Med 2022:2864170. https://doi.org/10.1155/2022/2864170
doi: 10.1155/2022/2864170
pubmed: 35360550
pmcid: 8964156
Gürsoy Çoruh A, Yenigün B, Uzun Ç et al (2021) A comparison of the fusion model of deep learning neural networks with human observation for lung nodule detection and classification. Br J Radiol 94:20210222. https://doi.org/10.1259/bjr.20210222
doi: 10.1259/bjr.20210222
pubmed: 34111976
pmcid: 8248221
Gao R, Tang Y, Khan MS et al (2021) Cancer risk estimation combining lung screening CT with clinical data elements. Radiol Artif Intell 3:e210032. https://doi.org/10.1148/ryai.2021210032
doi: 10.1148/ryai.2021210032
pubmed: 34870220
pmcid: 8637232
Gao R, Li T, Tang Y et al (2022) Reducing uncertainty in cancer risk estimation for patients with indeterminate pulmonary nodules using an integrated deep learning model. Comput Biol Med 150:106113. https://doi.org/10.1016/j.compbiomed.2022.106113
doi: 10.1016/j.compbiomed.2022.106113
pubmed: 36198225
pmcid: 10050219
Jacobs C, Setio AAA, Scholten ET et al (2021) Deep learning for lung cancer detection on screening CT scans: results of a large-scale public competition and an observer study with 11 radiologists. Radiol Artif Intell 3:e210027. https://doi.org/10.1148/ryai.2021210027
doi: 10.1148/ryai.2021210027
pubmed: 34870218
pmcid: 8637223
Liao F, Liang M, Li Z et al (2019) Evaluate the malignancy of pulmonary nodules using the 3-D deep leaky noisy-OR network. IEEE Trans Neural Netw Learn Syst 30:3484–3495. https://doi.org/10.1109/TNNLS.2019.2892409
doi: 10.1109/TNNLS.2019.2892409
pubmed: 30794190
Liu J, Zhao L, Han X et al (2021) Estimation of malignancy of pulmonary nodules at CT scans: effect of computer-aided diagnosis on diagnostic performance of radiologists. Asia Pac J Clin Oncol 17:216–221. https://doi.org/10.1111/ajco.13362
doi: 10.1111/ajco.13362
pubmed: 32757455
Trajanovski S, Mavroeidis D, Swisher CL et al (2021) Towards radiologist-level cancer risk assessment in CT lung screening using deep learning. Comput Med Imaging Graph 90:101883. https://doi.org/10.1016/j.compmedimag.2021.101883
doi: 10.1016/j.compmedimag.2021.101883
pubmed: 33895622
Venkadesh KV, Setio AAA, Schreuder A et al (2021) Deep learning for malignancy risk estimation of pulmonary nodules detected at low-dose screening CT. Radiology 300:438–447. https://doi.org/10.1148/radiol.2021204433
doi: 10.1148/radiol.2021204433
pubmed: 34003056
Ramspek CL, Jager KJ, Dekker FW et al (2021) External validation of prognostic models: what, why, how, when and where? Clin Kidney J 14:49–58. https://doi.org/10.1093/ckj/sfaa188
doi: 10.1093/ckj/sfaa188
pubmed: 33564405
Maldonado F, Lentz RJ (2020) Reducing uncertainty to a manageable level: the need for a nuanced and patient-centric approach to lung nodule management in the 21st century. J Thorac Dis 12(6):3242–3244. https://doi.org/10.21037/jtd.2020.03.65
doi: 10.21037/jtd.2020.03.65
pubmed: 32642246
pmcid: 7330741
Forte GC, Altmayer S, Silva RF et al (2022) Deep learning algorithms for diagnosis of lung cancer: a systematic review and meta-analysis. Cancers (Basel). https://doi.org/10.3390/cancers14163856
doi: 10.3390/cancers14163856
pubmed: 36428732
Wu Z, Wang F, Cao W et al (2022) Lung cancer risk prediction models based on pulmonary nodules: a systematic review. Thorac Cancer 13:664–677. https://doi.org/10.1111/1759-7714.14333
doi: 10.1111/1759-7714.14333
pubmed: 35137543
pmcid: 8888150
Mazumdar M, Zhong X, Ferket B (2021) Diagnostic trials. Principles and practice of clinical trials. Springer, Cham
Peikert T, Bartholmai BJ, Maldonado F (2020) Radiomics-based Management of indeterminate lung nodules? Are we there yet? Am J Respir Crit Care Med 202:165–167. https://doi.org/10.1164/rccm.202004-1279ED
doi: 10.1164/rccm.202004-1279ED
pubmed: 32383972
pmcid: 7365373
Gould MK, Donington J, Lynch WR et al (2013) Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:e93S-e120S. https://doi.org/10.1378/chest.12-2351
doi: 10.1378/chest.12-2351
pubmed: 23649456
pmcid: 3749714
Callister MEJ, Baldwin DR, Akram AR et al (2015) British thoracic society guidelines for the investigation and management of pulmonary nodules. Thorax. https://doi.org/10.1136/thoraxjnl-2015-207168
doi: 10.1136/thoraxjnl-2015-207168
pubmed: 26135833
Osarogiagbon RU, Liao W, Faris NR et al (2022) Lung cancer diagnosed through screening, lung nodule, and neither program: a prospective observational study of the detecting early lung cancer (DELUGE) in the Mississippi Delta Cohort. J Clin Oncol 40:2094–2105. https://doi.org/10.1200/JCO.21.02496
doi: 10.1200/JCO.21.02496
pubmed: 35258994
pmcid: 9242408
Hricak H, Abdel-Wahab M, Atun R et al (2021) Medical imaging and nuclear medicine: a lancet oncology commission. Lancet Oncol 22:e136–e172. https://doi.org/10.1016/S1470-2045(20)30751-8
doi: 10.1016/S1470-2045(20)30751-8
pubmed: 33676609
pmcid: 8444235
National Lung Screening Trial Research Team, Aberle DR, Adams AM et al (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365:395–409. https://doi.org/10.1056/NEJMoa1102873
doi: 10.1056/NEJMoa1102873
Paez R, Kammer MN, Tanner NT et al (2023) Update on biomarkers for the stratification of indeterminate pulmonary nodules. Chest. https://doi.org/10.1016/j.chest.2023.05.025
doi: 10.1016/j.chest.2023.05.025
pubmed: 38142773
Holland P, Spence H, Clubley A et al (2020) Reporting radiographers and their role in thoracic CT service improvement: managing the pulmonary nodule. BJR|Open 2(1):20190018. https://doi.org/10.1259/bjro.20190018
doi: 10.1259/bjro.20190018
pubmed: 33178958
pmcid: 7594904
Paez R, Kammer MN, Balar A et al (2023) Longitudinal lung cancer prediction convolutional neural network model improves the classification of indeterminate pulmonary nodules. Sci Rep 13:6157. https://doi.org/10.1038/s41598-023-33098-y
doi: 10.1038/s41598-023-33098-y
pubmed: 37061539
pmcid: 10105767
Landy R, Wang VL, Baldwin DR et al (2023) Recalibration of a deep learning model for low-dose computed tomographic images to inform lung cancer screening intervals. JAMA Netw Open 6:e233273. https://doi.org/10.1001/jamanetworkopen.2023.3273
doi: 10.1001/jamanetworkopen.2023.3273
pubmed: 36929398
pmcid: 10020880
Henschke CI, Yankelevitz DF, Mirtcheva R et al (2002) CT screening for lung cancer: frequency and significance of part-solid and nonsolid nodules. Am J Roentgenol 178:1053–1057. https://doi.org/10.2214/ajr.178.5.1781053
doi: 10.2214/ajr.178.5.1781053