Comparing a deep learning model's diagnostic performance to that of radiologists to detect Covid -19 features on chest radiographs.
Artificial intelligence
COVID 19
CXR
deep learning
Journal
The Indian journal of radiology & imaging
ISSN: 0971-3026
Titre abrégé: Indian J Radiol Imaging
Pays: Germany
ID NLM: 8503873
Informations de publication
Date de publication:
Jan 2021
Jan 2021
Historique:
received:
27
11
2020
revised:
09
12
2020
accepted:
17
12
2020
entrez:
5
4
2021
pubmed:
6
4
2021
medline:
6
4
2021
Statut:
ppublish
Résumé
Whether the sensitivity of Deep Learning (DL) models to screen chest radiographs (CXR) for CoVID-19 can approximate that of radiologists, so that they can be adopted and used if real-time review of CXRs by radiologists is not possible, has not been explored before. To evaluate the diagnostic performance of a doctor-trained DL model (Svita_DL8) to screen for COVID-19 on CXR, and to compare the performance of the DL model with that of expert radiologists. We used a pre-trained convolutional neural network to develop a publicly available online DL model to evaluate CXR examinations saved in .jpeg or .png format. The initial model was subsequently curated and trained by an internist and a radiologist using 1062 chest radiographs to classify a submitted CXR as either normal, COVID-19, or a non-COVID-19 abnormal. For validation, we collected a separate set of 430 CXR examinations from numerous publicly available datasets from 10 different countries, case presentations, and two hospital repositories. These examinations were assessed for COVID-19 by the DL model and by two independent radiologists. Diagnostic performance was compared between the model and the radiologists and the correlation coefficient calculated. For detecting COVID-19 on CXR, our DL model demonstrated sensitivity of 91.5%, specificity of 55.3%, PPV 60.9%, NPV 77.9%, accuracy 70.1%, and AUC 0.73 (95% CI: 0.86, 0.95). There was a significant correlation (r = 0.617, The DL model demonstrated high sensitivity for detecting COVID-19 on CXR. The doctor trained DL tool Svita_DL8 can be used in resource-constrained settings to quickly triage patients with suspected COVID-19 for further in-depth review and testing.
Sections du résumé
BACKGROUND
BACKGROUND
Whether the sensitivity of Deep Learning (DL) models to screen chest radiographs (CXR) for CoVID-19 can approximate that of radiologists, so that they can be adopted and used if real-time review of CXRs by radiologists is not possible, has not been explored before.
OBJECTIVE
OBJECTIVE
To evaluate the diagnostic performance of a doctor-trained DL model (Svita_DL8) to screen for COVID-19 on CXR, and to compare the performance of the DL model with that of expert radiologists.
MATERIALS AND METHODS
METHODS
We used a pre-trained convolutional neural network to develop a publicly available online DL model to evaluate CXR examinations saved in .jpeg or .png format. The initial model was subsequently curated and trained by an internist and a radiologist using 1062 chest radiographs to classify a submitted CXR as either normal, COVID-19, or a non-COVID-19 abnormal. For validation, we collected a separate set of 430 CXR examinations from numerous publicly available datasets from 10 different countries, case presentations, and two hospital repositories. These examinations were assessed for COVID-19 by the DL model and by two independent radiologists. Diagnostic performance was compared between the model and the radiologists and the correlation coefficient calculated.
RESULTS
RESULTS
For detecting COVID-19 on CXR, our DL model demonstrated sensitivity of 91.5%, specificity of 55.3%, PPV 60.9%, NPV 77.9%, accuracy 70.1%, and AUC 0.73 (95% CI: 0.86, 0.95). There was a significant correlation (r = 0.617,
CONCLUSIONS
CONCLUSIONS
The DL model demonstrated high sensitivity for detecting COVID-19 on CXR.
CLINICAL IMPACT
CONCLUSIONS
The doctor trained DL tool Svita_DL8 can be used in resource-constrained settings to quickly triage patients with suspected COVID-19 for further in-depth review and testing.
Identifiants
pubmed: 33814762
doi: 10.4103/ijri.IJRI_914_20
pii: IJRI-31-53
pmc: PMC7996677
doi:
Types de publication
Journal Article
Langues
eng
Pagination
S53-S60Informations de copyright
Copyright: © 2021 Indian Journal of Radiology and Imaging.
Déclaration de conflit d'intérêts
There are no conflicts of interest.
Références
Sci Rep. 2019 Oct 18;9(1):15000
pubmed: 31628424
J Thorac Imaging. 2020 Jul;35(4):219-227
pubmed: 32324653
Am J Med. 2004 Aug 15;117(4):249-54
pubmed: 15308434
Radiology. 2020 Sep;296(3):E166-E172
pubmed: 32384019
JAMA. 2012 Jun 20;307(23):2526-33
pubmed: 22797452
Med J Aust. 2020 Jul;213(2):54-56.e1
pubmed: 32572965
Clin Imaging. 2020 Aug;64:35-42
pubmed: 32302927
Diagnostics (Basel). 2020 May 30;10(6):
pubmed: 32486140
Radiology. 2020 Aug;296(2):E115-E117
pubmed: 32073353
Aust Paediatr J. 1984 May;20(2):109-12
pubmed: 6466225
Radiology. 2020 Aug;296(2):E72-E78
pubmed: 32216717
AJR Am J Roentgenol. 2020 Jun;214(6):1280-1286
pubmed: 32130038
IEEE Trans Med Imaging. 2020 Aug;39(8):2688-2700
pubmed: 32396075