Artificial Intelligence with Statistical Confidence Scores for Detection of Acute or Subacute Hemorrhage on Noncontrast CT Head Scans.

CT Convolutional Neural Network (CNN) Head/Neck Hemorrhage

Journal

Radiology. Artificial intelligence
ISSN: 2638-6100
Titre abrégé: Radiol Artif Intell
Pays: United States
ID NLM: 101746556

Informations de publication

Date de publication:
May 2022
Historique:
received: 30 04 2021
revised: 01 03 2022
accepted: 01 04 2022
entrez: 2 6 2022
pubmed: 3 6 2022
medline: 3 6 2022
Statut: epublish

Résumé

To present a method that automatically detects, subtypes, and locates acute or subacute intracranial hemorrhage (ICH) on noncontrast CT (NCCT) head scans; generates detection confidence scores to identify high-confidence data subsets with higher accuracy; and improves radiology worklist prioritization. Such scores may enable clinicians to better use artificial intelligence (AI) tools. This retrospective study included 46 057 studies from seven "internal" centers for development (training, architecture selection, hyperparameter tuning, and operating-point calibration; The areas under the receiver operating characteristic curve for ICH were 0.97 (0.97, 0.98) and 0.95 (0.94, 0.95) on internal and external center data, respectively. On 80% of the data stratified by calibrated classifier and Dempster-Shafer scores, the system improved the Youden indexes, increasing them from 0.84 to 0.93 (calibrated classifier) and from 0.84 to 0.92 (Dempster-Shafer) for internal centers and increasing them from 0.78 to 0.88 (calibrated classifier) and from 0.78 to 0.89 (Dempster-Shafer) for external centers ( AI that provided statistical confidence measures for ICH detection on NCCT scans reliably detected and subtyped hemorrhages, identified high-confidence predictions, and improved worklist prioritization in simulation.

Identifiants

DOI: 10.1148/ryai.210115 PMID: 35652116 PMC: PMC9152881
pubmed: 35652116
doi: 10.1148/ryai.210115
pmc: PMC9152881
doi:

Types de publication

Journal Article

Langues

eng

Pagination

e210115

Subventions

Organisme : NIBIB NIH HHS
ID : P41 EB017183
Pays : United States

Informations de copyright

© 2022 by the Radiological Society of North America, Inc.

Déclaration de conflit d'intérêts

Disclosures of conflicts of interest: E.G. Employed by Siemens Medical Solutions USA; inventor on related patent filing (to be held by Siemens Medical Solutions USA if granted); stock held in Siemens Healthineers. B.G. Employed by Siemens Healthineers. P.C. Employment at Siemens Healthineers, stock or stock options in Siemens Healthineers. P.H.T. Previously employed by Siemens Healthineers. Y.Y. Relevant patent filed December 3, 2021 (filing no. 2020E25048 US). J.D. Employed by Siemens Healthineers. T.J.R. Consultant fees from Siemens Healthineers, via intermediary Randstad North America (consulting payroll intermediary); support from Siemens Healthineers, Princeton, NJ for attending Radiological Society of North America Annual Meeting December 2018; patents planned, issued, or pending as follows: 16/270,918 (filed November 24, 2020), 21168065.7 (pending), 202110399157.6 (pending), 21180032.1 (pending), 16/946,435 (pending), 16/837,979 (pending), 17/249,783 (pending), 201910491911.1 (pending), 202110346295.8 (pending), 21165531.1 (pending), 16/356,086 (pending), 17/211,927 (pending), 19163699.2, (pending), 16/865,266 (pending), 19177891.9 (pending). V.R.S. Employed by Siemens Healthineers. A.B. Employed by Siemens Healthcare, India as Research & Technology Manager (radiologist by profession). E.E. Employed by Siemens Healthcare; patents planned, issued, or pending. A.C. Employed by Siemens Healthcare; patents planned, issued, or pending; stock/stock options. B.B. Previously employed by Siemens Healthineers. U.K.B. Support from Siemens; has grant. S.N. No relevant relationships. P.C.S. Grants from National Institutes of Health National Institute of Neurological Disorders and Stroke, Siemens Healthineers, and Neiman Health Policy Institute to author’s institution; leadership or fiduciary role in American Society of Neuroradiology. T.J.S. No relevant relationships. T.F. Employed by Siemens Healthcare. D.C. Employed by Siemens Healthineers; stock/stock options in Siemens Healthineers. Y.W.L. Support from Siemens Healthineers (institution received reimbursement for costs from institutional review board and IT related to data collection, clinical research support unit, and a finance and administration surcharge relating to this work).

Références

J Stroke Cerebrovasc Dis. 2015 Jun;24(6):1256-61
pubmed: 25920753
IEEE Trans Pattern Anal Mach Intell. 2019 Jan;41(1):176-189
pubmed: 29990011
Lancet. 2012 Jun 23;379(9834):2364-72
pubmed: 22632907
Stroke. 2016 Apr;47(4):1124-6
pubmed: 26892283
J Clin Epidemiol. 2008 May;61(5):517-8
pubmed: 18394548
Lancet Neurol. 2010 Feb;9(2):167-76
pubmed: 20056489
Radiol Artif Intell. 2020 Apr 29;2(3):e190211
pubmed: 33937827
J Am Coll Radiol. 2016 Jun;13(6):668-79
pubmed: 27262056
Int J Environ Res Public Health. 2019 Nov 29;16(23):
pubmed: 31795322
Neurology. 2006 Apr 25;66(8):1175-81
pubmed: 16636233
Stroke. 1993 Jul;24(7):987-93
pubmed: 8322400
Proc Natl Acad Sci U S A. 2019 Nov 5;116(45):22737-22745
pubmed: 31636195
J Am Coll Radiol. 2017 May;14(5S):S34-S61
pubmed: 28473091
NPJ Digit Med. 2018 Apr 4;1:9
pubmed: 31304294

Auteurs

Eli Gibson (E)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0001-9207-7280

Bogdan Georgescu (B)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0001-5388-5699

Pascal Ceccaldi (P)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0001-9536-5990

Pierre-Hugo Trigan (PH)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0003-1139-2022

Youngjin Yoo (Y)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-0001-0503

Jyotipriya Das (J)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).

Thomas J Re (TJ)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).

Vishwanath Rs (V)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-9494-471X

Abishek Balachandran (A)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-3726-7839

Eva Eibenberger (E)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0003-3560-830X

Andrei Chekkoury (A)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0003-3651-6298

Barbara Brehm (B)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).

Uttam K Bodanapally (UK)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0003-0455-816X

Savvas Nicolaou (S)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-4203-6605

Pina C Sanelli (PC)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-9633-3699

Thomas J Schroeppel (TJ)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0001-9164-5004

Thomas Flohr (T)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).

Dorin Comaniciu (D)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-5238-8647

Yvonne W Lui (YW)

Department of Digital Technology and Innovation, Siemens Healthineers, 755 College Rd E, Princeton, NJ 08540 (E.G., B.G., P.C., P.H.T., Y.Y., J.D., T.J.R., D.C.); Department of Digital Technology and Innovation, Siemens Healthineers, Bangalore, India (V.R.S., A.B.); Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany (E.E., A.C., B.B., T.F.); Department of Radiology, University of Maryland Medical Center, Baltimore, Md (U.K.B.); Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Northwell Health, New York, NY (P.C.S.); Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, Colo (T.J.S.); and Department of Radiology, NYU Langone Health, New York University School of Medicine, New York, NY (Y.W.L.).
ORCID: 0000-0002-9984-9164

Classifications MeSH