Psychophysical Evaluation of Visual vs. Computer-Aided Detection of Brain Lesions on Magnetic Resonance Images.
computer-aided diagnosis
deep learning
multiple sclerosis
psychophysics
Journal
Journal of magnetic resonance imaging : JMRI
ISSN: 1522-2586
Titre abrégé: J Magn Reson Imaging
Pays: United States
ID NLM: 9105850
Informations de publication
Date de publication:
08 2023
08 2023
Historique:
revised:
25
11
2022
received:
30
08
2022
accepted:
28
11
2022
medline:
11
7
2023
pubmed:
11
12
2022
entrez:
10
12
2022
Statut:
ppublish
Résumé
Magnetic resonance imaging (MRI) diagnosis is usually performed by analyzing contrast-weighted images, where pathology is detected once it reached a certain visual threshold. Computer-aided diagnosis (CAD) has been proposed as a way for achieving higher sensitivity to early pathology. To compare conventional (i.e., visual) MRI assessment of artificially generated multiple sclerosis (MS) lesions in the brain's white matter to CAD based on a deep neural network. Prospective. A total of 25 neuroradiologists (15 males, age 39 ± 9, 9 ± 9.8 years of experience) independently assessed all synthetic lesions. A 3.0 T, T MS lesions of varying severity levels were artificially generated in healthy volunteer MRI scans by manipulating T Diagnostic performance of the two approaches was compared using z-tests on TP rates, FP rates, and the logarithm of ORs across severity levels. A P-value <0.05 was considered statistically significant. ORs of identifying pathology were significantly higher for CAD vis-à-vis visual inspection for all lesions' severity levels. For a 6% change in T CAD is capable of detecting the presence or absence of more subtle lesions with greater precision than the representative group of 25 radiologists chosen in this study. 1 TECHNICAL EFFICACY: Stage 3.
Sections du résumé
BACKGROUND
Magnetic resonance imaging (MRI) diagnosis is usually performed by analyzing contrast-weighted images, where pathology is detected once it reached a certain visual threshold. Computer-aided diagnosis (CAD) has been proposed as a way for achieving higher sensitivity to early pathology.
PURPOSE
To compare conventional (i.e., visual) MRI assessment of artificially generated multiple sclerosis (MS) lesions in the brain's white matter to CAD based on a deep neural network.
STUDY TYPE
Prospective.
POPULATION
A total of 25 neuroradiologists (15 males, age 39 ± 9, 9 ± 9.8 years of experience) independently assessed all synthetic lesions.
FIELD STRENGTH/SEQUENCE
A 3.0 T, T
ASSESSMENT
MS lesions of varying severity levels were artificially generated in healthy volunteer MRI scans by manipulating T
STATISTICAL TESTS
Diagnostic performance of the two approaches was compared using z-tests on TP rates, FP rates, and the logarithm of ORs across severity levels. A P-value <0.05 was considered statistically significant.
RESULTS
ORs of identifying pathology were significantly higher for CAD vis-à-vis visual inspection for all lesions' severity levels. For a 6% change in T
DATA CONCLUSION
CAD is capable of detecting the presence or absence of more subtle lesions with greater precision than the representative group of 25 radiologists chosen in this study.
LEVEL OF EVIDENCE
1 TECHNICAL EFFICACY: Stage 3.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
642-649Commentaires et corrections
Type : CommentIn
Informations de copyright
© 2022 The Authors. Journal of Magnetic Resonance Imaging published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.
Références
Lüsebrink F, Sciarra A, Mattern H, Yakupov R, Speck O. T1-weighted in vivo human whole brain MRI dataset with an ultrahigh isotropic resolution of 250 μm. Sci Data 2017;4:170032.
Reich DS, Lucchinetti CF, Calabresi PA. Multiple sclerosis. N Engl J Med 2018;378:169-180.
Poewe W, Seppi K, Tanner CM, et al. Parkinson disease. Nat Rev Dis Primers 2017;3:17013.
Johnson KA, Fox NC, Sperling RA, Klunk WE. Brain imaging in Alzheimer disease. Cold Spring Harb Perspect Med 2012;2:a006213.
Hardie AD, Naik M, Hecht EM, et al. Diagnosis of liver metastases: Value of diffusion-weighted MRI compared with gadolinium-enhanced MRI. Eur Radiol 2010;20:1431-1441.
Harisinghani MG, O'Shea A, Weissleder R. Advances in clinical MRI technology. Sci Transl Med 2019;11:eaba2591.
Bloch F. Nuclear induction. Phys Rev 1946;70:460-474.
Ben-Eliezer N, Sodickson DK, Block KT. Rapid and accurate T 2 mapping from multi-spin-echo data using Bloch-simulation-based reconstruction. Magn Reson Med 2015;73:809-817.
Ben-Eliezer N. Advances in signal processing for relaxometry. In: Nicole S, Vikas G, editors. Quantitative magnetic resonance imaging. London, United Kingdom: Elsevier; Vol 1; 2020. p 123-147.
Weiskopf N, Edwards LJ, Helms G, Mohammadi S, Kirilina E. Quantitative magnetic resonance imaging of brain anatomy and in vivo histology. Nat Rev Phys 2021;3:570-588.
Shepherd TM, Kirov II, Charlson E, et al. New rapid, accurate T2 quantification detects pathology in normal-appearing brain regions of relapsing-remitting MS patients. Neuroimage Clin 2017;14:363-370.
Gracien R-M, Maiworm M, Brüche N, et al. How stable is quantitative MRI?-Assessment of intra- and inter-scanner-model reproducibility using identical acquisition sequences and data analysis programs. Neuroimage 2020;207:116364.
Messroghli DR, Moon JC, Ferreira VM, et al. Clinical recommendations for cardiovascular magnetic resonance mapping of T1, T2, T2* and extracellular volume: A consensus statement by the Society for Cardiovascular Magnetic Resonance (SCMR) endorsed by the European Association for Cardiovascular Imaging (EACVI). J Cardiovasc Magn Reson 2017;19:75.
Ben-Eliezer N, Raya JG, Babb JS, Youm T, Sodickson DK, Lattanzi R. A new method for cartilage evaluation in femoroacetabular impingement using quantitative T2 magnetic resonance imaging: Preliminary validation against arthroscopic findings. Cartilage 2021;13(1_suppl):1315 S-1323 S.
Kessler LG, Barnhart HX, Buckler AJ, et al. The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions. Stat Methods Med Res 2015;24:9-26.
deSouza NM, Achten E, Alberich-Bayarri A, et al. Validated imaging biomarkers as decision-making tools in clinical trials and routine practice: Current status and recommendations from the EIBALL* subcommittee of the European Society of Radiology (ESR). Insights Imaging 2019;10:87.
Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis. Lancet Digit Health 2019;1:e271-e297.
Piredda GF, Hilbert T, Granziera C, et al. Quantitative brain relaxation atlases for personalized detection and characterization of brain pathology. Magn Reson Med 2020;83:337-351.
Seah JCY, Tang CHM, Buchlak QD, et al. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: A retrospective, multireader multicase study. Lancet Digit Health 2021;3:e496-e506.
Schurink NW, van Kranen SR, Roberti S, et al. Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. Eur Radiol 2022;32:1506-1516.
Woo JH, Henry LP, Krejza J, Melhem ER. Detection of simulated multiple sclerosis lesions on T2-weighted and FLAIR images of the brain: Observer performance. Radiology 2006;241:206-212.
Altay EE, Fisher E, Jones SE, Hara-Cleaver C, Lee J-C, Rudick RA. Reliability of classifying multiple sclerosis disease activity using magnetic resonance imaging in a multiple sclerosis clinic. JAMA Neurol 2013;70:338-344.
Freeman K, Geppert J, Stinton C, Todkill D, Johnson S, Clarke A, et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: Systematic review of test accuracy. BMJ 2021;374:n1872. https://doi.org/10.1136/bmj.n1872
Bilello M, Arkuszewski M, Nasrallah I, Wu X, Erus G, Krejza J. Multiple sclerosis lesions in the brain: Computer-assisted assessment of lesion load dynamics on 3D FLAIR MR images. Neuroradiol J 2012;25:17-21.
Bilello M, Arkuszewski M, Nucifora P, et al. Multiple sclerosis: Identification of temporal changes in brain lesions with computer-assisted detection software. Neuroradiol J 2013;26:143-150.
Thompson AJ, Banwell BL, Barkhof F, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol 2018;17:162-173.
Hagiwara A, Hori M, Yokoyama K, et al. Utility of a multiparametric quantitative MRI model that assesses myelin and edema for evaluating plaques, Periplaque white matter, and normal-appearing white matter in patients with multiple sclerosis: A feasibility study. Am J Neuroradiol 2017;38:237-242.
Lesjak Ž, Galimzianova A, Koren A, et al. A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 2018;16:51-63.
Fischl B, Salat DH, Busa E, et al. Whole brain segmentation. Neuron 2002;33:341-355.
Greve DN, Fischl B. Accurate and robust brain image alignment using boundary-based registration. Neuroimage 2009;48:63-72.
Wickens TD. Forced-choice procedures. Elementary signal detection theory. Oxford: Oxford University Press; 2001. p 93-112.
Daw NW. Why after-images are not seen in normal circumstances. Nature 1962;196:1143-1145.
Mehta S, Mercan E, Bartlett J, Weaver D, Elmore J, Shapiro L. Y-Net: Joint segmentation and classification for diagnosis of breast biopsy images. International conference on medical image computing and computer-assisted intervention. arXiv; 2018. p 893-901. https://arxiv.org/abs/1806.01313
Tan M, Le Q. EfficientNet: Rethinking model scaling for convolutional neural networks. In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th international conference on machine learning. arXiv; Vol 97; 2019. p 6105-6114. https://arxiv.org/abs/1905.11946
McHugh ML. Interrater reliability: The kappa statistic. Biochem Med 2012;22:276-282.
Shmueli O, Solomon C, Ben-Eliezer N, Greenspan H. Deep learning based multiple sclerosis lesion detection utilizing synthetic data generation and soft attention mechanism. In: Iftekharuddin KM, Drukker K, Mazurowski MA, Lu H, Muramatsu C, Samala RK, editors. Medical imaging 2022: Computer-aided diagnosis. San Diego: Computer-Aided Diagnosis; 2022. p 120330R.
Dahan A, Wang W, Gaillard F. Computer-aided detection can bridge the skill gap in multiple sclerosis monitoring. J Am Coll Radiol 2018;15:93-96.
Aljabri M, AlAmir M, AlGhamdi M, Abdel-Mottaleb M, Collado-Mesa F. Towards a better understanding of annotation tools for medical imaging: A survey. Multimed Tools Appl 2022;81:25877-25911.
Cha KH, Petrick N, Pezeshk A, et al. Evaluation of data augmentation via synthetic images for improved breast mass detection on mammograms using deep learning. J Med Imaging 2019;7:1.
MacKay A, Laule C, Vavasour I, Bjarnason T, Kolind S, Mädler B. Insights into brain microstructure from the T2 distribution. Magn Reson Imaging 2006;24:515-525.