A Multicenter Longitudinal MRI Study Assessing LeMan-PV Software Accuracy in the Detection of White Matter Lesions in Multiple Sclerosis Patients.
lesion activity
lesion segmentation
longitudinal analysis
longitudinal lesion segmentation
multiple sclerosis
white matter lesions
Journal
Journal of magnetic resonance imaging : JMRI
ISSN: 1522-2586
Titre abrégé: J Magn Reson Imaging
Pays: United States
ID NLM: 9105850
Informations de publication
Date de publication:
09 2023
09 2023
Historique:
revised:
13
01
2023
received:
24
10
2022
accepted:
14
01
2023
medline:
8
8
2023
pubmed:
29
1
2023
entrez:
28
1
2023
Statut:
ppublish
Résumé
Detecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking. To assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting. Retrospective, longitudinal. A total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2 years (range: 36.9-52.8 years); 70 males. Fluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T. The study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual "reference standard" provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T. Intraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers. The interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10 In this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow. 4 TECHNICAL EFFICACY: Stage 2.
Sections du résumé
BACKGROUND
Detecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking.
PURPOSE
To assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting.
STUDY TYPE
Retrospective, longitudinal.
SUBJECTS
A total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2 years (range: 36.9-52.8 years); 70 males.
FIELD STRENGTH/SEQUENCE
Fluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T.
ASSESSMENT
The study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual "reference standard" provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T.
STATISTICAL TESTS
Intraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers.
RESULTS
The interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10
DATA CONCLUSION
In this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow.
EVIDENCE LEVEL
4 TECHNICAL EFFICACY: Stage 2.
Types de publication
Multicenter Study
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
864-876Commentaires et corrections
Type : CommentIn
Informations de copyright
© 2023 The Authors. Journal of Magnetic Resonance Imaging published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.
Références
Lassmann H, Brück W, Lucchinetti CF. The immunopathology of multiple sclerosis: An overview. Brain Pathol 2007;17:210-218.
Thompson AJ, Banwell BL, Barkhof F, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol 2018;17:162-173.
Polman CH, Reingold SC, Banwell B, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 2011;69:292-302.
Lassmann H. Targets of therapy in progressive MS. Mult Scler 2017;23:1593-1599.
Río J, Castilló J, Rovira A, et al. Measures in the first year of therapy predict the response to interferon beta in MS. Mult Scler 2009;15:848-853.
Sormani MP, Bruzzi P. MRI lesions as a surrogate for relapses in multiple sclerosis: A meta-analysis of randomised trials. Lancet Neurol 2013;12:669-676.
Egger C, Opfer R, Wang C, et al. MRI FLAIR lesion segmentation in multiple sclerosis: Does automated segmentation hold up with manual annotation? Neuroimage (Amst) 2017;13:264-270.
Altay EE, Fisher E, Jones SE, Hara-Cleaver C, Lee JC, Rudick RA. Reliability of classifying multiple sclerosis disease activity using magnetic resonance imaging in a multiple sclerosis clinic. JAMA Neurol 2013;70:338-344.
Solomon J, Sood A. 4-D lesion detection using expectation-maximization and hidden markov model. 2004 2nd IEEE Int Symp Biomed Imaging Macro to Nano, Vol 1 United States: IEEE; 2004. p 125-128.
Köhler C, Wahl H, Ziemssen T, Linn J, Kitzler HH. Exploring individual multiple sclerosis lesion volume change over time: Development of an algorithm for the analyses of longitudinal quantitative MRI measures. NeuroImage Clin 2019;21:101623.
Lladó X, Ganiler O, Oliver A, et al. Automated detection of multiple sclerosis lesions in serial brain MRI. Neuroradiology 2012;54:787-807.
Moraal B, Wattjes MP, Geurts JJG, et al. Improved detection of active multiple sclerosis lesions: 3D subtraction imaging. Radiology 2010;255:154-163.
Jain S, Ribbens A, Sima DM, et al. Two time point MS lesion segmentation in brain MRI: An expectation-maximization framework. Front Neurosci 2016;10.
Fartaria MJ, Todea A, Kober T, et al. Partial volume-aware assessment of multiple sclerosis lesions. NeuroImage Clin 2018;18:245-253.
Fartaria MJ, Kober T, Granziera C, Bach Cuadra M. Longitudinal analysis of white matter and cortical lesions in multiple sclerosis. NeuroImage Clin 2019;23:101938.
Fartaria MJ, Bonnier G, Roche A, et al. Automated detection of white matter and cortical lesions in early stages of multiple sclerosis. J Magn Reson Imaging 2016;43:1445-1454.
Disanto G, Benkert P, Lorscheider J, et al. The Swiss multiple sclerosis cohort-study (SMSC): A prospective Swiss wide investigation of key phases in disease evolution and new treatment options. PLoS One 2016;11:e0152347.
Kurtzke JF. Rating neurologic impairment in multiple sclerosis. Neurology 1983;33:1444-1452.
Yushkevich PA, Piven J, Hazlett HC, et al. User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability. Neuroimage 2006;31:1116-1128.
Filippi M, Preziosa P, Banwell BL, et al. Assessment of lesions on magnetic resonance imaging in multiple sclerosis: Practical guidelines. Brain 2019;142:1858-1875.
Klein S, Staring M, Murphy K, Viergever MA, Pluim JPW. Elastix: A toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 2010;29:196-205.
Rovira A, Tintoré M, Álvarez-Cermeño JC, Izquierdo G, Prieto JM. Recommendations for using and interpreting magnetic resonance imaging in multiple sclerosis. Neurologia 2010;25:248-265.
Valverde S, Oliver A, Lladó X. A white matter lesion-filling approach to improve brain tissue volume measurements. NeuroImage Clin 2014;6:86-92.
McHugh ML. Interrater reliability: The kappa statistic. Biochem Med 2012;22:276-282.
Liljequist D, Elfving B, Roaldsen KS. Intraclass correlation - A discussion and demonstration of basic features. PLoS One 2019;14:e0219854.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155-163.
Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979;86:420-428.
Cabezas M, Oliver A, Roura E, et al. Automatic multiple sclerosis lesion detection in brain MRI by FLAIR thresholding. Comput Methods Prog Biomed 2014;115:147-161.
Kaunzner UW, Gauthier SA. MRI in the assessment and monitoring of multiple sclerosis: An update on best practice. Ther Adv Neurol Disord 2017;10:247-261.
McKinley R, Wepfer R, Grunder L, et al. Automatic detection of lesion load change in multiple sclerosis using convolutional neural networks with segmentation confidence. NeuroImage Clin 2020;25:102104.
Disanto G, Berlanga AJ, Handel AE, et al. Heterogeneity in multiple sclerosis: Scratching the surface of a complex disease. Autoimmune Dis 2010;2011:1-12.
Krüger J, Opfer R, Gessert N, et al. Fully automated longitudinal segmentation of new or enlarged multiple sclerosis lesions using 3D convolutional neural networks. NeuroImage Clin 2020;28:102445.