Reproducibility of arterial spin labeling cerebral blood flow image processing: A report of the ISMRM open science initiative for perfusion imaging (OSIPI)_and the ASL MRI challenge.
ASL
cerebral blood flow
challenges
image analysis
reproducibility
Journal
Magnetic resonance in medicine
ISSN: 1522-2594
Titre abrégé: Magn Reson Med
Pays: United States
ID NLM: 8505245
Informations de publication
Date de publication:
19 Mar 2024
19 Mar 2024
Historique:
revised:
20
02
2024
received:
28
04
2023
accepted:
21
02
2024
medline:
19
3
2024
pubmed:
19
3
2024
entrez:
19
3
2024
Statut:
aheadofprint
Résumé
Arterial spin labeling (ASL) is a widely used contrast-free MRI method for assessing cerebral blood flow (CBF). Despite the generally adopted ASL acquisition guidelines, there is still wide variability in ASL analysis. We explored this variability through the ISMRM-OSIPI ASL-MRI Challenge, aiming to establish best practices for more reproducible ASL analysis. Eight teams analyzed the challenge data, which included a high-resolution T1-weighted anatomical image and 10 pseudo-continuous ASL datasets simulated using a digital reference object to generate ground-truth CBF values in normal and pathological states. We compared the accuracy of CBF quantification from each team's analysis to the ground truth across all voxels and within predefined brain regions. Reproducibility of CBF across analysis pipelines was assessed using the intra-class correlation coefficient (ICC), limits of agreement (LOA), and replicability of generating similar CBF estimates from different processing approaches. Absolute errors in CBF estimates compared to ground-truth synthetic data ranged from 18.36 to 48.12 mL/100 g/min. Realistic motion incorporated into three datasets produced the largest absolute error and variability between teams, with the least agreement (ICC and LOA) with ground-truth results. Fifty percent of the submissions were replicated, and one produced three times larger CBF errors (46.59 mL/100 g/min) compared to submitted results. Variability in CBF measurements, influenced by differences in image processing, especially to compensate for motion, highlights the significance of standardizing ASL analysis workflows. We provide a recommendation for ASL processing based on top-performing approaches as a step toward ASL standardization.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Wellcome Trust
ID : Sir Henry Dale Fellowship 220204/Z/20/Z
Pays : United Kingdom
Informations de copyright
© 2024 The Authors. Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.
Références
Detre JA, Leigh JS, Williams DS, Koretsky AP. Perfusion imaging. Magn Reson med. 1992;23:37-45.
Alsop DC, Detre JA, Golay X, et al. Recommended implementation of arterial spin-labeled perfusion mri for clinical applications: a consensus of the ISMRM perfusion study group and the European consortium for ASL in dementia. Magn Reson med. 2015;73:102-116. doi:10.1002/mrm.25197
Pujol S, Wells W, Pierpaoli C, et al. The DTI challenge: toward standardized evaluation of diffusion tensor imaging Tractography for neurosurgery. J Neuroimaging. 2015;25:875-882. doi:10.1111/jon.12283
Grissom WA, Setsompop K, Hurley SA, Tsao J, Velikina JV, Samsonov AA. Advancing RF pulse design using an open-competition format: report from the 2015 ISMRM challenge. Magn Reson med. 2017;78:1352-1361. doi:10.1002/mrm.26512
Nath V, Schilling KG, Parvathaneni P, et al. Tractography reproducibility challenge with empirical data (TraCED): the 2017 ISMRM diffusion study group challenge. J Magn Reson Imaging. 2020;51:234-249. doi:10.1002/jmri.26794
Schilling KG, Daducci A, Maier-Hein K, et al. Challenges in diffusion MRI tractography-lessons learned from international benchmark competitions. Magn Reson Imaging. 2019;57:194-209. doi:10.1016/j.mri.2018.11.014
Veronese M, Rizzo G, Belzunce M, et al. Reproducibility of findings in modern PET neuroimaging: insight from the NRM2018 grand challenge. J Cereb Blood Flow Metab. 2021;41:2778-2796. doi:10.1177/0271678X211015101
Maffei C, Girard G, Schilling KG, et al. Insights from the IronTract challenge: optimal methods for mapping brain pathways from multi-shell diffusion MRI. Neuroimage. 2022;257:119327. doi:10.1016/j.neuroimage.2022.119327
Bossier H, Roels SP, Seurinck R, et al. The empirical replicability of task-based fMRI as a function of sample size. Neuroimage. 2020;212:116601. doi:10.1016/j.neuroimage.2020.116601
Fanelli D. Is science really facing a reproducibility crisis, and do we need it to? Proc Natl Acad Sci. 2018;115:2628-2631. doi:10.1073/pnas.1708272114
Botvinik-Nezer R, Holzmeister F, Camerer CF, et al. Variability in the analysis of a single neuroimaging dataset by many teams. Nature. 2020;582:84-88. doi:10.1038/s41586-020-2314-9
Anazodo U, Pinto J, Mcconnell FAK, et al. The Open Source Initiative for Perfusion Imaging (OSIPI) ASL MRI Challenge. In Proceedings of the 29th Annual Meeting of the International Society of Magnetic Resonance in Medicine. Vol c. Virtual Meeting; 2021:1-3. Abstract 2714.
Clement P, Petr J, Dijsselhof MBJ, et al. A Beginner's guide to arterial spin labeling (ASL) image processing. Front Radiol. 2022;2:929533. doi:10.3389/fradi.2022.929533
Dai W, Garcia D, de Bazelaire C, Alsop DC. Continuous flow-driven inversion for arterial spin labeling using pulsed radio frequency and gradient fields. Magn Reson med. 2008;60:1488-1497. doi:10.1002/mrm.21790
Clement P, Castellaro M, Okell TW, et al. ASL-BIDS, the brain imaging data structure extension for arterial spin labeling. Sci Data. 2022;9:543. doi:10.1038/s41597-022-01615-9
Anazodo U, Croal P, Paschoal AM. OSIPI ASL MRI Challenge. 2021. doi:10.17605/OSF.IO/6XYU3
Lorenzini L, Ingala S, Wink AM, et al. The open-access European prevention of Alzheimer's dementia (EPAD) MRI dataset and processing workflow. Neuroimage Clin. 2022;35:103106. doi:10.1016/j.nicl.2022.103106
Oliver-Taylor AM, Hampshire T, Stritt M, et al. ASLDRO: digital reference object software for arterial spin labelling. In Proceedings of the 29th Annual Meeting of the International Society of Magnetic Resonance in Medicine. Vol 2731. 2021. Virtual https://pypi.org/project/asldro/ Abstract 2731.
Van Essen DC, Smith SM, Barch DM, Behrens TEJ, Yacoub E, Ugurbil K. The WU-Minn human connectome project: an overview. Neuroimage. 2013;80:62-79. doi:10.1016/j.neuroimage.2013.05.041
Wickham H. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag; 2016 https://ggplot2.tidyverse.org
Schwalbe M, ed. Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results. National Academies Press; 2016.
Gerke O. Reporting standards for a bland-altman agreement analysis: a review of methodological reviews. Diagnostics. 2020;10:1-17. doi:10.3390/diagnostics10050334
Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr med. 2016;15:155-163. doi:10.1016/j.jcm.2016.02.012
R Core Team. R: A Language and Environment for Statistical Computing. Foundation for Statistical Computing https://www.r-project.org/; https://www.r-project.org/ 2020.
Nichols TE, das S, Eickhoff SB, et al. Best practices in data analysis and sharing in neuroimaging using MRI. Nat Neurosci. 2017;20:299-303. doi:10.1038/nn.4500
Praag CGV. COBIDAS reporting breakdown.
O'Keefe S. Calculating document quality (QUACK). Scriptorium. Accessed November 9, 2023. https://www.scriptorium.com/2010/05/calculating-document-quality-quack/. Published May 14, 2010
Buxton RB, Frank LR, Wong EC, Siewert B, Warach S, Edelman RR. A general kinetic model for quantitative perfusion imaging with arterial spin labeling. Magn Reson med. 1998;40:383-396.
ASL Analysis Tab-Quantiphyse documentation. Accessed November 9, 2023. https://quantiphyse.readthedocs.io/en/latest/asl/asl_analysis.html
Chappell MA, Groves AR, Whitcher B, Woolrich MW. Variational Bayesian inference for a nonlinear forward model. IEEE Trans Signal Process. 2009;57:223-236. doi:10.1109/TSP.2008.2005752
Mutsaerts HJMM, Petr J, Groot P, et al. ExploreASL: an image processing pipeline for multi-center ASL perfusion MRI studies. Neuroimage. 2020;219:117031. doi:10.1016/j.neuroimage.2020.117031
MRIcloud. Accessed November 9, 2023. https://braingps.mricloud.org/docs/Manual_ASL_processing.v2.pdf.
Bron EE, Steketee RME, Houston GC, et al. Diagnostic classification of arterial spin labeling and structural MRI in presenile early stage dementia. Hum Brain Mapp. 2014;35:4916-4931. doi:10.1002/hbm.22522
Zhao C. LOFT_ASL_toolbox. GitHub. 2022. Accessed November 9, 2023. https://github.com/chenyang9526/LOFT_ASL_toolbox
Fan H, Mutsaerts HJMM, Anazodo U, et al. ISMRM Open Science initiative for perfusion imaging (OSIPI): ASL pipeline inventory. Magn Reson Med. 2023;1-16. doi:10.1002/mrm.29869
Adebimpe A, Bertolero M, Dolui S, et al. ASLPrep: a platform for processing of arterial spin labeled MRI and quantification of regional brain perfusion. Nat Methods. 2022;19:683-686. doi:10.1038/s41592-022-01458-7
Wang Z, Aguirre GK, Rao H, et al. Empirical optimization of ASL data analysis using an ASL data processing toolbox: ASLtbx. Magn Reson Imaging. 2008;26:261-269. doi:10.1016/j.mri.2007.07.003
Groves AR, Chappell MA, Woolrich MW. Combined spatial and non-spatial prior for inference on MRI time-series. Neuroimage. 2009;45:795-809. doi:10.1016/j.neuroimage.2008.12.027
Almeida JRC, Greenberg T, Lu H, et al. Test-retest reliability of cerebral blood flow in healthy individuals using arterial spin labeling: findings from the EMBARC study. Magn Reson Imaging. 2018;45:26-33. doi:10.1016/j.mri.2017.09.004
Ssali T, Anazodo UC, Bureau Y, MacIntosh BJ, Günther M, St Lawrence K. Mapping long-term functional changes in cerebral blood flow by arterial spin labeling. PLoS ONE. 2016;11:e0164112. doi:10.1371/journal.pone.0164112
COBIDAS checklist. July 2019. 10.17605/OSF.IO/ANVQY.
Dolui S, Tisdall D, Vidorreta M, et al. Characterizing a perfusion-based periventricular small vessel region of interest. Neuroimage Clin. 2019;23:101897. doi:10.1016/j.nicl.2019.101897
Shirzadi Z, Stefanovic B, Chappell MA, et al. Enhancement of automated blood flow estimates (ENABLE) from arterial spin-labeled MRI. J Magn Reson Imaging. 2018;47:647-655. doi:10.1002/jmri.25807
Mutsaerts HJMM, Petr J, Thomas DL, et al. Comparison of arterial spin labeling registration strategies in the multi-center GENetic frontotemporal dementia initiative (GENFI). J Magn Reson Imaging. 2018;47:131-140. doi:10.1002/jmri.25751
Pauli R, Bowring A, Reynolds R, Chen G, Nichols TE, Maumet C. Exploring fMRI results space: 31 variants of an fMRI analysis in AFNI, FSL, and SPM. Front Neuroinformatics. 2016;10:10. doi:10.3389/fninf.2016.00024
Kazemi K, Noorizadeh N. Quantitative comparison of SPM, FSL, and Brainsuite for brain MR image segmentation. J Biomed Phys Eng. 2014;4:13-26.
Seiger R, Ganger S, Kranz GS, Hahn A, Lanzenberger R. Cortical thickness estimations of FreeSurfer and the CAT12 toolbox in patients with Alzheimer's disease and healthy controls. J Neuroimaging. 2018;28:515-523. doi:10.1111/jon.12521
Marjańska M, Deelchand DK, Kreis R. The 2016 ISMRM MRS study group fitting challenge team. Results and interpretation of a fitting challenge for MR spectroscopy set up by the MRS study group of ISMRM. Magn Reson med. 2022;87:11-32. doi:10.1002/mrm.28942