Tractography reproducibility challenge with empirical data (TraCED): The 2017 ISMRM diffusion study group challenge.
DW-MRI
HARDI
challenge
in vivo
reproducibility
tractography
Journal
Journal of magnetic resonance imaging : JMRI
ISSN: 1522-2586
Titre abrégé: J Magn Reson Imaging
Pays: United States
ID NLM: 9105850
Informations de publication
Date de publication:
01 2020
01 2020
Historique:
received:
07
01
2019
accepted:
06
05
2019
pubmed:
11
6
2019
medline:
14
4
2021
entrez:
11
6
2019
Statut:
ppublish
Résumé
Fiber tracking with diffusion-weighted MRI has become an essential tool for estimating in vivo brain white matter architecture. Fiber tracking results are sensitive to the choice of processing method and tracking criteria. To assess the variability for an algorithm in group studies reproducibility is of critical context. However, reproducibility does not assess the validity of the brain connections. Phantom studies provide concrete quantitative comparisons of methods relative to absolute ground truths, yet do no capture variabilities because of in vivo physiological factors. The ISMRM 2017 TraCED challenge was created to fulfill the gap. A systematic review of algorithms and tract reproducibility studies. Single healthy volunteers. 3.0T, two different scanners by the same manufacturer. The multishell acquisition included b-values of 1000, 2000, and 3000 s/mm Nine international groups submitted 46 tractography algorithm entries each consisting 16 tracts per scan. The algorithms were assessed using intraclass correlation (ICC) and the Dice similarity measure. Containment analysis was performed to assess if the submitted algorithms had containment within tracts of larger volume submissions. This also serves the purpose to detect if spurious submissions had been made. The top five submissions had high ICC and Dice >0.88. Reproducibility was high within the top five submissions when assessed across sessions or across scanners: 0.87-0.97. Containment analysis shows that the top five submissions are contained within larger volume submissions. From the total of 16 tracts as an outcome relatively the number of tracts with high, moderate, and low reproducibility were 8, 4, and 4. The different methods clearly result in fundamentally different tract structures at the more conservative specificity choices. Data and challenge infrastructure remain available for continued analysis and provide a platform for comparison. 5 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:234-249.
Sections du résumé
BACKGROUND
Fiber tracking with diffusion-weighted MRI has become an essential tool for estimating in vivo brain white matter architecture. Fiber tracking results are sensitive to the choice of processing method and tracking criteria.
PURPOSE
To assess the variability for an algorithm in group studies reproducibility is of critical context. However, reproducibility does not assess the validity of the brain connections. Phantom studies provide concrete quantitative comparisons of methods relative to absolute ground truths, yet do no capture variabilities because of in vivo physiological factors. The ISMRM 2017 TraCED challenge was created to fulfill the gap.
STUDY TYPE
A systematic review of algorithms and tract reproducibility studies.
SUBJECTS
Single healthy volunteers.
FIELD STRENGTH/SEQUENCE
3.0T, two different scanners by the same manufacturer. The multishell acquisition included b-values of 1000, 2000, and 3000 s/mm
ASSESSMENT
Nine international groups submitted 46 tractography algorithm entries each consisting 16 tracts per scan. The algorithms were assessed using intraclass correlation (ICC) and the Dice similarity measure.
STATISTICAL TESTS
Containment analysis was performed to assess if the submitted algorithms had containment within tracts of larger volume submissions. This also serves the purpose to detect if spurious submissions had been made.
RESULTS
The top five submissions had high ICC and Dice >0.88. Reproducibility was high within the top five submissions when assessed across sessions or across scanners: 0.87-0.97. Containment analysis shows that the top five submissions are contained within larger volume submissions. From the total of 16 tracts as an outcome relatively the number of tracts with high, moderate, and low reproducibility were 8, 4, and 4.
DATA CONCLUSION
The different methods clearly result in fundamentally different tract structures at the more conservative specificity choices. Data and challenge infrastructure remain available for continued analysis and provide a platform for comparison.
LEVEL OF EVIDENCE
5 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;51:234-249.
Identifiants
pubmed: 31179595
doi: 10.1002/jmri.26794
pmc: PMC6900461
mid: NIHMS1029852
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
234-249Subventions
Organisme : NCATS NIH HHS
ID : UL1 TR000445
Pays : United States
Organisme : NIBIB NIH HHS
ID : T32 EB001628
Pays : United States
Organisme : NIBIB NIH HHS
ID : R01 EB017230
Pays : United States
Organisme : NINDS NIH HHS
ID : R01 NS096606
Pays : United States
Organisme : NCRR NIH HHS
ID : UL1 RR024975-01
Pays : United States
Organisme : NCRR NIH HHS
ID : UL1 RR024975
Pays : United States
Informations de copyright
© 2019 International Society for Magnetic Resonance in Medicine.
Références
IEEE Trans Med Imaging. 2014 Feb;33(2):384-99
pubmed: 24132007
J Neurosci. 2016 Jun 22;36(25):6758-70
pubmed: 27335406
J Neuroimaging. 2015 Nov-Dec;25(6):875-82
pubmed: 26259925
Med Image Anal. 2001 Jun;5(2):143-56
pubmed: 11516708
Neuroimage. 2003 Oct;20(2):870-88
pubmed: 14568458
Neuroimage. 2008 Jan 1;39(1):62-79
pubmed: 17919935
Magn Reson Med. 2009 Jun;61(6):1336-49
pubmed: 19319973
Cereb Cortex. 2015 Nov;25(11):4628-37
pubmed: 26048951
PLoS One. 2012;7(11):e49790
pubmed: 23166771
Neuroimage. 2002 Sep;17(1):77-94
pubmed: 12482069
Neuroimage. 2019 Jan 15;185:1-11
pubmed: 30317017
Neuroimage. 2018 Apr 15;170:283-295
pubmed: 28712994
Brain Struct Funct. 2016 Dec;221(9):4705-4721
pubmed: 26754839
Neuroimage. 2004 Nov;23(3):1176-85
pubmed: 15528117
Med Image Anal. 2013 Oct;17(7):844-57
pubmed: 23706753
Cereb Cortex. 2015 Nov;25(11):4299-309
pubmed: 25787833
Magn Reson Med. 2009 Sep;62(3):717-30
pubmed: 19623619
Neuroimage. 2016 Sep;138:197-210
pubmed: 27184203
IEEE Trans Med Imaging. 2015 Nov;34(11):2320-32
pubmed: 25966471
Neuroimage. 2018 Dec;183:239-253
pubmed: 30086412
Med Image Comput Comput Assist Interv. 2016 Oct;9900:201-209
pubmed: 28090602
Hum Brain Mapp. 2015 Oct;36(10):4116-34
pubmed: 26178765
Neuroimage. 2016 Feb 15;127:158-172
pubmed: 26691524
Magn Reson Med. 2014 Nov;72(5):1460-70
pubmed: 24323973
Proc SPIE Int Soc Opt Eng. 2017 Feb 11;10133:
pubmed: 28736471
J Med Imaging (Bellingham). 2018 Jan;5(1):014005
pubmed: 29531965
NMR Biomed. 2019 Apr;32(4):e3785
pubmed: 28945294
Biophys J. 1994 Jan;66(1):259-67
pubmed: 8130344
IEEE Trans Med Imaging. 2001 Jan;20(1):45-57
pubmed: 11293691
Neuroimage. 2007 Oct 1;37(4):1267-77
pubmed: 17706434
Nat Commun. 2017 Nov 7;8(1):1349
pubmed: 29116093
Neuroimage. 2007 Aug 15;37(2):530-8
pubmed: 17604650
Neuroimage. 2016 Oct 1;139:346-359
pubmed: 27282476
IEEE Trans Med Imaging. 2015 Jan;34(1):246-57
pubmed: 25167548
Proc Natl Acad Sci U S A. 2014 Nov 18;111(46):16574-9
pubmed: 25368179
Hum Brain Mapp. 2013 Nov;34(11):2747-66
pubmed: 22611035
Neuroimage. 2006 Jul 15;31(4):1487-505
pubmed: 16624579
Magn Reson Imaging. 2017 Jul;40:62-74
pubmed: 28438712
Magn Reson Med. 2004 Dec;52(6):1358-72
pubmed: 15562495
Neuroimage. 2014 Sep;98:266-78
pubmed: 24816531
Front Neuroinform. 2014 Feb 21;8:8
pubmed: 24600385
Magn Reson Imaging. 2019 Apr;57:194-209
pubmed: 30503948
Brain. 2007 Mar;130(Pt 3):630-53
pubmed: 17293361
Neuroimage. 2004;23 Suppl 1:S208-19
pubmed: 15501092
Med Image Anal. 2015 Dec;26(1):316-31
pubmed: 26606457
Med Image Anal. 2013 Feb;17(2):194-208
pubmed: 23265798
Neuroimage. 2007 May 1;35(4):1459-72
pubmed: 17379540
Proc SPIE Int Soc Opt Eng. 2016 Feb 27;9784:
pubmed: 27127332
Neuroimage. 2007 Jul 1;36(3):630-44
pubmed: 17481925
Neuroimage. 2016 Jan 15;125:1063-1078
pubmed: 26481672