The case for preregistering all region of interest (ROI) analyses in neuroimaging research.
Journal
The European journal of neuroscience
ISSN: 1460-9568
Titre abrégé: Eur J Neurosci
Pays: France
ID NLM: 8918110
Informations de publication
Date de publication:
01 2021
01 2021
Historique:
received:
08
04
2020
revised:
16
07
2020
accepted:
12
08
2020
pubmed:
28
8
2020
medline:
29
6
2021
entrez:
28
8
2020
Statut:
ppublish
Résumé
In neuroimaging studies, small sample sizes and the resultant reduced statistical power to detect effects that are not large, combined with inadequate analytic choices, concur to produce inflated or false-positive findings. To mitigate these issues, researchers often restrict analyses to specific brain areas, using the region of interest (ROI) approach. Crucially, ROI analysis assumes the a priori justified definition of the target region. Nonetheless, reports often lack details about where in the timeline, ranging from study conception to the data analysis and interpretation of findings, were ROIs selected. Frequently, the rationale for ROI selection is vague or inadequately founded on the existing literature. These shortcomings have important implications for ROI-based studies, augmenting the risk that observed effects are inflated or even false positives. Tools like preregistration and registered reports could address this problem, ensuring the validity of ROI-based studies. The benefits could be enhanced by additional practices such as selection of ROIs using quantitative methods (i.e., meta-analysis) and the sharing of whole-brain unthresholded maps of effect size, as well as of binary ROIs, in publicly accessible repositories.
Types de publication
Editorial
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
357-361Informations de copyright
© 2020 Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
Références
Albajes-Eizagirre, A., Solanes, A., Vieta, E., & Radua, J. (2019). Voxel-based meta-analysis via permutation of subject images (PSI): Theory and implementation for SDM. NeuroImage, 186, 174-184. https://doi.org/10.1016/j.neuroimage.2018.10.077
Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M. … Schonberg, T. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582, 84-88. https://doi.org/10.1038/s41586-020-2314-9
Bruns, S. B., & Ioannidis, J. P. A. (2016). p-Curve and p-Hacking in observational research. PLoS One, 11, e0149144. https://doi.org/10.1371/journal.pone.0149144
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafo, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. https://doi.org/10.1038/nrn3475
Chambers, C. D. (2013). Registered reports: A new publishing initiative at Cortex. Cortex, 49, 609-610.
Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017). fMRI clustering and false-positive rates. Proceedings of the National Academy of Sciences, 114(17), E3370-E3371.
Dockès, J., Poldrack, R. A., Primet, R., Gözükan, H., Yarkoni, T., Suchanek, F., … Varoquaux, G. (2020). NeuroQuery, comprehensive meta-analysis of human brain mapping. eLife, 9. https://doi.org/10.7554/eLife.53385
Eklund, A., Nichols, T. E., & Knutsson, H. (2016). Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences of the United States of America, 113, 7900-7905. https://doi.org/10.1073/pnas.1602413113
Gelman, A., & Loken, E. (2014). The statistical crisis in science. American Scientist, 102, 460. https://doi.org/10.1511/2014.111.460
Gentili, C., Messerotti Benvenuti, S., Lettieri, G., Costa, C., & Cecchetti, L. (2019). ROI and phobias: The effect of ROI approach on an ALE meta-analysis of specific phobias. Human Brain Mapping, 40, 1814-1828. https://doi.org/10.1002/hbm.24492
Gorgolewski, K. J., Alfaro-Almagro, F., Auer, T., Bellec, P., Capotă, M., Chakravarty, M. M., … Poldrack, R. A. (2017). BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods. PLoS Computational Biology, 13, e1005209. https://doi.org/10.1371/journal.pcbi.1005209
Gorgolewski, K. J., Varoquaux, G., Rivera, G., Schwarz, Y., Ghosh, S. S., Maumet, C., … Margulies, D. S. (2015). NeuroVault.org: A web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Front Neuroinform, 9, 8. https://doi.org/10.3389/fninf.2015.00008
Hardwicke, T. E., & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793-796. https://doi.org/10.1038/s41562-018-0444-y
Haynes, J. D. (2015). A primer on pattern-based approaches to fMRI: Principles, pitfalls, and perspectives. Neuron, 87, 257-270. https://doi.org/10.1016/j.neuron.2015.05.025
Hong, Y. W., Yoo, Y., Han, J., Wager, T. D., & Woo, C. W. (2019). False-positive neuroimaging: Undisclosed flexibility in testing spatial hypotheses allows presenting anything as a replicated finding. NeuroImage, 195, 384-395. https://doi.org/10.1016/j.neuroimage.2019.03.070
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Med, 2, e124.
Ioannidis, J. P. (2008). Why most discovered true associations are inflated. Epidemiology, 19(5), 640-648.
Keator, D. B., Helmer, K., Steffener, J., Turner, J. A., Van Erp, T. G., Gadde, S., … Nichols, B. N. (2013). Towards structured sharing of raw and derived neuroimaging data across existing resources. NeuroImage, 82, 647-661. https://doi.org/10.1016/j.neuroimage.2013.05.094
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in systems neuroscience: The dangers of double dipping. Nature Neuroscience, 12, 535-540. https://doi.org/10.1038/nn.2303
Lettieri, G., Handjaras, G., Ricciardi, E., Leo, A., Papale, P., Betta, M., … Cecchetti, L. (2019). Emotionotopy in the human right temporo-parietal cortex. Nature Communications, 10, 5568. https://doi.org/10.1038/s41467-019-13599-z
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., … Eickhoff, S. B. (2018). Ten simple rules for neuroimaging meta-analysis. Neuroscience & Biobehavioral Reviews, 84, 151-161.
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., … Yeo, B. T. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nature Neuroscience, 20, 299-303. https://doi.org/10.1038/nn.4500
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences of the United States of America, 115, 2600-2606. https://doi.org/10.1073/pnas.1708274114
Poldrack, R. A. (2007). Region of interest analysis for fMRI. Soc Cogn Affect Neurosci, 2, 67-70. https://doi.org/10.1093/scan/nsm006
Poldrack, R., Baker, C., Durnez, J., Gorgolewski, K., Matthews, P., Munafò, M., … Yarkoni, T. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, 18. https://doi.org/10.1038/nrn.2016.167
Reddan, M. C., Lindquist, M. A., & Wager, T. D. (2017). Effect size estimation in neuroimaging. JAMA Psychiatry, 74, 207-208.
Srivastava, S. (2018). Sound inference in complicated research: A multi-strategy approach. PsyArXiv.
Szucs, D., & Ioannidis, J. P. (2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology, 15, e2000797. https://doi.org/10.1371/journal.pbio.2000797
Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. NeuroImage, 180, 68-77.
Varoquaux, G., Raamana, P. R., Engemann, D. A., Hoyos-Idrobo, A., Schwartz, Y., & Thirion, B. (2017). Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. NeuroImage, 145, 166-179.
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J. J., Appleton, G., Axton, M., Baak, A., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
Yamada, Y. (2018). How to crack pre-registration: Toward transparent and open science. Front Psychol, 9, 1831. https://doi.org/10.3389/fpsyg.2018.01831
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8, 665-670. https://doi.org/10.1038/nmeth.1635