Effect sizes of associations between neuroimaging measures and affective symptoms: A meta-analysis.
brain-affective symptom associations
effect size
meta-analysis
power
study sample size
translational neuroimaging
Journal
Depression and anxiety
ISSN: 1520-6394
Titre abrégé: Depress Anxiety
Pays: United States
ID NLM: 9708816
Informations de publication
Date de publication:
01 2022
01 2022
Historique:
revised:
14
07
2021
received:
11
04
2021
accepted:
20
08
2021
pubmed:
14
9
2021
medline:
8
3
2022
entrez:
13
9
2021
Statut:
ppublish
Résumé
The utility of brain-based biomarkers for psychiatric disorders hinges among other factors on their ability to explain a significant portion of the phenotypic variance. In particular, many small scale studies have been unable to arbitrate whether structural or functional magnetic resonance imaging has potential to be a biological marker for these disorders. This study conducted a meta-analysis to examine the relationship between study power and published effect sizes for the relationship between affective symptoms and structural or functional magnetic resonance imaging measures. The current analyses are based on 821 brain-affective symptom association effect sizes derived from 120 publications, which employed a univariate region-of-interest approach. For self-assessed affective symptoms published brain imaging measures accounted for on average 8% (confidence interval: 1.6%-23%) of between-subject variation. This average effect size was based mostly on studies with small sample sizes, which have likely led to inflation of these effect size estimates. These findings support the conclusion that brain imaging measures currently account for a smaller proportion of the interindividual variance in affective symptoms than has been previously reported. The current findings support the need for both large-sample clinical studies and new statistical and theoretical models to more robustly capture systematic variance of brain-affective symptom relationships.
Sections du résumé
BACKGROUND
The utility of brain-based biomarkers for psychiatric disorders hinges among other factors on their ability to explain a significant portion of the phenotypic variance. In particular, many small scale studies have been unable to arbitrate whether structural or functional magnetic resonance imaging has potential to be a biological marker for these disorders.
METHODS
This study conducted a meta-analysis to examine the relationship between study power and published effect sizes for the relationship between affective symptoms and structural or functional magnetic resonance imaging measures. The current analyses are based on 821 brain-affective symptom association effect sizes derived from 120 publications, which employed a univariate region-of-interest approach.
RESULTS
For self-assessed affective symptoms published brain imaging measures accounted for on average 8% (confidence interval: 1.6%-23%) of between-subject variation. This average effect size was based mostly on studies with small sample sizes, which have likely led to inflation of these effect size estimates.
CONCLUSIONS
These findings support the conclusion that brain imaging measures currently account for a smaller proportion of the interindividual variance in affective symptoms than has been previously reported. The current findings support the need for both large-sample clinical studies and new statistical and theoretical models to more robustly capture systematic variance of brain-affective symptom relationships.
Types de publication
Journal Article
Meta-Analysis
Research Support, Non-U.S. Gov't
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
19-25Informations de copyright
© 2021 Wiley Periodicals LLC.
Références
Abi-Dargham, A., & Horga, G. (2016). The search for imaging biomarkers in psychiatric disorders. Nature Medicine, 22(11), 1248-1255.
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100(2), 431-449. https://doi.org/10.1037/a0038047
Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M., Kirchler, M., Iwanir, R., Mumford, J. A., Adcock, R. A., Avesani, P., Baczkowski, B. M., Bajracharya, A., Bakst, L., Ball, S., Barilari, M., Bault, N., Beaton, D., Beitner, J., … Rieck, J. R. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582(7810), 84-88. https://doi.org/10.1038/s41586-020-2314-9
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365-376.
Canli, T., Zhao, Z., Desmond, J. E., Kang, E., Gross, J., & Gabrieli, J. D. (2001). An fMRI study of personality influences on brain reactivity to emotional stimuli. Behavioral Neuroscience, 115(1), 33-42.
Casey, B. J., Cannonier, T., Conley, M. I., Cohen, A. O., Barch, D. M., Heitzeg, M. M., Soules, M. E., Teslovich, T., Dellarco, D. V., Garavan, H., Orr, C. A., Wager, T. D., Banich, M. T., Speer, N. K., Sutherland, M. T., Riedel, M. C., Dick, A. S., Bjork, J. M., Thomas, K. M., … ABCD Imaging Acquisition, W. (2018). The adolescent brain cognitive development (ABCD) study: Imaging acquisition across 21 sites. Developmental Cognitive Neuroscience, 32, 43-54.
Champely, S., Ekstrom, C., Dalgaard, P., Gill, J., Weibelzahl, S., Anandkumar, A., & De Rosario, M. H. (2018). Package ‘pwr’. R package version, 1-2.
Cremers, H. R., Wager, T. D., & Yarkoni, T. (2017). The relation between statistical power and inference in fMRI. PLoS One, 12(11), 0184923.
Esteban, O., Ciric, R., Finc, K., Blair, R. W., Markiewicz, C. J., Moodie, C. A., Kent, J. D., Goncalves, M., DuPre, E., Gomez, D., Ye, Z., Salo, T., Valabregue, R., Amlien, I. K., Liem, F., Jacoby, N., Stojić, H., Cieslak, M., Urchs, S., … Gorgolewski, K. J. (2020). Analysis of task-based functional MRI data preprocessed with fMRIPrep. Nature Protocols, 15, 2186-2202. https://doi.org/10.1038/s41596-020-0327-3
Gignac, G. E., & Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74-78.
Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015). The fickle P value generates irreproducible results. Nature Methods, 12(3), 179-185.
Hemphill, J. F. (2003). Interpreting the magnitudes of correlation coefficients.
Huhn, M., Tardy, M., Spineli, L. M., Kissling, W., Förstl, H., Pitschel-Walz, G., Leucht, C., Samara, M., Dold, M., Davis, J. M., & Leucht, S. (2014). Efficacy of pharmacotherapy and psychotherapy for adult psychiatric disorders: A systematic overview of meta-analyses. JAMA Psychiatry, 71(6), 706-715. https://doi.org/10.1001/jamapsychiatry.2014.112
Ingre, M. (2013). Why small low-powered studies are worse than large high-powered studies and how to protect against “trivial” findings in research: Comment on Friston (2012). NeuroImage, 81, 496-498.
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., & Wang, P. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. In: Am Psychiatric Assoc.
Ioannidis, J. P. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640-648.
Kotov, R., Krueger, R. F., Watson, D., Achenbach, T. M., Althoff, R. R., Bagby, R. M., Brown, T. A., Carpenter, W. T., Caspi, A., Clark, L. A., Eaton, N. R., Forbes, M. K., Forbush, K. T., Goldberg, D., Hasin, D., Hyman, S. E., Ivanova, M. Y., Lynam, D. R., Markon, K., … Zimmerman, M. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454-477.
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in systems neuroscience: The dangers of double dipping. Nature Neuroscience, 12(5), 535-540.
Lahey, B. B., Krueger, R. F., Rathouz, P. J., Waldman, I. D., & Zald, D. H. (2017). A hierarchical causal taxonomy of psychopathology across the life span. Psychological Bulletin, 143(2), 142-186.
Luber, B. M., Davis, S., Bernhardt, E., Neacsiu, A., Kwapil, L., Lisanby, S. H., & Strauman, T. J. (2017). Reprint of ‘‘Using neuroimaging to individualize TMS treatment for depression: Toward a new paradigm for imaging-guided intervention’. NeuroImage, 151, 65-71.
Masouleh, S. K., Eickhoff, S. B., Hoffstaedter, F., Genon, S., & Alzheimer's Disease Neuroimaging, I. (2019). Empirical examination of the replicability of associations between brain structure and psychological variables. eLife, 8, e43464.
Miller, K. L., Alfaro-Almagro, F., Bangerter, N. K., Thomas, D. L., Yacoub, E., Xu, J., Bartsch, A. J., Jbabdi, S., Sotiropoulos, S. N., Andersson, J. L., Griffanti, L., Douaud, G., Okell, T. W., Weale, P., Dragonu, I., Garratt, S., Hudson, S., Collins, R., Jenkinson, M., … Smith, S. M. (2016). Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nature Neuroscience, 19(11), 1523-1536.
Munafò, M. R., Cremers, H. R., Wager, T. D., & Yarkoni, T. (2019). Power and design considerations in imaging research, Casting Light on the Dark Side of Brain Imaging (pp. 73-78). Elsevier
Paulus, M. P., Squeglia, L. M., Bagot, K., Jacobus, J., Kuplicki, R., Breslin, F. J., Bodurka, J., Morris, A. S., Thompson, W. K., Bartsch, H., & Tapert, S. F. (2019). Screen media activity and brain structure in youth: Evidence for diverse structural correlation networks from the ABCD study. NeuroImage, 185, 140-153.
Paulus, M. P., & Thompson, W. K. (2019a). The challenges and opportunities of small effects: The new normal in academic psychiatry. JAMA psychiatry, 76(4), 353-354.
Paulus, M. P., & Thompson, W. K. (2019b). Computational approaches and machine learning for individual-level treatment predictions. Psychopharmacology, 238, 1-9.
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J. B., Vul, E., & Yarkoni, T. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, 18(2), 115-126.
Sander, D., Grandjean, D., Pourtois, G., Schwartz, S., Seghier, M. L., Scherer, K. R., & Vuilleumier, P. (2005). Emotion and attention interactions in social cognition: brain regions involved in processing anger prosody. Neuroimage, 28(4), 848-858.
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & PRISMA-P, G. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ, 349, g7647.
Shrout, P. E., & Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69, 487-510.
Sui, J., Jiang, R., Bustillo, J., & Calhoun, V. (2020). Neuroimaging-based individualized prediction of cognition and behavior for mental disorders and health: methods and promises. Biological Psychiatry, 88, 818-828.
Szucs, D., & Ioannidis, J. P. (2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology, 15(3), 2000797.
Walter, M., Alizadeh, S., Jamalabadi, H., Lueken, U., Dannlowski, U., Walter, H., Olbrich, S., Colic, L., Kambeitz, J., Koutsouleris, N., Hahn, T., & Dwyer, D. B. (2019). Translational machine learning for psychiatric neuroimaging. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 91, 113-121.
Walum, H., Waldman, I. D., & Young, L. J. (2016). Statistical and methodological considerations for the interpretation of intranasal oxytocin studies. Biological Psychiatry, 79(3), 251-257.
Woo, C.-W., Chang, L. J., Lindquist, M. A., & Wager, T. D. (2017). Building better biomarkers: Brain models in translational neuroimaging. Nature Neuroscience, 20(3), 365-377.
Yang, Z., Gu, S., Honnorat, N., Linn, K. A., Shinohara, R. T., Aselcioglu, I., Bruce, S., Oathes, D. J., Davatzikos, C., Satterthwaite, T. D., Bassett, D. S., & Sheline, Y. I. (2018). Network changes associated with transdiagnostic depressive symptom improvement following cognitive behavioral therapy in MDD and PTSD. Molecular Psychiatry, 23(12), 2314-2323.
Yarkoni, T. (2009). Big correlations in little studies: Inflated fMRI correlations reflect low statistical power-Commentary on Vul et al.(2009). Perspectives on Psychological Science, 4(3), 294-298.
Yarkoni, T., & Braver, T. S. (2010). Cognitive neuroscience approaches to individual differences in working memory and executive control: Conceptual and methodological issues, In Handbook of individual differences in cognition (pp. 87-107). Springer
Zuo, X.-N., Xu, T., & Milham, M. P. (2019). Harnessing reliability for neuroscience research. Nature Human Behaviour, 3(8), 768-771.