A large-scale examination of inductive biases shaping high-level visual representation in brains and machines.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
30 Oct 2024
30 Oct 2024
Historique:
received:
07
09
2023
accepted:
01
10
2024
medline:
31
10
2024
pubmed:
31
10
2024
entrez:
31
10
2024
Statut:
epublish
Résumé
The rapid release of high-performing computer vision models offers new potential to study the impact of different inductive biases on the emergent brain alignment of learned representations. Here, we perform controlled comparisons among a curated set of 224 diverse models to test the impact of specific model properties on visual brain predictivity - a process requiring over 1.8 billion regressions and 50.3 thousand representational similarity analyses. We find that models with qualitatively different architectures (e.g. CNNs versus Transformers) and task objectives (e.g. purely visual contrastive learning versus vision- language alignment) achieve near equivalent brain predictivity, when other factors are held constant. Instead, variation across visual training diets yields the largest, most consistent effect on brain predictivity. Many models achieve similarly high brain predictivity, despite clear variation in their underlying representations - suggesting that standard methods used to link models to brains may be too flexible. Broadly, these findings challenge common assumptions about the factors underlying emergent brain alignment, and outline how we can leverage controlled model comparison to probe the common computational principles underlying biological and artificial visual systems.
Identifiants
pubmed: 39477923
doi: 10.1038/s41467-024-53147-y
pii: 10.1038/s41467-024-53147-y
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
9383Subventions
Organisme : National Science Foundation (NSF)
ID : NSF-CAREER BCS-1942438
Informations de copyright
© 2024. The Author(s).
Références
DiCarlo, J. J., Zoccolan, D. & Rust, N. C. How Does the Brain Solve Visual Object Recognition? Neuron 73, 415–434 (2012).
doi: 10.1016/j.neuron.2012.01.010
pubmed: 22325196
Hubel, D. H., & Wiesel, T. N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195, 215–243 (1968).
Olshausen, B. A., Field, D. J. et al. Sparse coding of natural images produces localized, oriented, bandpass receptive fields. Submitted to Nature. Available electronically as ftp://redwood. psych. cornell. edu/pub/papers/sparse-coding. ps, 1995. Citeseer.
Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 1097–1105 (2012).
Yamins, D. L. et al. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl Acad. Sci. 111, 8619–8624, (2014).
Kriegeskorte, N. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu. Rev. Vis. Sci. 1, 417–446, (2015).
Eickenberg, M., Gramfort, A., Varoquaux, G. & Thirion, B. Seeing it all: Convolutional network layers map the function of the human visual system. NeuroImage, 152, 184–194, (2017).
Kay, K. N. Principles for models of neural information processing. NeuroImage 180, 101–109 (2018).
doi: 10.1016/j.neuroimage.2017.08.016
pubmed: 28793238
Kietzmann, T. C., McClure, P. & Kriegeskorte, N. Deep neural networks in computational neuroscience. https://doi.org/10.1093/acrefore/9780190264086.013.46 (2019).
Thomas, S. Deep learning: the good, the bad, and the ugly. Annu. Rev. Vis. Sci. 5, 399–426. https://doi.org/10.1146/annurev-vision-091718-014951 (2019).
Richards, B. A. et al. A deep learning framework for neuroscience. Nat. Neurosci. 22, 1761–1770. https://doi.org/10.1038/s41593-019-0520-2 (2019).
Cao, R. & Yamins, D. Explanatory models in neuroscience, Part 1: Taking mechanistic abstraction seriously. Cognitive Systems Research, 101244 (2024).
Doerig, A. et al. The neuroconnectionist research programme. Nat. Rev. Neurosci. 24, 431–450 (2023).
Kanwisher, N., Khosla, M. & Dobs, K. Using artificial neural networks to ask ‘why’ questions of minds and brains. Trends Neurosci. 46, 240–254 (2023).
Güçlü, U. & van Gerven, M. A. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015).
doi: 10.1523/JNEUROSCI.5023-14.2015
pubmed: 26157000
Yamins, D. L. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356 (2016).
Bashivan, P., Kar, K. & DiCarlo, J. J. Neural population control via deep image synthesis. Science, 364, https://doi.org/10.1126/science.aav9436 (2019).
Xiao, W. & Kreiman, G. XDream: Finding preferred stimuli for visual neurons using generative networks and gradient-free optimization. PLoS Comput. Biol. 16, e1007973 (2020).
Khaligh-Razavi, S. M. & Kriegeskorte, N. Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol. 10, e1003915 (2014).
Güçlü, U. & van Gerven, M. A. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015).
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A. & Oliva, A. Comparison of deep neural networks to spatiotemporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep. 6, https://doi.org/10.1038/srep27755 (2016).
Long, B., Yu, C. P. & Konkle, T. Mid-level visual features underlie the high-level categorical organization of the ventral stream. Proc. Natl Acad. Sci. 115, E9015–E9024 (2018).
Wen, H., Shi, J., Chen, W. & Liu, Z. Deep residual network predicts cortical representation and organization of visual features for rapid categorization. Sci. Rep. 8, 1–17 (2018).
St-Yves, G. & Naselaris, T. The feature-weighted receptive field: an interpretable encoding model for complex feature spaces. NeuroImage, 180, 188–202 (2018).
Storrs, K. R., Kietzmann, T. C., Walther, A., Mehrer, J. & Kriegeskorte, N. Diverse Deep Neural Networks All Predict Human Inferior Temporal Cortex Well, After Training and Fitting. J. Cogn. Neurosci. 33, 2044–2064 (2021).
Martin S. et al. Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like? bioRxiv preprint, https://doi.org/10.1101/407007 (2018).
Martin S. et al. Integrative benchmarking to advance neurally mechanistic models of human intelligence. Neuron 108, 413–423 (2020).
Radoslaw, M. C. et al. The algonauts project: A platform for communication between the sciences of biological and artificial intelligence. arXiv Prepr. arXiv 1905, 05675 (2019).
K. F. Willeke et al. The Sensorium competition on predicting large-scale mouse primary visual cortex activity. arXiv preprint arXiv:2206.08666, 2022.
Allen, E. J. et al. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neurosci. 25, 116–126 (2022).
Kriegeskorte, N., Mur, M. & Bandettini, P. Representational similarity analysis- connecting the branches of systems neuroscience. Fronti. Syst. Neurosci. 2, 4 (2008).
Khaligh-Razavi, S. M., Henriksson, L., Kay, K. & Kriegeskorte, N. Fixed versus mixed rsa: Explaining visual representations by fixed and mixed feature sets from shallow and deep computational models. J. Math. Psychol. 76, 184–197 (2017).
doi: 10.1016/j.jmp.2016.10.007
pubmed: 28298702
Kaniuth, P. & Hebart, M. N. Feature-reweighted representational similarity analysis: A method for improving the fit between computational models, brains, and behavior. NeuroImage 257, 119294 (2022).
doi: 10.1016/j.neuroimage.2022.119294
pubmed: 35580810
Konkle, T. & Alvarez, G. A. A self-supervised domain-general learning framework for human ventral stream representation. Nat. Commun. 13, 1–12 (2022).
Yann, L., Yoshua, B. & Geoffrey, H. Deep Learning. Nature 521, 436–444 (2015).
Zhuang, L. et al. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022.
McGreivy, N. & Hakim A. Convolutional layers are not translation equivariant. arXiv preprint arXiv:2206.04979, 2022.
Maithra, R., Thomas, U., Simon, K., Chiyuan, Z. & Alexey, D. Do vision transformers see like convolutional neural networks? Adv. Neural Inf. Process. Syst. 34, 12116–12128 (2021).
Muhammad Muzammal, N. et al. Intriguing properties of vision transformers. Adv. Neural Inf. Process. Syst. 34, 23296–23308 (2021).
Hong-Yu, Z., Chixiang, L., Sibei, Y. & Yizhou, Y. ConvNets vs. Transformers: Whose visual representations are more transferable? In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2230–2238 (2021).
Zamir, A. R. et al. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3712–3722 (2018).
Goyal, P. et al VISSL, https://github.com/facebookresearch/vissl 2021.
Mu, N., Kirillov, A., Wagner, D. & Xie, S. Slip: Self-supervision meets language-image pre-training. European conference on computer vision 529–544 (Springer Nature Switzerland, Cham, 2022).
Sax, A. et al. Learning to Navigate Using Mid-Level Visual Priors. arXiv:1912.11121 [cs], URL http://arxiv.org/abs/1912.11121 . arXiv: 1912.11121 (2019).
Geirhos, R. et al. On the surprising similarities between supervised and self-supervised models. In: Proceedings of the Shared Visual Representations in Humans & Machines Workshop (NeurIPS) (2020).
Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning 1597–1607 (PMLR, 2020).
Conwell, C., Mayo, D., Barbu, A., Buice, M., Alvarez, G. & Katz, B. Neural regression, representational similarity, model zoology & neural taskonomy at scale in rodent visual cortex. Adv. Neural Inf. Process. Syst. 34, 5590–5607 (2021).
Nayebi, A. et al. Unsupervised Models of Mouse Visual Cortex. bioRxiv. (Cold Spring Harbor Laboratory, 2021).
Zhuang, C. et al. Unsupervised neural network models of the ventral visual stream. Proc. Natl Acad. Sci. 118, e2014196118 (2021).
Radford, A. et al. Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, PMLR, 8748–8763 (2021).
Wortsman, M. et al. Robust fine- tuning of zero-shot models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7959–7971 (2022).
Wang, A. Y., Kay, K., Naselaris, T., Tarr, M. J., & Wehbe, L. Incorporating natural language into vision models improves prediction and understanding of higher visual cortex. BioRxiv, 2022–09 (Cold Spring Harbor Laboratory, 2022).
Kaplan, J. et al. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
Goyal, P. et al. Vision models are more robust and fair when pretrained on uncurated images without supervision. arXiv preprint arXiv, (2022). 2202.08360.
Puigcerver, J. et al. Scalable transfer learning with expert models. arXiv preprint arXiv, (2020). 2009.13239.
Ridnik, T., Ben-Baruch, E., Noy, A. & Zelnik-Manor, L. Imagenet-21k pretraining for the masses. arXiv preprint arXiv, (2021). 2104.10972.
Wightman, R. Pytorch image models. https://github.com/rwightman/ pytorch-image-models, (2019).
Gallicchio, C. & Scardapane, S. Deep randomized neural networks. In Recent Trends in Learning From Data: Tutorials from the INNS Big Data and Deep Learning Conference (INNSB- DDL2019), 43–68. (Springer, https://doi.org/10.1007/978-3-030-43883-8_3 2020).
Cadena, S. A. et al. How well do deep neural networks trained on object recognition characterize the mouse visual system? In NeurIPS Neuro AI Workshop, (2019).
Hermann, K. & Lampinen, A. What shapes feature representations? exploring datasets, architectures, and training. Adv. Neural Inf. Process. Syst. 33, 9995–10006 (2020).
Konkle, T. B. et al. Face detection in untrained deep neural networks. Nat. Commun. 12, 7328 (2021).
doi: 10.1038/s41467-021-27606-9
Prince, J. S., Alvarez, G. A. & Konkle, T. Contrastive learning explains the emergence and function of visual category-selective regions. Science Advances, 10, (2024).
Nonaka, S., Majima, K., Aoki, S. C. & Kamitani, Y. Brain hierarchy score: Which deep neural networks are hierarchically brain-like? IScience 24, 103013 (2021).
doi: 10.1016/j.isci.2021.103013
pubmed: 34522856
Ratan Murty, N. A., Bashivan, P., Abate, A., DiCarlo, J. J. & Kanwisher, N. Compu- tational models of category-selective brain regions enable high-throughput tests of selectivity. Nat. Commun. 12, 5540 (2021).
doi: 10.1038/s41467-021-25409-6
pubmed: 34545079
Marco Del Giudice. Effective dimensionality: A tutorial. Multivariate behavioral research, 56, 527–542. https://doi.org/10.1080/00273171.2020.1743631 . (Taylor & Francis, 2021).
Elmoznino, E. & Bonner, M. F. High-performing neural network models of visual cortex benefit from high latent dimensionality. bioRxiv, 2022–07, (Cold Spring Harbor Laboratory, 2022).
Garrido, Q., Balestriero, R., Najman, L. & Lecun, Y. Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank. In International conference on machine learning 10929–10974 (PMLR, 2023).
Yerxa, T., Kuang, Y., Simoncelli, E. & Chung, S. Learning efficient coding of natural images with maximum manifold capacity representations. Advances in Neural Information Processing Systems, 36, 24103–24128 (2023).
Linsley, D. et al. Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex. Advances in Neural Information Processing Systems, 36 (2024).
Geirhos, R. et al. Partial success in closing the gap between human and machine vision. Adv. Neural Inf. Process. Syst. 34, 23885–23899 (2021).
Muttenthaler, L., Dippel, J., Linhardt, L., Vandermeulen, R. A. & Kornblith, S. Human alignment of neural network representations. In: Proceedings of the 11th International Conference on Learning Representations (ICLR), (2023).
Dehghani, M. et al. Scaling vision transformers to 22 billion parameters. arXiv preprint arXiv, (2023). 2302.05442.
Wood, J. N., Lee, D., Wood, B. & Wood, S. M. W. Reverse engineering the origins of visual intelligence. In CogSci, 2020.
Vong, W. K., Wang, W., Orhan, A. E. & Lake, B. M. Grounded language acquisition through the eyes and ears of a single child. Science 383, 504–511 (2024).
doi: 10.1126/science.adi1374
pubmed: 38300999
Wang, A., Tarr, M. & Wehbe, L. Neural taskonomy: Inferring the similarity of task-derived representations from brain activity. Adv. Neural Inform. Proc. Syst. 32, (2019).
Dwivedi, K., Bonner, M. F., Cichy, R. M. & Roig, G. Unveiling functions of the visual cortex using task-specific deep neural networks. PLoS Comput. Biol. 17, e1009267 (2021).
Cadena, S. A. et al. Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks. bioRxiv 2022–05. (Cold Spring Harbor Laboratory, 2022).
Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. image Process. 13, 600–612 (2004).
doi: 10.1109/TIP.2003.819861
pubmed: 15376593
Dosselmann, R. & Yang, X. D. A comprehensive assessment of the structural similarity index. Signal, Image Video Process. 5, 81–91 (2011).
doi: 10.1007/s11760-009-0144-1
Gatys, L. A., Ecker, A. S., Bethge, M., Hertzmann, A. and Shechtman, E. Controlling perceptual factors in neural style transfer. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3985–3993 (2017).
Jing, Y. et al. Neural style transfer: A review. IEEE Trans. Vis. computer Graph. 26, 3365–3385 (2019).
doi: 10.1109/TVCG.2019.2921336
Abbas, A., Tirumala, K., Simig, D., Ganguli, S. & Morcos, A. S. Semdedup: Data-efficient learning at web-scale through semantic deduplication. In: Proceedings of the ICLR 2023: Multimodal Representation Learning Workshop (2023).
Gadre, S. Y. et al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, vol. 36, (2024).
Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S. & Ari, M. Beyond neural scaling laws: beating power law scaling via data pruning. Adv. Neural Inf. Process. Syst. 35, 19523–19536 (2022).
Wightman, R., Touvron, H. & Jégou, H. Resnet strikes back: An improved training procedure in timm. arxiv 2021. arXiv preprint arXiv:2110.00476, 2021.
Zhang, H., Cisse, M., Dauphin, Y. N. & Lopez-Paz, D. mixup: Beyond empirical risk minimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2018).
Yun, S. et al Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, 6023–6032 (2019).
The Mosaic ML Team. composer. https://github.com/mosaicml/composer/ , (2021).
Guillaume L., Andrew I., Logan E., Sung Min P., Hadi S. and Aleksander M. dry. Ffcv: Accelerating training by removing data bottlenecks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12011–12020 (2023).
Han, Y., Poggio, T. A. & Cheung, B. System Identification of Neural Systems: If We Got It Right, Would We Know? In: Proceedings of the 40th International Conference on Machine Learning, PMLR, vol. 202, 12430–12444 (2023).
Arend, L. et al. Single units in a deep neural network functionally correspond with neurons in the brain: preliminary results. Technical report, Center for Brains, Minds and Machines (CBMM), (2018).
Khosla, M. & Williams, A. H. Soft Matching Distance: A metric on neural representations that captures single-neuron tuning. Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, (PMLR, 2024).
Lee, H. et al. Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network. bioRxiv, 2020–07 (2020).
Nicholas, M. Blauch, Marlene Behrmann, and David C Plaut. A connectivity-constrained computational account of topographic organization in primate high-level visual cortex. Proc. Natl Acad. Sci. 119, e2112566119 (2022).
doi: 10.1073/pnas.2112566119
Margalit, E. et al. A unifying principle for the functional organization of visual cortex. bioRxiv, 2023–05, (2023).
Doshi, F. R. & Konkle, T. Cortical topographic motifs emerge in a self-organized map of object space. Sci. Adv. 9, eade8187 (2023).
doi: 10.1126/sciadv.ade8187
pubmed: 37343093
pmcid: 10284546
Williams, A. H., Kunz, E., Kornblith, S. & Linderman, S. W. Generalized shape metrics on neural representations. Adv. Neural Inf. Process. Syst. 34, 4738–4750 (2021).
pubmed: 38170102
Sucholutsky, I. et al. Getting aligned on representational alignment. arXiv preprint arXiv:2310.13018, 2023.
Geirhos, R. et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. Proceedings of the International Conference on Learning Representations (ICLR) (2019).
Xu, Y. & Vaziri-Pashkam, M. Limits to visual representational correspondence between convolutional neural networks and the human brain. Nat. Commun. 12, 2065 (2021).
doi: 10.1038/s41467-021-22244-7
pubmed: 33824315
Feather, J., Leclerc, G., Mądry, A. & McDermott, J. H. Model metamers reveal divergent invariances between biological and artificial neural networks. Nat. Neurosci. 26, 2017–2034 (2023).
doi: 10.1038/s41593-023-01442-0
pubmed: 37845543
Rust, N. C. & Movshon, J. A. In praise of artifice. Nat. Neurosci. 8, 1647–1650 (2005).
doi: 10.1038/nn1606
pubmed: 16306892
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).
doi: 10.1016/j.neuron.2016.12.041
pubmed: 28182904
Bowers J. S. et al. Deep problems with neural network models of human vision. Behav. Brain Sci. 1–74, (2022).
Golan, T., Raju, P. C. & Kriegeskorte, N. Controversial stimuli: Pitting neural networks against each other as models of human cognition. Proc. Natl Acad. Sci. 117, 29330–29337 (2020).
doi: 10.1073/pnas.1912334117
pubmed: 33229549
Golan, T., Guo, W., Schütt, H. H. & Kriegeskorte, N. Distinguishing representational geometries with controversial stimuli: Bayesian experimental design and its application to face dissimilarity judgments. Proceedings of the SVRHM 2022 Workshop @ NeurIPS (2022).
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N. & Kietzmann, T. C. An ecologically motivated image dataset for deep learning yields better models of human vision. Proc. Natl Acad. Sci. 118, e2011417118 (2021).
doi: 10.1073/pnas.2011417118
pubmed: 33593900
Prasad A., Manor U. & Pereira T. Exploring the role of image domain in self-supervised dnn models of rodent brains. In SVRHM 2022 Workshop@ NeurIPS, 2022.
Cohen, U., Chung, S., Lee, D. D. & Sompolinsky, H. Separability and geometry of object manifolds in deep neural networks. Nat. Commun. 11, 746 (2020).
doi: 10.1038/s41467-020-14578-5
pubmed: 32029727
Sorscher, B., Ganguli, S. & Sompolinsky, H. Neural representational geometry underlies few-shot concept learning. Proc. Natl Acad. Sci. 119, e2200800119 (2022).
doi: 10.1073/pnas.2200800119
pubmed: 36251997
Kuoch, M. et al. Probing biological and artificial neural networks with task-dependent neural manifolds. In Conference on Parsimony and Learning, 395–418. (PMLR, 2024).
Popham, S. F. et al. Visual and linguistic semantic representations are aligned at the border of human visual cortex. Nat. Neurosci. 24, 1628–1636 (2021).
Tang, J., Du, M., Vo, V., Lal, V. & Huth, A. Brain encoding models based on multimodal transformers can transfer across language and vision. Adv. Neural Inf. Process. Syst. 36 (2024).
Wang, A. Y., Kay, K., Naselaris, T., Tarr, M. J. & Wehbe, L. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nat. Mach. Intell. 5, 1415–1426 (2023).
doi: 10.1038/s42256-023-00753-y
Prince, J. S. & Konkle, T. Computational evidence for integrated rather than specialized feature tuning in category-selective regions. J. Vis. 20, 1577–1577 (2020).
doi: 10.1167/jov.20.11.1577
Ren, Y. & Bashivan, P. How well do models of visual cortex generalize to out of distribution samples? bioRxiv, 2023–05 (2023).
Paszke, A. et al. PyTorch: An Imperative Style, High- Performance Deep Learning Library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché- Buc, E. Fox, and R. Garnett, editors, Adv. Neural Inform. Proc. Syst. 32, 8024–8035. Curran Associates, Inc., http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf (2019).
Sax, A. et al. Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies. (2018).
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y. & Girshick, R. Detectron2, https://github.com/facebookresearch/detectron2 2019.
Lin, T. -Y. et al. Microsoft coco: Common objects in context. In European conference on computer vision, 740–755 (Springer, 2014).
Prince, J. S. et al. Improving the accuracy of single-trial fMRI response estimates using GLMsingle. Elife, 11, e77599 (2022).
Tarhan, L. & Konkle T. Reliability-based voxel selection. NeuroImage, 207, 116350 (2020).
Kay, K. et al. Disentangling signal and noise in neural responses through generative modeling. bioRxiv. (2024).
Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Achlioptas, D. Database-friendly random projections. In Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 274–281 (2001).
Kriegeskorte, N. et al. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60, 1126–1141 (2008).
R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, http://www.R-project.org/.ISBN3-900051-07-0 (2013).
Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).
doi: 10.18637/jss.v067.i01
Lüdecke, D., Ben-Shachar, M. S., Patil, I. & Makowski, D. Extracting, computing and exploring the parameters of statistical models using R. J. Open Source Softw. 5, 2445 (2020).
doi: 10.21105/joss.02445
Poldrack, R. A. et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115–126 (2017).
doi: 10.1038/nrn.2016.167
pubmed: 28053326
Naselaris, T., Allen, E. & Kay, K. Extensive sampling for complete models of individual brains. Curr. Opin. Behav. Sci. 40, 45–51 (2021).
doi: 10.1016/j.cobeha.2020.12.008
Vito, M. R. Muggeo. Estimating regression models with unknown break-points. Stat. Med. 22, 3055–3071 (2003).
doi: 10.1002/sim.1545
Vito, M. R. Muggeo. segmented: an r package to fit regression models with broken-line relationships. R. N. 8, 20–25, https://cran.r-project.org/doc/Rnews/ (2008).
Vito, M. R. Muggeo. Interval estimation for the breakpoint in segmented regression: a smoothed score-based approach. Aust. N.Z. J. Stat. 59, 311–322 (2017).
doi: 10.1111/anzs.12200
John, C. G. Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika 53, 325–338 (1966).
doi: 10.1093/biomet/53.3-4.325
Pedersen, T. L. ggforce: Accelerating ’ggplot2’, https://ggforce.data-imaginist.com , https://github.com/thomasp85/ggforce (2022).
Richard, D. M. et al. Confidence intervals from normalized data: A correction to cousineau (2005). Tutorials in Quantitative Methods for Psychology 4, 61–64 (2008).