Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
28 Jun 2024
28 Jun 2024
Historique:
received:
01
09
2023
accepted:
13
06
2024
medline:
29
6
2024
pubmed:
29
6
2024
entrez:
28
6
2024
Statut:
epublish
Résumé
While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.
Identifiants
pubmed: 38942766
doi: 10.1038/s41467-024-49669-0
pii: 10.1038/s41467-024-49669-0
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
5477Subventions
Organisme : U.S. Department of Health & Human Services | NIH | National Eye Institute (NEI)
ID : R21EY031867
Informations de copyright
© 2024. The Author(s).
Références
Strasburger, H. Seven myths on crowding and peripheral vision. i-Percept. 11, 2041669520913052 (2020).
Epstein, R. A. & Baker, C. I. Scene perception in the human brain. Annu. Rev. Vis. Sci. 5, 373–397 (2019).
pubmed: 31226012
pmcid: 6989029
doi: 10.1146/annurev-vision-091718-014809
Dilks, D. D., Kamps, F. S. & Persichetti, A. S. Three cortical scene systems and their development. Trends. Cogn. Sci. 26, 117–127 (2022).
Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392, 598–601 (1998).
pubmed: 9560155
doi: 10.1038/33402
O’Craven, K. M. & Kanwisher, N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J. Cogn. Neurosci. 12, 1013–1023 (2000).
pubmed: 11177421
doi: 10.1162/08989290051137549
Silson, E. H., Steel, A. D. & Baker, C. I. Scene-selectivity and retinotopy in medial parietal cortex. Front. Hum. Neurosci. 10, 412 (2016).
pubmed: 27588001
pmcid: 4988988
doi: 10.3389/fnhum.2016.00412
Grill-Spector, K. The neural basis of object perception. Curr. Opin. Neurobiol. 13, 159–166 (2003).
pubmed: 12744968
doi: 10.1016/S0959-4388(03)00040-0
Dilks, D. D., Julian, J. B., Paunov, A. M. & Kanwisher, N. The occipital place area is causally and selectively involved in scene perception. J. Neurosci. 33, 1331–1336 (2013).
pubmed: 23345209
pmcid: 3711611
doi: 10.1523/JNEUROSCI.4081-12.2013
Walther, D. B., Caddigan, E., Fei-Fei, L. & Beck, D. M. Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29, 10573–10581 (2009).
pubmed: 19710310
pmcid: 2774133
doi: 10.1523/JNEUROSCI.0559-09.2009
Epstein, R. A. & Morgan, L. K. Neural responses to visual scenes reveals inconsistencies between fmri adaptation and multivoxel pattern analysis. Neuropsychologia 50, 530–543 (2012).
pubmed: 22001314
doi: 10.1016/j.neuropsychologia.2011.09.042
Kornblith, S., Cheng, X., Ohayon, S. & Tsao, D. Y. A network for scene processing in the macaque temporal lobe. Neuron 79, 766–781 (2013).
pubmed: 23891401
pmcid: 8127731
doi: 10.1016/j.neuron.2013.06.015
Park, J. & Park, S. Conjoint representation of texture ensemble and location in the parahippocampal place area. J. Neurophysiol. 117, 1595–1607 (2017).
pubmed: 28123006
pmcid: 5376611
doi: 10.1152/jn.00338.2016
Oliva, A. & Torralba, A. Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001).
doi: 10.1023/A:1011139631724
Bonner, M. F. & Epstein, R. A. Coding of navigational affordances in the human visual system. Proc. Natl. Acad. Sci. 114, 4793–4798 (2017).
pubmed: 28416669
pmcid: 5422815
doi: 10.1073/pnas.1618228114
Marchette, S. A., Vass, L. K., Ryan, J. & Epstein, R. A. Anchoring the neural compass: coding of local spatial reference frames in human medial parietal lobe. Nat. Neurosci. 17, 1598–1606 (2014).
pubmed: 25282616
pmcid: 4309016
doi: 10.1038/nn.3834
Persichetti, A. S. & Dilks, D. D. Dissociable neural systems for recognizing places and navigating through them. J. Neurosci. 38, 10295–10304 (2018).
pubmed: 30348675
pmcid: 6596208
doi: 10.1523/JNEUROSCI.1200-18.2018
Kamps, F. S., Lall, V. & Dilks, D. D. The occipital place area represents first-person perspective motion information through scenes. Cortex 83, 17–26 (2016).
pubmed: 27474914
pmcid: 5042880
doi: 10.1016/j.cortex.2016.06.022
Robertson, C. E., Hermann, K. L., Mynick, A., Kravitz, D. J. & Kanwisher, N. Neural representations integrate the current field of view with the remembered 360 panorama in scene-selective cortex. Curr. Biol. 26, 2463–2468 (2016).
pubmed: 27618266
doi: 10.1016/j.cub.2016.07.002
Ferrara, K. & Park, S. Neural representation of scene boundaries. Neuropsychologia 89, 180–190 (2016).
pubmed: 27181883
pmcid: 4996703
doi: 10.1016/j.neuropsychologia.2016.05.012
Park, J. & Park, S. Coding of navigational distance and functional constraint of boundaries in the human scene-selective cortex. J. Neurosci. 40, 3621–3630 (2020).
pubmed: 32209608
pmcid: 7189766
doi: 10.1523/JNEUROSCI.1991-19.2020
Ellis, C. et al. Re-imagining fmri for awake behaving infants. Nat. Commun. 11, 4523 (2020).
pubmed: 32908125
pmcid: 7481790
doi: 10.1038/s41467-020-18286-y
Pitzalis, S. et al. Wide-field retinotopy defines human cortical visual area v6. J. Neurosci. 26, 7962–7973 (2006).
pubmed: 16870741
pmcid: 6674231
doi: 10.1523/JNEUROSCI.0178-06.2006
Stenbacka, L. & Vanni, S. fmri of peripheral visual field representation. Clin. Neurophysiol. 118, 1303–1314 (2007).
pubmed: 17449320
doi: 10.1016/j.clinph.2007.01.023
Yan, T., Jin, F., He, J. & Wu, J. Development of a wide-view visual presentation system for visual retinotopic mapping during functional mri. J. Magn. Reson. Imaging 33, 441–447 (2011).
pubmed: 21274987
doi: 10.1002/jmri.22404
Wu, J., Yan, T., Zhang, Z., Jin, F. & Guo, Q. Retinotopic mapping of the peripheral visual field to human visual cortex by functional magnetic resonance imaging. Hum. Brain Mapp. 33, 1727–1740 (2012).
pubmed: 22438122
pmcid: 6870278
doi: 10.1002/hbm.21324
Arnoldussen, D. M., Goossens, J. & van den Berg, A. V. Adjacent visual representations of self-motion in different reference frames. Proc. Natl Acad. Sci. 108, 11668–11673 (2011).
pubmed: 21709244
pmcid: 3136271
doi: 10.1073/pnas.1102984108
Wu, J. et al. Development of a method to present wide-view visual stimuli in mri for peripheral visual studies. J. Neurosci. Methods 214, 126–136 (2013).
pubmed: 23376498
doi: 10.1016/j.jneumeth.2013.01.021
Greco, V. et al. A low-cost and versatile system for projecting wide-field visual stimuli within fmri scanners. Behav. Res. Methods 48, 614–620 (2016).
pubmed: 26092392
pmcid: 5040504
doi: 10.3758/s13428-015-0605-0
Mikellidou, K. et al. Area prostriata in the human brain. Curr. Biol. 27, 3056–3060 (2017).
pubmed: 28966090
doi: 10.1016/j.cub.2017.08.065
Levy, I., Hasson, U., Avidan, G., Hendler, T. & Malach, R. Center–periphery organization of human object areas. Nat. Neurosci. 4, 533–539 (2001).
pubmed: 11319563
doi: 10.1038/87490
Silson, E. H., Chan, A. W.-Y., Reynolds, R. C., Kravitz, D. J. & Baker, C. I. A retinotopic basis for the division of high-level scene processing between lateral and ventral human occipitotemporal cortex. J. Neurosci. 35, 11921–11935 (2015).
pubmed: 26311774
pmcid: 4549403
doi: 10.1523/JNEUROSCI.0137-15.2015
Baldassano, C., Esteva, A., Fei-Fei, L. & Beck, D. M. Two distinct scene-processing networks connecting vision and memory. Eneuro 3, 5 (2016).
doi: 10.1523/ENEURO.0178-16.2016
McCotter, M., Gosselin, F., Sowden, P. & Schyns, P. The use of visual information in natural scenes. Vis. Cogn. 12, 938–953 (2005).
doi: 10.1080/13506280444000599
Park, S., Brady, T. F., Greene, M. R. & Oliva, A. Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31, 1333–1340 (2011).
pubmed: 21273418
pmcid: 6623596
doi: 10.1523/JNEUROSCI.3885-10.2011
Loschky, L. et al. The contributions of central and peripheral vision to scene gist recognition with a 180° visual field. J. Vis. 15, 570–570 (2015).
doi: 10.1167/15.12.570
Hasson, U., Levy, I., Behrmann, M., Hendler, T. & Malach, R. Eccentricity bias as an organizing principle for human high-order object areas. Neuron 34, 479–490 (2002).
pubmed: 11988177
doi: 10.1016/S0896-6273(02)00662-1
Harel, A., Kravitz, D. J. & Baker, C. I. Deconstructing visual scenes in cortex: gradients of object and spatial layout information. Cereb. Cortex 23, 947–957 (2013).
pubmed: 22473894
doi: 10.1093/cercor/bhs091
Choo, H. & Walther, D. B. Contour junctions underlie neural representations of scene categories in high-level human visual cortex. Neuroimage 135, 32–44 (2016).
pubmed: 27118087
doi: 10.1016/j.neuroimage.2016.04.021
Cant, J. S. & Xu, Y. Object ensemble processing in human anterior-medial ventral visual cortex. J. Neurosci. 32, 7685–7700 (2012).
pubmed: 22649247
pmcid: 6703596
doi: 10.1523/JNEUROSCI.3325-11.2012
Stansbury, D. E., Naselaris, T. & Gallant, J. L. Natural scene statistics account for the representation of scene categories in human visual cortex. Neuron 79, 1025–1034 (2013).
pubmed: 23932491
pmcid: 5464350
doi: 10.1016/j.neuron.2013.06.034
Van Essen, D. & Drury, H. Structural and functional analyses of human cerebral cortex using a surface-based atlas. J. Neurosci. 17, 7079–7102 (1997).
pubmed: 9278543
pmcid: 6573261
doi: 10.1523/JNEUROSCI.17-18-07079.1997
Hasson, U., Harel, M., Levy, I. & Malach, R. Large-scale mirror-symmetry organization of human occipito-temporal object areas. Neuron 37, 1027–1041 (2003).
pubmed: 12670430
doi: 10.1016/S0896-6273(03)00144-2
Steel, A., Billings, M. M., Silson, E. H. & Robertson, C. E. A network linking scene perception and spatial memory systems in posterior cerebral cortex. Nat. Commun. 12, 2632 (2021).
pubmed: 33976141
pmcid: 8113503
doi: 10.1038/s41467-021-22848-z
Pitzalis, S. et al. Human v6: the medial motion area. Cereb. Cortex 20, 411–424 (2010).
pubmed: 19502476
doi: 10.1093/cercor/bhp112
Cardin, V. & Smith, A. T. Sensitivity of human visual and vestibular cortical regions to egomotion-compatible visual stimulation. Cereb. Cortex 20, 1964–1973 (2010).
pubmed: 20034998
doi: 10.1093/cercor/bhp268
Malach, R., Levy, I. & Hasson, U. The topography of high-order human object areas. Trends Cogn. Sci. 6, 176–184 (2002).
pubmed: 11912041
doi: 10.1016/S1364-6613(02)01870-3
Baldassano, C., Fei-Fei, L. & Beck, D. M. Pinpointing the peripheral bias in neural scene-processing networks during natural viewing. J. Vis. 16, 9–9 (2016).
pubmed: 27187606
doi: 10.1167/16.2.9
Kamps, F. S., Hendrix, C. L., Brennan, P. A. & Dilks, D. D. Connectivity at the origins of domain specificity in the cortical face and place networks. Proc. Natl Acad. Sci. 117, 6163–6169 (2020).
pubmed: 32123077
pmcid: 7084100
doi: 10.1073/pnas.1911359117
Arcaro, M. J. & Livingstone, M. S. A hierarchical, retinotopic proto-organization of the primate visual system at birth. Elife 6, e26196 (2017).
pubmed: 28671063
pmcid: 5495573
doi: 10.7554/eLife.26196
Finzi, D. Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections. Nat. Commun. 12, 2278 (2021).
pubmed: 33859195
pmcid: 8050273
doi: 10.1038/s41467-021-22524-2
Kay, K. N., Weiner, K. S. & Grill-Spector, K. Attention reduces spatial uncertainty in human ventral temporal cortex. Curr. Biol. 25, 595–600 (2015).
pubmed: 25702580
pmcid: 4348205
doi: 10.1016/j.cub.2014.12.050
Rima, S. & Schmid, M. C. V1-bypassing thalamo-cortical visual circuits in blindsight and developmental dyslexia. Curr. Opin. Physiol. 16, 14–20 (2020).
doi: 10.1016/j.cophys.2020.05.001
Beltramo, R. & Scanziani, M. A collicular visual cortex: neocortical space for an ancient midbrain visual structure. Science 363, 64–69 (2019).
pubmed: 30606842
doi: 10.1126/science.aau7052
Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyllou, C. & Kanwisher, N. Differential selectivity for dynamic versus static information in face-selective cortical regions. Neuroimage 56, 2356–2363 (2011).
pubmed: 21473921
doi: 10.1016/j.neuroimage.2011.03.067
Pitcher, D. & Ungerleider, L. G. Evidence for a third visual pathway specialized for social perception. Trends Cogn. Sci. 25, 100–110 (2021).
pubmed: 33334693
doi: 10.1016/j.tics.2020.11.006
Frey, M., Nau, M. & Doeller, C. F. Magnetic resonance-based eye tracking using deep neural networks. Nat. Neurosci. 24, 1772–1779 (2021).
pubmed: 34750593
pmcid: 10097595
doi: 10.1038/s41593-021-00947-w
Tootell, R. B. et al. Functional analysis of human mt and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15, 3215–3230 (1995).
pubmed: 7722658
pmcid: 6577785
doi: 10.1523/JNEUROSCI.15-04-03215.1995
Cheng, K., Fujita, H., Kanno, I., Miura, S. & Tanaka, K. Human cortical regions activated by wide-field visual motion: an h2 (15) o pet study. J. Neurophysiol. 74, 413–427 (1995).
pubmed: 7472342
doi: 10.1152/jn.1995.74.1.413
Stenbacka, L. & Vanni, S. Central luminance flicker can activate peripheral retinotopic representation. Neuroimage 34, 342–348 (2007).
pubmed: 17049885
doi: 10.1016/j.neuroimage.2006.08.029
Nasr, S. et al. In vivo functional localization of the temporal monocular crescent representation in human primary visual cortex. Neuroimage 209, 116516 (2020).
pubmed: 31904490
doi: 10.1016/j.neuroimage.2020.116516
Park, J., Josephs, E. & Konkle, T. Ramp-shaped neural tuning supports graded population-level representation of the object-to-scene continuum. Sci Rep. 12, 18081 (2022).
Josephs, E. L. & Konkle, T. Large-scale dissociations between views of objects, scenes, and reachable-scale environments in visual cortex. Proc. Natl Acad. Sci. 117, 29354–29362 (2020).
pubmed: 33229533
pmcid: 7703543
doi: 10.1073/pnas.1912333117
Park, J., Josephs, E. & Konkle, T. Systematic transition from boundary extension to contraction along an object-to-scene continuum. J. Vis. 24, 9–9 (2024).
pubmed: 38252521
pmcid: 10810016
doi: 10.1167/jov.24.1.9
Bainbridge, W. A. & Baker, C. I. Boundaries extend and contract in scene memory depending on image properties. Curr. Biol. 30, 537–543 (2020).
pubmed: 31983637
pmcid: 7187786
doi: 10.1016/j.cub.2019.12.004
Ragni, F., Tucciarelli, R., Andersson, P. & Lingnau, A. Decoding stimulus identity in occipital, parietal and inferotemporal cortices during visual mental imagery. Cortex 127, 371–387 (2020).
pubmed: 32289581
doi: 10.1016/j.cortex.2020.02.020
Dal Ben, R. Shine_color: controlling low-level properties of colorful images. MethodsX. 11, 102377 (2021).
Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).
pubmed: 9176952
doi: 10.1163/156856897X00357
Pelli, D. G. & Vision, S. The videotoolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437–442 (1997).
pubmed: 9176953
doi: 10.1163/156856897X00366
Van der Kouwe, A. J., Benner, T., Salat, D. H. & Fischl, B. Brain morphometry with multiecho mprage. Neuroimage 40, 559–569 (2008).
pubmed: 18242102
doi: 10.1016/j.neuroimage.2007.12.025
Moeller, S. et al. Multiband multislice ge-epi at 7 tesla, with 16-fold acceleration using partial parallel imaging with application to high spatial and temporal whole-brain fmri. Magn. Reson. Med. 63, 1144–1153 (2010).
pubmed: 20432285
pmcid: 2906244
doi: 10.1002/mrm.22361
Feinberg, D. A. et al. Multiplexed echo planar imaging for sub-second whole brain fmri and fast diffusion imaging. PloS One 5, e15710 (2010).
pubmed: 21187930
pmcid: 3004955
doi: 10.1371/journal.pone.0015710
Setsompop, K. & Gagoski, B. A. et al. Blipped-controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g-factor penalty. Magn. Reson. Med. 67, 1210–1224 (2012).
pubmed: 21858868
doi: 10.1002/mrm.23097
Xu, J. et al. Evaluation of slice accelerations using multiband echo planar imaging at 3 t. Neuroimage 83, 991–1001 (2013).
pubmed: 23899722
doi: 10.1016/j.neuroimage.2013.07.055