Multiscale temporal integration organizes hierarchical computation in human auditory cortex.
Journal
Nature human behaviour
ISSN: 2397-3374
Titre abrégé: Nat Hum Behav
Pays: England
ID NLM: 101697750
Informations de publication
Date de publication:
03 2022
03 2022
Historique:
received:
30
10
2020
accepted:
18
11
2021
pubmed:
12
2
2022
medline:
20
4
2022
entrez:
11
2
2022
Statut:
ppublish
Résumé
To derive meaning from sound, the brain must integrate information across many timescales. What computations underlie multiscale integration in human auditory cortex? Evidence suggests that auditory cortex analyses sound using both generic acoustic representations (for example, spectrotemporal modulation tuning) and category-specific computations, but the timescales over which these putatively distinct computations integrate remain unclear. To answer this question, we developed a general method to estimate sensory integration windows-the time window when stimuli alter the neural response-and applied our method to intracranial recordings from neurosurgical patients. We show that human auditory cortex integrates hierarchically across diverse timescales spanning from ~50 to 400 ms. Moreover, we find that neural populations with short and long integration windows exhibit distinct functional properties: short-integration electrodes (less than ~200 ms) show prominent spectrotemporal modulation selectivity, while long-integration electrodes (greater than ~200 ms) show prominent category selectivity. These findings reveal how multiscale integration organizes auditory computation in the human brain.
Identifiants
pubmed: 35145280
doi: 10.1038/s41562-021-01261-y
pii: 10.1038/s41562-021-01261-y
pmc: PMC8957490
mid: NIHMS1758260
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
455-469Subventions
Organisme : Howard Hughes Medical Institute
Pays : United States
Organisme : NIDCD NIH HHS
ID : K99 DC018051
Pays : United States
Organisme : NIBIB NIH HHS
ID : R01 EB028155
Pays : United States
Organisme : NIDCD NIH HHS
ID : R00 DC018051
Pays : United States
Organisme : NIH HHS
ID : S10 OD018211
Pays : United States
Organisme : NINDS NIH HHS
ID : R01 NS084142
Pays : United States
Organisme : NIDCD NIH HHS
ID : R01 DC018805
Pays : United States
Organisme : NIDCD NIH HHS
ID : R01 DC014279
Pays : United States
Commentaires et corrections
Type : CommentIn
Informations de copyright
© 2022. The Author(s), under exclusive licence to Springer Nature Limited.
Références
Brodbeck, C., Hong, L. E. & Simon, J. Z. Rapid transformation from auditory to linguistic representations of continuous speech. Curr. Biol. 28, 3976–3983 (2018).
pubmed: 30503620
pmcid: 6339854
doi: 10.1016/j.cub.2018.10.042
DeWitt, I. & Rauschecker, J. P. Phoneme and word recognition in the auditory ventral stream. Proc. Natl Acad. Sci. USA 109, E505–E514 (2012).
pubmed: 22308358
pmcid: 3286918
doi: 10.1073/pnas.1113427109
Hickok, G. & Poeppel, D. The cortical organization of speech processing. Nat. Rev. Neurosci. 8, 393–402 (2007).
pubmed: 17431404
doi: 10.1038/nrn2113
Santoro, R. et al. Encoding of natural sounds at multiple spectral and temporal resolutions in the human auditory cortex. PLoS Comput. Biol. 10, e1003412 (2014).
pubmed: 24391486
pmcid: 3879146
doi: 10.1371/journal.pcbi.1003412
Hullett, P. W., Hamilton, L. S., Mesgarani, N., Schreiner, C. E. & Chang, E. F. Human superior temporal gyrus organization of spectrotemporal modulation tuning derived from speech stimuli. J. Neurosci. 36, 2014–2026 (2016).
pubmed: 26865624
pmcid: 4748082
doi: 10.1523/JNEUROSCI.1779-15.2016
Schönwiesner, M. & Zatorre, R. J. Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI. Proc. Natl Acad. Sci. USA 106, 14611–14616 (2009).
pubmed: 19667199
pmcid: 2732853
doi: 10.1073/pnas.0907682106
Barton, B., Venezia, J. H., Saberi, K., Hickok, G. & Brewer, A. A. Orthogonal acoustic dimensions define auditory field maps in human cortex. Proc. Natl Acad. Sci. USA 109, 20738–20743 (2012).
pubmed: 23188798
pmcid: 3528571
doi: 10.1073/pnas.1213381109
Leaver, A. M. & Rauschecker, J. P. Cortical representation of natural complex sounds: effects of acoustic features and auditory object category. J. Neurosci. 30, 7604–7612 (2010).
pubmed: 20519535
pmcid: 2930617
doi: 10.1523/JNEUROSCI.0296-10.2010
Norman-Haignere, S. V., Kanwisher, N. G. & McDermott, J. H. Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron 88, 1281–1296 (2015).
pubmed: 26687225
pmcid: 4740977
doi: 10.1016/j.neuron.2015.11.035
Kell, A. J., Yamins, D. L., Shook, E. N., Norman-Haignere, S. V. & McDermott, J. H. A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy. Neuron 98, 630–644 (2018).
Overath, T., McDermott, J. H., Zarate, J. M. & Poeppel, D. The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nat. Neurosci. 18, 903–911 (2015).
pubmed: 25984889
pmcid: 4769593
doi: 10.1038/nn.4021
Davis, M. H. & Johnsrude, I. S. Hierarchical processing in spoken language comprehension. J. Neurosci. 23, 3423–3431 (2003).
pubmed: 12716950
pmcid: 6742313
doi: 10.1523/JNEUROSCI.23-08-03423.2003
Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P. & Pike, B. Voice-selective areas in human auditory cortex. Nature 403, 309–312 (2000).
pubmed: 10659849
doi: 10.1038/35002078
Zuk, N. J., Teoh, E. S. & Lalor, E. C. EEG-based classification of natural sounds reveals specialized responses to speech and music. NeuroImage 210, 116558 (2020).
pubmed: 31962174
doi: 10.1016/j.neuroimage.2020.116558
Di Liberto, G. M., O’Sullivan, J. A. & Lalor, E. C. Low-frequency cortical entrainment to speech reflects phoneme-level processing. Curr. Biol. 25, 2457–2465 (2015).
pubmed: 26412129
doi: 10.1016/j.cub.2015.08.030
Ding, N. et al. Temporal modulations in speech and music. Neurosci. Biobehav. Rev. 81, 181–187 (2017).
Elhilali, M. in Timbre: Acoustics, Perception, and Cognition (eds Siedenburg, K. et al.) 335–359 (Springer, 2019).
Patel, A. D. Music, Language, and the Brain (Oxford Univ. Press, 2007).
Norman-Haignere, S. V. & McDermott, J. H. Neural responses to natural and model-matched stimuli reveal distinct computations in primary and nonprimary auditory cortex. PLoS Biol. 16, e2005127 (2018).
pubmed: 30507943
pmcid: 6292651
doi: 10.1371/journal.pbio.2005127
Theunissen, F. & Miller, J. P. Temporal encoding in nervous systems: a rigorous definition. J. Comput. Neurosci. 2, 149–162 (1995).
pubmed: 8521284
doi: 10.1007/BF00961885
Lerner, Y., Honey, C. J., Silbert, L. J. & Hasson, U. Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J. Neurosci. 31, 2906–2915 (2011).
pubmed: 21414912
pmcid: 3089381
doi: 10.1523/JNEUROSCI.3684-10.2011
Chen, C., Read, H. L. & Escabí, M. A. Precise feature based time scales and frequency decorrelation lead to a sparse auditory code. J. Neurosci. 32, 8454–8468 (2012).
pubmed: 22723685
pmcid: 3483313
doi: 10.1523/JNEUROSCI.6506-11.2012
Meyer, A. F., Williamson, R. S., Linden, J. F. & Sahani, M. Models of neuronal stimulus-response functions: elaboration, estimation, and evaluation. Front. Syst. Neurosci. 10, 109 (2017).
pubmed: 28127278
pmcid: 5226961
doi: 10.3389/fnsys.2016.00109
Khatami, F. & Escabí, M. A. Spiking network optimized for word recognition in noise predicts auditory system hierarchy. PLoS Comput. Biol. 16, e1007558 (2020).
pubmed: 32559204
pmcid: 7329140
doi: 10.1371/journal.pcbi.1007558
Harper, N. S. et al. Network receptive field modeling reveals extensive integration and multi-feature selectivity in auditory cortical neurons. PLoS Comput. Biol. 12, e1005113 (2016).
pubmed: 27835647
pmcid: 5105998
doi: 10.1371/journal.pcbi.1005113
Keshishian, M. et al. Estimating and interpreting nonlinear receptive field of sensory neural responses with deep neural network models. eLife 9, e53445 (2020).
pubmed: 32589140
pmcid: 7347387
doi: 10.7554/eLife.53445
Albouy, P., Benjamin, L., Morillon, B. & Zatorre, R. J. Distinct sensitivity to spectrotemporal modulation supports brain asymmetry for speech and melody. Science 367, 1043–1047 (2020).
pubmed: 32108113
doi: 10.1126/science.aaz3468
Flinker, A., Doyle, W. K., Mehta, A. D., Devinsky, O. & Poeppel, D. Spectrotemporal modulation provides a unifying framework for auditory cortical asymmetries. Nat. Hum. Behav. 3, 393–405 (2019).
Teng, X. & Poeppel, D. Theta and Gamma bands encode acoustic dynamics over wide-ranging timescales. Cereb. Cortex 30, 2600–2614 (2020).
pubmed: 31761952
doi: 10.1093/cercor/bhz263
Obleser, J., Eisner, F. & Kotz, S. A. Bilateral speech comprehension reflects differential sensitivity to spectral and temporal features. J. Neurosci. 28, 8116–8123 (2008).
pubmed: 18685036
pmcid: 6670773
doi: 10.1523/JNEUROSCI.1290-08.2008
Baumann, S. et al. The topography of frequency and time representation in primate auditory cortices. eLife 4, e03256 (2015).
pmcid: 4398946
doi: 10.7554/eLife.03256
Rogalsky, C., Rong, F., Saberi, K. & Hickok, G. Functional anatomy of language and music perception: temporal and structural factors investigated using functional magnetic resonance imaging. J. Neurosci. 31, 3843–3852 (2011).
pubmed: 21389239
pmcid: 3066175
doi: 10.1523/JNEUROSCI.4515-10.2011
Farbood, M. M., Heeger, D. J., Marcus, G., Hasson, U. & Lerner, Y. The neural processing of hierarchical structure in music and speech at different timescales. Front. Neurosci. 9, 157 (2015).
pubmed: 26029037
pmcid: 4429236
doi: 10.3389/fnins.2015.00157
Angeloni, C. & Geffen, M. N. Contextual modulation of sound processing in the auditory cortex. Curr. Opin. Neurobiol. 49, 8–15 (2018).
pubmed: 29125987
doi: 10.1016/j.conb.2017.10.012
Griffiths, T. D. et al. Direct recordings of pitch responses from human auditory cortex. Curr. Biol. 20, 1128–1132 (2010).
pubmed: 20605456
pmcid: 3221038
doi: 10.1016/j.cub.2010.04.044
Mesgarani, N., Cheung, C., Johnson, K. & Chang, E. F. Phonetic feature encoding in human superior temporal gyrus. Science 343, 1006–1010 (2014).
pubmed: 24482117
pmcid: 4350233
doi: 10.1126/science.1245994
Ray, S. & Maunsell, J. H. R. Different origins of gamma rhythm and high-gamma activity in macaque visual cortex. PLoS Biol. 9, e1000610 (2011).
pubmed: 21532743
pmcid: 3075230
doi: 10.1371/journal.pbio.1000610
Manning, J. R., Jacobs, J., Fried, I. & Kahana, M. J. Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humans. J. Neurosci. 29, 13613–13620 (2009).
pubmed: 19864573
pmcid: 3001247
doi: 10.1523/JNEUROSCI.2041-09.2009
Slaney, M. Auditory toolbox. Interval Res. Corporation, Tech. Rep. 10, 1998 (1998).
Chi, T., Ru, P. & Shamma, S. A. Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118, 887–906 (2005).
pubmed: 16158645
doi: 10.1121/1.1945807
Singh, N. C. & Theunissen, F. E. Modulation spectra of natural sounds and ethological theories of auditory processing. J. Acoust. Soc. Am. 114, 3394–3411 (2003).
pubmed: 14714819
doi: 10.1121/1.1624067
Di Liberto, G. M., Wong, D., Melnik, G. A. & de Cheveigné, A. Low-frequency cortical responses to natural speech reflect probabilistic phonotactics. Neuroimage 196, 237–247 (2019).
pubmed: 30991126
doi: 10.1016/j.neuroimage.2019.04.037
Leonard, M. K., Bouchard, K. E., Tang, C. & Chang, E. F. Dynamic encoding of speech sequence probability in human temporal cortex. J. Neurosci. 35, 7203–7214 (2015).
pubmed: 25948269
pmcid: 4420784
doi: 10.1523/JNEUROSCI.4100-14.2015
Schoppe, O., Harper, N. S., Willmore, B. D., King, A. J. & Schnupp, J. W. Measuring the performance of neural models. Front. Comput. Neurosci. 10, 10 (2016).
pubmed: 26903851
pmcid: 4748266
doi: 10.3389/fncom.2016.00010
Mizrahi, A., Shalev, A. & Nelken, I. Single neuron and population coding of natural sounds in auditory cortex. Curr. Opin. Neurobiol. 24, 103–110 (2014).
pubmed: 24492086
doi: 10.1016/j.conb.2013.09.007
Chien, H.-Y. S. & Honey, C. J. Constructing and forgetting temporal context in the human cerebral cortex. Neuron 106, 675–686 (2020).
Panzeri, S., Brunel, N., Logothetis, N. K. & Kayser, C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, 111–120 (2010).
pubmed: 20045201
doi: 10.1016/j.tins.2009.12.001
Joris, P. X., Schreiner, C. E. & Rees, A. Neural processing of amplitude-modulated sounds. Physiol. Rev. 84, 541–577 (2004).
pubmed: 15044682
doi: 10.1152/physrev.00029.2003
Wang, X., Lu, T., Bendor, D. & Bartlett, E. Neural coding of temporal information in auditory thalamus and cortex. Neuroscience 154, 294–303 (2008).
pubmed: 18555164
doi: 10.1016/j.neuroscience.2008.03.065
Gao, X. & Wehr, M. A coding transformation for temporally structured sounds within auditory cortical neurons. Neuron 86, 292–303 (2015).
pubmed: 25819614
pmcid: 4393373
doi: 10.1016/j.neuron.2015.03.004
McDermott, J. H. & Simoncelli, E. P. Sound texture perception via statistics of the auditory periphery: evidence from sound synthesis. Neuron 71, 926–940 (2011).
pubmed: 21903084
pmcid: 4143345
doi: 10.1016/j.neuron.2011.06.032
Cohen, M. R. & Kohn, A. Measuring and interpreting neuronal correlations. Nat. Neurosci. 14, 811–819 (2011).
Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).
Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H. & Wang, X.-J. A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex. Neuron 88, 419–431 (2015).
pubmed: 26439530
pmcid: 4630024
doi: 10.1016/j.neuron.2015.09.008
Rauschecker, J. P. & Scott, S. K. Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat. Neurosci. 12, 718–724 (2009).
pubmed: 19471271
pmcid: 2846110
doi: 10.1038/nn.2331
Sharpee, T. O., Atencio, C. A. & Schreiner, C. E. Hierarchical representations in the auditory cortex. Curr. Opin. Neurobiol. 21, 761–767 (2011).
pubmed: 21704508
pmcid: 3223290
doi: 10.1016/j.conb.2011.05.027
Zatorre, R. J., Belin, P. & Penhune, V. B. Structure and function of auditory cortex: music and speech. Trends Cogn. Sci. 6, 37–46 (2002).
pubmed: 11849614
doi: 10.1016/S1364-6613(00)01816-7
Poeppel, D. The analysis of speech in different temporal integration windows: cerebral lateralization as ‘asymmetric sampling in time’. Speech Commun. 41, 245–255 (2003).
doi: 10.1016/S0167-6393(02)00107-3
Hamilton, L. S., Oganian, Y., Hall, J. & Chang, E. F. Parallel and distributed encoding of speech across human auditory cortex. Cell 184, 4626–4639 (2021).
pubmed: 34411517
doi: 10.1016/j.cell.2021.07.019
Nourski, K. V. et al. Functional organization of human auditory cortex: investigation of response latencies through direct recordings. NeuroImage 101, 598–609 (2014).
Bartlett, E. L. The organization and physiology of the auditory thalamus and its role in processing acoustic features important for speech perception. Brain Lang. 126, 29–48 (2013).
pubmed: 23725661
pmcid: 3707394
doi: 10.1016/j.bandl.2013.03.003
Gattass, R., Gross, C. G. & Sandell, J. H. Visual topography of V2 in the macaque. J. Comp. Neurol. 201, 519–539 (1981).
pubmed: 7287933
doi: 10.1002/cne.902010405
Dumoulin, S. O. & Wandell, B. A. Population receptive field estimates in human visual cortex. Neuroimage 39, 647–660 (2008).
pubmed: 17977024
doi: 10.1016/j.neuroimage.2007.09.034
Ding, N., Melloni, L., Zhang, H., Tian, X. & Poeppel, D. Cortical tracking of hierarchical linguistic structures in connected speech. Nat. Neurosci. 19, 158–164 (2016).
pubmed: 26642090
doi: 10.1038/nn.4186
Suied, C., Agus, T. R., Thorpe, S. J., Mesgarani, N. & Pressnitzer, D. Auditory gist: recognition of very short sounds from timbre cues. J. Acoust. Soc. Am. 135, 1380–1391 (2014).
pubmed: 24606276
doi: 10.1121/1.4863659
Donhauser, P. W. & Baillet, S. Two distinct neural timescales for predictive speech processing. Neuron 105, 385–393 (2020).
pubmed: 31806493
doi: 10.1016/j.neuron.2019.10.019
Ulanovsky, N., Las, L., Farkas, D. & Nelken, I. Multiple time scales of adaptation in auditory cortex neurons. J. Neurosci. 24, 10440–10453 (2004).
pubmed: 15548659
pmcid: 6730303
doi: 10.1523/JNEUROSCI.1905-04.2004
Lu, K. et al. Implicit memory for complex sounds in higher auditory cortex of the ferret. J. Neurosci. 38, 9955–9966 (2018).
pubmed: 30266740
pmcid: 6234296
doi: 10.1523/JNEUROSCI.2118-18.2018
Chew, S. J., Mello, C., Nottebohm, F., Jarvis, E. & Vicario, D. S. Decrements in auditory responses to a repeated conspecific song are long-lasting and require two periods of protein synthesis in the songbird forebrain. Proc. Natl Acad. Sci. USA 92, 3406–3410 (1995).
pubmed: 7724575
pmcid: 42175
doi: 10.1073/pnas.92.8.3406
Bianco, R. et al. Long-term implicit memory for sequential auditory patterns in humans. eLife 9, e56073 (2020).
pubmed: 32420868
pmcid: 7338054
doi: 10.7554/eLife.56073
Miller, K. J., Honey, C. J., Hermes, D., Rao, R. P. & Ojemann, J. G. Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. Neuroimage 85, 711–720 (2014).
pubmed: 24018305
doi: 10.1016/j.neuroimage.2013.08.070
Leszczyński, M. et al. Dissociation of broadband high-frequency activity and neuronal firing in the neocortex. Sci. Adv. 6, eabb0977 (2020).
pubmed: 32851172
pmcid: 7423365
doi: 10.1126/sciadv.abb0977
Günel, B., Thiel, C. M. & Hildebrandt, K. J. Effects of exogenous auditory attention on temporal and spectral resolution. Front. Psychol. 9, 1984 (2018).
pubmed: 30405479
pmcid: 6206225
doi: 10.3389/fpsyg.2018.01984
Norman-Haignere, S. V. et al. Pitch-responsive cortical regions in congenital amusia. J. Neurosci. 36, 2986–2994 (2016).
pubmed: 26961952
pmcid: 6601753
doi: 10.1523/JNEUROSCI.2705-15.2016
Norman-Haignere, S. et al. Intracranial recordings from human auditory cortex reveal a neural population selective for musical song. Preprint at bioRxiv https://doi.org/10.1101/696161 (2020).
Boebinger, D., Norman-Haignere, S. V., McDermott, J. H. & Kanwisher, N. Music-selective neural populations arise without musical training. J. Neurophysiol. 125, 2237–2263 (2021).
pubmed: 33596723
doi: 10.1152/jn.00588.2020
Morosan, P. et al. Human primary auditory cortex: cytoarchitectonic subdivisions and mapping into a spatial reference system. Neuroimage 13, 684–701 (2001).
pubmed: 11305897
doi: 10.1006/nimg.2000.0715
Baumann, S., Petkov, C. I. & Griffiths, T. D. A unified framework for the organization of the primate auditory cortex. Front. Syst. Neurosci. 7, 11 (2013).
pubmed: 23641203
pmcid: 3639404
doi: 10.3389/fnsys.2013.00011
Barr, D. J., Levy, R., Scheepers, C. & Tily, H. J. Random effects structure for confirmatory hypothesis testing: keep it maximal. J. Mem. Lang. 68, 255–278 (2013).
doi: 10.1016/j.jml.2012.11.001
Kuznetsova, A., Brockhoff, P. B. & Christensen, R. H. lmerTest package: tests in linear mixed effects models. J. Stat. Softw. 82, 1–26 (2017).
doi: 10.18637/jss.v082.i13
Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models (Cambridge Univ. Press, 2006).
Schielzeth, H. et al. Robustness of linear mixed-effects models to violations of distributional assumptions. Methods Ecol. Evol. 11, 1141–1152 (2020).
doi: 10.1111/2041-210X.13434
de Cheveigné, A. & Parra, L. C. Joint decorrelation, a versatile tool for multichannel data analysis. Neuroimage 98, 487–505 (2014).
pubmed: 24990357
doi: 10.1016/j.neuroimage.2014.05.068
Murphy, K. P. Machine Learning: A Probabilistic Perspective (MIT Press, 2012).
de Heer, W. A., Huth, A. G., Griffiths, T. L., Gallant, J. L. & Theunissen, F. E. The hierarchical cortical organization of human speech processing. J. Neurosci. 37, 6539–6557 (2017).
Marquardt, D. W. An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963).
doi: 10.1137/0111030
Fisher, W. M. tsylb: NIST syllabification software, version 2 revised (1997).