Semantic priming from McGurk words: Priming depends on perception.
Multisensory processing
Semantic priming
Journal
Attention, perception & psychophysics
ISSN: 1943-393X
Titre abrégé: Atten Percept Psychophys
Pays: United States
ID NLM: 101495384
Informations de publication
Date de publication:
May 2023
May 2023
Historique:
accepted:
23
02
2023
medline:
10
5
2023
pubmed:
8
5
2023
entrez:
8
5
2023
Statut:
ppublish
Résumé
The McGurk effect is an illusion in which visible articulations alter the perception of auditory speech (e.g., video 'da' dubbed with audio 'ba' may be heard as 'da'). To test the timing of the multisensory processes that underlie the McGurk effect, Ostrand et al. Cognition 151, 96-107, 2016 used incongruent stimuli, such as auditory 'bait' + visual 'date' as primes in a lexical decision task. These authors reported that the auditory word, but not the perceived (visual) word, induced semantic priming, suggesting that the auditory signal alone can provide the input for lexical access, before multisensory integration is complete. Here, we conceptually replicate the design of Ostrand et al. (2016), using different stimuli chosen to optimize the success of the McGurk illusion. In contrast to the results of Ostrand et al. (2016), we find that the perceived (i.e., visual) word of the incongruent stimulus usually induced semantic priming. We further find that the strength of this priming corresponded to the magnitude of the McGurk effect for each word combination. These findings suggest, in contrast to the findings of Ostrand et al. (2016), that lexical access makes use of integrated multisensory information which is perceived by the listener. These findings further suggest that which unimodal signal of a multisensory stimulus is used in lexical access is dependent on the perception of that stimulus.
Identifiants
pubmed: 37155085
doi: 10.3758/s13414-023-02689-2
pii: 10.3758/s13414-023-02689-2
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1219-1237Informations de copyright
© 2023. The Psychonomic Society, Inc.
Références
Alsius, A., Paré, M., & Munhall, K. G. (2018). Forty years after hearing lips and seeing voices: The McGurk effect revisited. Multisensory Research, 31(1/2), 111–144. https://doi.org/10.1163/22134808-00002565
doi: 10.1163/22134808-00002565
pubmed: 31264597
Arnold, P., & Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92(2), 339–355. https://doi.org/10.1348/000712601162220
doi: 10.1348/000712601162220
pubmed: 11802877
Baart, M., & Samuel, A. G. (2015). Turning a blind eye to the lexicon: ERPs show no cross-talk between lip-read and lexical context during speech sound processing. Journal of Memory and Language, 85(2015). https://doi.org/10.1016/j.jml.2015.06.00
Barutchu, A., Crewther, S. G., Kiely, P., Murphy, M. J., & Crewther, D. P. (2008). When /b/ill with /g/ill becomes /d/ill: Evidence for a lexical effect in audiovisual speech perception. European Journal of Cognitive Psychology, 20(1), 1–11. https://doi.org/10.1080/09541440601125623
doi: 10.1080/09541440601125623
Basu Mallick, D., Magnotti, J. F., & Beauchamp, M. S. (2015). Variability and stability in the McGurk effect: Contributions of participants, stimuli, time, and response type. Psychonomic Bulletin and Review, 22(5), 1299–1307. https://doi.org/10.3758/s13423-015-0817-4
doi: 10.3758/s13423-015-0817-4
Bebko, J. M., Schroeder, J. H., & Weiss, J. A. (2014). The McGurk effect in children with autism and asperger syndrome. Autism Research, 7(1), 50–59. https://doi.org/10.1002/aur.1343
doi: 10.1002/aur.1343
pubmed: 24136870
Besle, J., Fischer, C., Bidet-Caulet, A., Lecaignard, F., Bertrand, O., & Giard, M.-H. (2008). Visual activation and audiovisual interactions in the auditory cortex during speech perception: Intracranial recordings in humans. Journal of Neuroscience, 28(52), 14301–14310. https://doi.org/10.1523/JNEUROSCI.2875-08.2008
doi: 10.1523/JNEUROSCI.2875-08.2008
pubmed: 19109511
Brancazio, L. (2004). Lexical influences in audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 30(3), 445–463. https://doi.org/10.1037/0096-1523.30.3.445
doi: 10.1037/0096-1523.30.3.445
pubmed: 15161378
Brancazio, L., & Miller, J. L. (2005). Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect. Perception & Psychophysics, 67(5), 759–769. https://doi.org/10.3758/BF03193531
doi: 10.3758/BF03193531
Brown, V. A., Hedayati, M., Zanger, A., Mayn, S., Ray, L., Dillman-Hasso, N., & Strand, J. F. (2018). What accounts for individual differences in susceptibility to the McGurk effect? PLOS ONE, 13(11), 1–20. https://doi.org/10.1371/journal.pone.0207160
doi: 10.1371/journal.pone.0207160
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P. K., Woodruff, P. W., Iversen, S. D., & David, A. S. (1997). Activation of auditory cortex during silent lipreading. Science, 276(5312), 593–596. https://doi.org/10.1126/science.276.5312.593
doi: 10.1126/science.276.5312.593
pubmed: 9110978
Cho, H. C., & Abe, S. (2013). Is two-tailed testing for directional research hypotheses tests legitimate? Journal of Business Research, 66(9), 1261–1266.
doi: 10.1016/j.jbusres.2012.02.023
Dias, J. W., & Rosenblum, L. D. (2011). Visual influences on interactive speech alignment. Perception, 40, 1457–1466. https://doi.org/10.1068/p7071
doi: 10.1068/p7071
pubmed: 22474764
pmcid: 8098124
Dorsi, J. (2019). Understanding how lexical and multisensory contexts support speech perception. University of California, Riverside.
Dorsi, J., Rosenblum, L. D., Samuel, A. G., & Zadoorian, S. (2021). Selective adaptation in speech: Measuring the effects of visual and lexical contexts. Journal of Experimental Psychology. Human Perception and Performance, 47(8), 1023–1042. https://doi.org/10.1037/xhp0000769
doi: 10.1037/xhp0000769
pubmed: 34516210
Fort, M., Kandel, S., Chipot, J., Savariaux, C., Granjon, L., & Spinelli, E. (2013). Seeing the initial articulatory gestures of a word triggers lexical access. Language and Cognitive Processes, 28(8), 1–17. https://doi.org/10.1080/01690965.2012.701758
doi: 10.1080/01690965.2012.701758
Fowler, C. A., Brown, J. M., & Mann, V. A. (2000). Contrast effects do not underlie effects of preceding liquids on stop-consonant identification by humans. Journal of Experimental Psychology: Human Perception and Performance, 26(3), 877–888. https://doi.org/10.1037/0096-1523.26.3.877
doi: 10.1037/0096-1523.26.3.877
pubmed: 10883999
Goldinger, S. D. (1996). Auditory lexical decision. Language and Cognitive Processes, 11(6), 559–568. https://doi.org/10.1080/016909696386944
doi: 10.1080/016909696386944
Green, K. P., & Kuhl, P. K. (1989). The role of visual information in the processing of place and manner features in speech perception. Perception & Psychophysics, 45(1), 34–42. https://doi.org/10.3758/BF03208030
doi: 10.3758/BF03208030
Green, K. P., & Miller, J. L. (1985). On the role of visual rate information in phonetic perception. Perception & Psychophysics, 38(3), 269–276 http://www.ncbi.nlm.nih.gov/pubmed/4088819
doi: 10.3758/BF03207154
Green K. P. & Norrix L. W. (2001). Perception of/r/and/l/in a stop cluster: Evidence of cross-modal context effects. Journal of Experimental Psychology: Human Perception and Performance, 27(1), 166.
Green, K. P., & Norrix, L. W. (1997). Acoustic cues to place of articulation and the McGurk effect: The role of release bursts, aspiration, and formant transitions. Journal of Speech, Language and Hearing Reseach, 40(3), 646–665. https://doi.org/10.1044/jslhr.4003.646
doi: 10.1044/jslhr.4003.646
Kimmel, H. D. (1957). Three criteria for the use of one-tailed tests. Psychological Bulletin, 54(4), 351–353.
doi: 10.1037/h0046737
pubmed: 13465928
Kiss, G. R., Armstrong, C. A., & Milroy, R. (1972). An associative thesaurus of English (pp. 153–165). Medical Research Council, Speech and Communication Unit, University of Edinburgh.
MacDonald, J., & McGurk, H. (1978). Visual influences on speech perception processes. Perception & Psychophysics, 24(3), 253–257 http://www.ncbi.nlm.nih.gov/pubmed/704285
doi: 10.3758/BF03206096
Magnotti, J. F., & Beauchamp, M. S. (2015). The noisy encoding of disparity model of the McGurk effect. Psychonomic Bulletin & Review, 22(3), 701–709. https://doi.org/10.3758/s13423-014-0722-2
doi: 10.3758/s13423-014-0722-2
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.
doi: 10.1038/264746a0
pubmed: 1012311
Mitterer, H., & Reinisch, E. (2017). Visual speech influences speech perception immediately but not automatically. Attention, Perception, & Psychophysics, 79(2), 660–678. https://doi.org/10.3758/s13414-016-1249-6
doi: 10.3758/s13414-016-1249-6
Musacchia, G., Sams, M., Nicol, T., & Kraus, N. (2006). Seeing speech affects acoustic information processing in the human brainstem. Experimental Brain Research, 168(1/2), 1–10. https://doi.org/10.1007/s00221-005-0071-5
doi: 10.1007/s00221-005-0071-5
pubmed: 16217645
Namasivayam, A. K., Yiu, W., & Wong, S. (2015). Visual speech gestures modulate efferent auditory system. Journal of Integrative Neuroscience, 14(1), 73–83. https://doi.org/10.1142/S0219635215500016
doi: 10.1142/S0219635215500016
pubmed: 25597277
Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71(1), 4–12. https://doi.org/10.1007/s00426-005-0031-5
doi: 10.1007/s00426-005-0031-5
pubmed: 16362332
Neely, J. H. (1977). Semantic priming and retrieval from lexical memory: Roles of inhibitionless spreading activation and limited-capacity attention. Journal of Experimental Psychology: General, 106(3), 226–254. https://doi.org/10.1037/0096-3445.106.3.226
doi: 10.1037/0096-3445.106.3.226
Nelson, D., McEvoy, C. L., & Schreiber, T. A. (1998). The University of South Florida Word Association, Rhyme, and Word Fragment norms. Behavior Research Methods, Instruments, & Computers, 36(3) http://www.usf.edu/FreeAssociation/
Ostrand, R., Blumstein, S. E., & Morgan, J. L. (2011). When hearing lips and seeing voices becomes perceiving speech: Auditory-visual integration in lexical access. Proceedings of the Annual Meeting of the Cognitive Science Society, 33(33), 1376–1381. https://doi.org/10.1016/j.neuroimage.2010.12.063.Discrete
doi: 10.1016/j.neuroimage.2010.12.063.Discrete
Ostrand, R., Blumstein, S. E., Ferreira, V. S., & Morgan, J. L. (2016). What you see isn’t always what you get: Auditory word signals trump consciously perceived words in lexical access. Cognition, 151(2016), 96–107. https://doi.org/10.1016/j.cognition.2016.02.019
doi: 10.1016/j.cognition.2016.02.019
pubmed: 27011021
pmcid: 4850493
Pekkola, J., Ojanen, V., Autti, T., Jääskeläinen, I. P., Möttönen, R., Tarkiainen, A., & Sams, M. (2005). Primary auditory cortex activation by visual speech: An fMRI study at 3 T. NeuroReport, 16(2), 125–128. https://doi.org/10.1097/00001756-200502080-00010
doi: 10.1097/00001756-200502080-00010
pubmed: 15671860
Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A speechreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97–113). Erlbaum.
Roberts, M., & Summerfield, Q. (1981). Audiovisual presentation demonstrates that selective adaptation in speech perception is purely auditory. Perception & Psychophysics, 30(4), 309–314. https://doi.org/10.3758/BF03206144
doi: 10.3758/BF03206144
Rosenblum, L. D. (2019, August). Audiovisual speech perception and the McGurk effect. Oxford Research Encyclopedia of Linguistics, August, 1–28. https://doi.org/10.1093/acrefore/9780199384655.013.420
Rosenblum, L. D., & Saldaña, H. M. (1992). Discrimination tests of visually influenced syllables. Perception & Psychophysics, 52(4), 461–473. https://doi.org/10.3758/BF03206706
doi: 10.3758/BF03206706
Rosenblum, L. D., Dias, J. W., & Dorsi, J. (2016a). The supramodal brain: Implications for auditory perception. Journal of Cognitive Psychology, 5911(1), 1–23. https://doi.org/10.1080/20445911.2016.1181691
doi: 10.1080/20445911.2016.1181691
Rosenblum, L. D., Dorsi, J., & Dias, J. W. (2016b). The impact and status of Carol Fowler’s supramodal theory of multisensory speech perception. Ecological Psychology, 28(4), 262–294. https://doi.org/10.1080/10407413.2016.1230373
doi: 10.1080/10407413.2016.1230373
Ruxton, G. D., & Neuhäuser, M. (2010). When should we use one-tailed hypothesis testing? Methods in Ecology and Evolution, 1(2), 114–117.
doi: 10.1111/j.2041-210X.2010.00014.x
Saldaña, H. M., & Rosenblum, L. D. (1994). Selective adaptation in speech perception using a compelling audiovisual adaptor. The Journal of the Acoustical Society of America, 95(6), 3658–3661. https://doi.org/10.1121/1.409935
doi: 10.1121/1.409935
pubmed: 8046153
Sams, M., Manninen, P., Surakka, V., Helin, P., & Kättö, R. (1998). McGurk effect in Finnish syllables, isolated words, and words in sentences: Effects of word meaning and sentence context. Speech Communication, 26(1/2), 75–87. https://doi.org/10.1016/S0167-6393(98)00051-X
doi: 10.1016/S0167-6393(98)00051-X
Samuel, A. G. (1997). Lexical activation produces potent phonemic percepts. Cognitive Psychology, 127(2), 97–127.
doi: 10.1006/cogp.1997.0646
Samuel, A. G., & Lieblich, J. (2014). Visual speech acts differently than lexical context in supporting speech perception. Journal of Experimental Psychology. Human Perception and Performance, 40(4), 1479–1490. https://doi.org/10.1037/a0036656
doi: 10.1037/a0036656
pubmed: 24749935
pmcid: 4122614
Sanchez, K., Miller, R. M., & Rosenblum, L. D. (2010). Visual influences on alignment. Journal of Speech, Language, and Hearing Research, 53, 262–272.
doi: 10.1044/1092-4388(2009/08-0247)
pubmed: 20220027
Stropahl, M., Schellhardt, S., & Debener, S. (2016). McGurk stimuli for the investigation of multisensory integration in cochlear implant users: The Oldenburg Audio Visual Speech Stimuli (OLAVS). Psychonomic Bulletin & Review, 1–10. https://doi.org/10.3758/s13423-016-1148-9
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26(2), 212–215.
doi: 10.1121/1.1907309
Teinonen, T., Aslin, R. N., Alku, P., & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 108(3), 850–855. https://doi.org/10.1016/j.cognition.2008.05.009
doi: 10.1016/j.cognition.2008.05.009
pubmed: 18590910
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45(3), 598–607. https://doi.org/10.1016/j.neuropsychologia.2006.01.001
doi: 10.1016/j.neuropsychologia.2006.01.001
pubmed: 16530232
Warren, R. M. (1970). Perceptual restoration of missing speech sounds. Science, 167(3917), 392–393. https://doi.org/10.1126/science.167.3917.392
doi: 10.1126/science.167.3917.392
pubmed: 5409744
Windmann, S. (2004). Effects of sentence context and expectation on the McGurk illusion. Journal of Memory and Language, 50(2), 212–230. https://doi.org/10.1016/j.jml.2003.10.001
doi: 10.1016/j.jml.2003.10.001