Progressing, not regressing: A possible solution to the problem of regression to the mean in unconscious processing studies.
Consciousness
Power
Regression to the mean
Reliability
Unconscious processing
Journal
Psychonomic bulletin & review
ISSN: 1531-5320
Titre abrégé: Psychon Bull Rev
Pays: United States
ID NLM: 9502924
Informations de publication
Date de publication:
01 Aug 2023
01 Aug 2023
Historique:
accepted:
06
06
2023
medline:
2
8
2023
pubmed:
2
8
2023
entrez:
1
8
2023
Statut:
aheadofprint
Résumé
How convincing is current evidence for unconscious processing? Recently, a major criticism suggested that some, if not much, of this evidence might be explained by a mere statistical phenomenon: regression to the mean (RttM). Excluding participants based on an awareness assessment is a common practice in studies of unconscious processing, and this post hoc data selection might lead to false effects that are driven by RttM for aware participants wrongfully classified as unaware. Here, we examined this criticism using both simulations and data from 12 studies probing unconscious processing (35 effects overall). In line with the original criticism, we confirmed that the reliability of awareness measures in the field is concerningly low. Yet, using simulations, we showed that reliability measures might be unsuitable for estimating error in awareness measures. Furthermore, we examined other solutions for assessing whether an effect is genuine or reflects RttM; all suffered from substantial limitations, such as a lack of specificity to unconscious processing, lack of power, or unjustified assumptions. Accordingly, we suggest a new nonparametric solution, which enjoys high specificity and relatively high power. Together, this work emphasizes the need to account for measurement error in awareness measures and evaluate its consequences for unconscious processing effects. It further suggests a way to meet the important challenge posed by RttM, in an attempt to establish a reliable and robust corpus of knowledge in studying unconscious processing.
Identifiants
pubmed: 37528278
doi: 10.3758/s13423-023-02326-x
pii: 10.3758/s13423-023-02326-x
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2023. The Author(s).
Références
Albrecht, T., Klapötke, S., & Mattler, U. (2010). Individual differences in metacontrast masking are enhanced by perceptual learning. Consciousness and Cognition, 19(2), 656–666.
pubmed: 20044281
doi: 10.1016/j.concog.2009.12.002
Avneon, M., & Lamy, D. (2018). Reexamining unconscious response priming: A liminal-prime paradigm. Consciousness and Cognition, 59, 87–103.
pubmed: 29329968
doi: 10.1016/j.concog.2017.12.006
Biderman, D., Shir, Y., & Mudrik, L. (2020). B or 13? Unconscious top-down contextual effects at the categorical but not the lexical level. Psychological Science, 31(6), 663–677.
pubmed: 32384011
doi: 10.1177/0956797620915887
Biderman, N., & Mudrik, L. (2017). Evidence for implicit—But not unconscious—Processing of object-scene relations. Psychological Science, 29(2), 266–277.
pubmed: 29283750
doi: 10.1177/0956797617735745
Breitmeyer, B. G. (2015). Psychophysical “blinding” methods reveal a functional hierarchy of unconscious visual processing. Consciousness and Cognition, 35, 234–250.
pubmed: 25704454
doi: 10.1016/j.concog.2015.01.012
Campbell, D. T., & Kenny, D. A. (1999). A primer on regression artifacts. Guilford Press.
Cheesman, J., & Merikle, P. M. (1986). Distinguishing conscious from unconscious perceptual processes. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 40(4), 343.
doi: 10.1037/h0080103
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98.
doi: 10.1037/0021-9010.78.1.98
Dosher, B. A. (1998). The response-window regression method—Some problematic assumptions: Comment on Draine and Greenwald (1998). Journal of Experimental Psychology: General, 127(3), 311–317.
doi: 10.1037/0096-3445.127.3.311
Faivre, N., Mudrik, L., Schwartz, N., & Koch, C. (2014). Multisensory integration in complete unawareness: Evidence from audiovisual congruency priming. Psychological Science, 25(11), 2006–2016.
pubmed: 25269620
doi: 10.1177/0956797614547916
Faivre, N., Salomon, R., Vuillaume, L., & Blanke, O. (2015). Numerical priming between touch and vision depends on tactile discrimination. Perception, 45(1/2), 114–124.
pubmed: 26562854
Goldstein, A., Sklar, A. Y., & Siegelman, N. (2022). Accurately measuring nonconscious processing using a generative Bayesian framework. Psychology of Consciousness: Theory, Research, and Practice. https://doi.org/10.1037/cns0000316
Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186.
pubmed: 28726177
doi: 10.3758/s13428-017-0935-1
Hesselmann, G., Darcy, N., Ludwig, K., & Sterzer, P. (2016). Priming in a shape task but not in a category task under continuous flash suppression. Journal of Vision, 16(3), 17.
pubmed: 26885629
doi: 10.1167/16.3.17
Hesselmann, G., Darcy, N., Sterzer, P., & Knops, A. (2015). Exploring the boundary conditions of unconscious numerical priming effects with continuous flash suppression. Consciousness and Cognition, 31, 60–72.
pubmed: 25460241
doi: 10.1016/j.concog.2014.10.009
Huang, Y.-F., Tan, E. G. F., Soon, C. S., & Hsieh, P.-J. (2014). Unconscious cues bias first saccades in a free-saccade task. Consciousness and Cognition, 29, 48–55.
pubmed: 25108793
doi: 10.1016/j.concog.2014.07.009
Huber, D. E., Potter, K. W., & Huszar, L. D. (2019). Less “story” and more “reliability” in cognitive neuroscience. Cortex: A Journal Devoted to the Study of the Nervous System and Behavior, 113, 347.
pubmed: 30638584
doi: 10.1016/j.cortex.2018.10.030
Karpinski, A., Briggs, J. C., & Yale, M. (2019). A direct replication: Unconscious arithmetic processing. European Journal of Social Psychology, 49(3), 637–644.
doi: 10.1002/ejsp.2390
Kiefer, M. (2019). Cognitive control over unconscious cognition: Flexibility and generalizability of task set influences on subsequent masked semantic priming. Psychological Research, 83(7), 1556–1570.
pubmed: 29654364
doi: 10.1007/s00426-018-1011-x
Kim, C.-Y., & Blake, R. (2005). Psychophysical magic: Rendering the visible ‘invisible’. Trends in Cognitive Sciences, 9(8), 381–388.
pubmed: 16006172
doi: 10.1016/j.tics.2005.06.012
Klauer, K. C., Greenwald, A. G., & Draine, S. C. (1998). Correcting for measurement error in detecting unconscious cognition: Comment on Draine and Greenwald (1998). Journal of Experimental Psychology: General, 127(3), 318–319.
pubmed: 9742719
doi: 10.1037/0096-3445.127.3.318
Kouider, S., & Dehaene, S. (2007). Levels of processing during non-conscious perception: A critical review of visual masking. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 857–875.
doi: 10.1098/rstb.2007.2093
Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2020). Improving inferences about null effects with Bayes factors and equivalence tests. The Journals of Gerontology: Series B, 75(1), 45–57.
doi: 10.1093/geronb/gby065
Leganes-Fonteneau, M., Scott, R., Duka, T., & Dienes, Z. (2021). Avoiding pitfalls: Bayes factors can be a reliable tool for post hoc data selection in implicit learning. Psychonomic Bulletin & Review, 28(6), 1848–1859.
doi: 10.3758/s13423-021-01901-4
Lin, Z., & Murray, S. O. (2014). Priming of awareness or how not to measure visual awareness. Journal of Vision, 14(1), 27.
pubmed: 24474824
doi: 10.1167/14.1.27
Merikle, P. M., & Reingold, E. M. (1998). On demonstrating unconscious perception: Comment on Draine and Greenwald (1998). Journal of Experimental Psychology: General, 127(3), 304–310.
pubmed: 9742718
doi: 10.1037/0096-3445.127.3.304
Meyen, S., Zerweck, I. A., Amado, C., von Luxburg, U., & Franz, V. H. (2022). Advancing research on unconscious priming: When can scientists claim an indirect task advantage? Journal of Experimental Psychology: General, 151(1), 65–81.
pubmed: 34264714
doi: 10.1037/xge0001065
Michel, M. (2022). How (not) to underestimate unconscious perception. Mind and Language. https://doi.org/10.1111/mila.12406
Moors, P., Boelens, D., van Overwalle, J., & Wagemans, J. (2016). Scene integration without awareness. Psychological Science, 27(7), 945–956.
pubmed: 27225219
doi: 10.1177/0956797616642525
Moors, P., & Hesselmann, G. (2017). A critical reexamination of doing arithmetic nonconsciously. Psychonomic Bulletin & Review, 25(1), 472–481.
doi: 10.3758/s13423-017-1292-x
Mudrik, L., Breska, A., Lamy, D., & Deouell, L. Y. (2011). Integration without awareness: Expanding the limits of unconscious processing. Psychological Science, 22(6), 764–770.
pubmed: 21555524
doi: 10.1177/0956797611408736
Nakashima, Y., & Sugita, Y. (2018). Size-contrast illusion induced by unconscious context. Journal of Vision, 18(3), 16.
pubmed: 29677332
doi: 10.1167/18.3.16
Newell, B. R., & Shanks, D. R. (2014). Unconscious influences on decision making: A critical review. Behavioral and Brain Sciences, 37(1), 1–19.
pubmed: 24461214
doi: 10.1017/S0140525X12003214
Pessoa, L., Japee, S., Sturman, D., & Ungerleider, L. G. (2006). Target visibility and visual awareness modulate amygdala responses to fearful faces. Cerebral Cortex, 16(3), 366–375.
pubmed: 15930371
doi: 10.1093/cercor/bhi115
Peters, M. A. K., Kentridge, R. W., Phillips, I., & Block, N. (2017). Does unconscious perception really exist? Continuing the ASSC20 debate. Neuroscience of Consciousness, 2017(1). https://doi.org/10.1093/nc/nix015
Reber, T. P., Luechinger, R., Boesiger, P., & Henke, K. (2014). Detecting analogies unconsciously. Frontiers in Behavioral Neuroscience, 8(JAN). https://doi.org/10.3389/fnbeh.2014.00009
Reber, T. P., & Henke, K. (2012). Integrating unseen events over time. Consciousness and Cognition, 21(2), 953–960.
pubmed: 22425537
doi: 10.1016/j.concog.2012.02.013
Reingold, E. M., & Merikle, P. M. (1988). Using direct and indirect measures to study perception without awareness. Perception & Psychophysics, 44(6), 563–575.
doi: 10.3758/BF03207490
Rothkirch, M., & Hesselmann, G. (2017). What we talk about when we talk about unconscious processing—A plea for best practices. Frontiers in Psychology, 8, 835.
pubmed: 28588539
pmcid: 5440724
doi: 10.3389/fpsyg.2017.00835
Rothkirch, M., Madipakkam, A. R., Rehn, E., & Sterzer, P. (2015). Making eye contact without awareness. Cognition, 143, 108–114.
pubmed: 26133642
doi: 10.1016/j.cognition.2015.06.012
Rothkirch, M., Shanks, D. R., & Hesselmann, G. (2022). The pervasive problem of post hoc data selection in studies on unconscious processing. Experimental Psychology. https://doi.org/10.1027/1618-3169/a000541
Rouder, J. N., Morey, R. D., Speckman, P. L., & Pratte, M. S. (2007). Detecting chance: A solution to the null sensitivity problem in subliminal priming. Psychonomic Bulletin & Review, 14(4), 597–605.
doi: 10.3758/BF03196808
Rowe, E. G., Tsuchiya, N., & Garrido, M. I. (2020). Detecting (un)seen change: The neural underpinnings of (un)conscious prediction errors. Frontiers in Systems Neuroscience, 14, 81.
doi: 10.3389/fnsys.2020.541670
Sandberg, K., Timmermans, B., Overgaard, M., & Cleeremans, A. (2010). Measuring consciousness: Is one measure better than the other? Consciousness and Cognition, 19(4), 1069–1078.
pubmed: 20133167
doi: 10.1016/j.concog.2009.12.013
Schmidt, T. (2015). Invisible stimuli, implicit thresholds: Why invisibility judgments cannot be interpreted in isolation. Advances in Cognitive Psychology, 11(2), 31–41.
pubmed: 26203311
pmcid: 4510198
doi: 10.5709/acp-0169-3
Schwiedrzik, C. M., Singer, W., & Melloni, L. (2009). Sensitivity and perceptual awareness increase with practice in metacontrast masking. Journal of Vision, 9(10), 18–18.
doi: 10.1167/9.10.18
Seger, C. A. (1994). Implicit learning. Psychological Bulletin, 115(2), 163–196.
pubmed: 8165269
doi: 10.1037/0033-2909.115.2.163
Sergent, C., Baillet, S., & Dehaene, S. (2005). Timing of the brain events underlying access to consciousness during the attentional blink. Nature Neuroscience, 8(10), 1391–1400.
pubmed: 16158062
doi: 10.1038/nn1549
Shanks, D. R. (2017). Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processes. Psychonomic Bulletin & Review, 24(3), 752–775.
doi: 10.3758/s13423-016-1170-y
Shanks, D. R., Malejka, S., & Vadillo, M. A. (2021). The challenge of inferring unconscious mental processes. Experimental Psychology, 68(3), 113–129.
pubmed: 34435511
doi: 10.1027/1618-3169/a000517
Sklar, A. Y., Levy, N., Goldstein, A., Mandel, R., Maril, A., & Hassin, R. R. (2012). Reading and doing arithmetic nonconsciously. Proceedings of the National Academy of Sciences, 109(48), 19614–19619.
doi: 10.1073/pnas.1211645109
Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 1904–1920, 3(3), 271–295.
doi: 10.1111/j.2044-8295.1910.tb00206.x
Stein, T., Kaiser, D., Fahrenfort, J. J., & van Gaal, S. (2021). The human visual system differentially represents subjectively and objectively invisible stimuli. PLOS Biology, 19(5), e3001241.
pubmed: 33951043
pmcid: 8128378
doi: 10.1371/journal.pbio.3001241
Stein, T., Utz, V., & van Opstal, F. (2020). Unconscious semantic priming from pictures under backward masking and continuous flash suppression. Consciousness and Cognition, 78, 102864.
pubmed: 31896031
doi: 10.1016/j.concog.2019.102864
Vadillo, M. A., Malejka, S., Lee, D. Y. H., Dienes, Z., & Shanks, D. R. (2022). Raising awareness about measurement error in research on unconscious mental processes. Psychonomic Bulletin & Review, 29(1), 21–43.
doi: 10.3758/s13423-021-01923-y
van Gaal, S., Naccache, L., Meuwese, J. D. I., van Loon, A. M., Leighton, A. H., Cohen, L., & Dehaene, S. (2014). Can the meaning of multiple words be integrated unconsciously? Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1641), 20130212.
doi: 10.1098/rstb.2013.0212
Van Opstal, F., Calderon, C. B., Gevers, W., & Verguts, T. (2011a). Setting the stage subliminally: Unconscious context effects. Consciousness and Cognition, 20(4), 1860–1864.
pubmed: 21963331
doi: 10.1016/j.concog.2011.09.004
Van Opstal, F., de Lange, F., & Dehaene, S. (2011b). Rapid parallel semantic processing of numbers without awareness. Cognition, 120(1), 136–147.
pubmed: 21489415
doi: 10.1016/j.cognition.2011.03.005
Van Opstal, F., Gevers, W., Osman, M., & Verguts, T. (2010). Unconscious task application. Consciousness and Cognition, 19(4), 999–1006.
pubmed: 20537925
doi: 10.1016/j.concog.2010.05.002
Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. The Journal of Strength & Conditioning Research, 19(1), 231–240.
Zerweck, I. A., Kao, C.-S., Meyen, S., Amado, C., von Eltz, M., Klimm, M., & Franz, V. H. (2021). Number processing outside awareness? Systematically testing sensitivities of direct and indirect measures of consciousness. Attention, Perception, & Psychophysics, 83(6), 2510–2529.
doi: 10.3758/s13414-021-02312-2