Gravitational models explain shifts on human visual attention.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
01 10 2020
01 10 2020
Historique:
received:
23
02
2020
accepted:
11
09
2020
entrez:
2
10
2020
pubmed:
3
10
2020
medline:
2
1
2021
Statut:
epublish
Résumé
Visual attention refers to the human brain's ability to select relevant sensory information for preferential processing, improving performance in visual and cognitive tasks. It proceeds in two phases. One in which visual feature maps are acquired and processed in parallel. Another where the information from these maps is merged in order to select a single location to be attended for further and more complex computations and reasoning. Its computational description is challenging, especially if the temporal dynamics of the process are taken into account. Numerous methods to estimate saliency have been proposed in the last 3 decades. They achieve almost perfect performance in estimating saliency at the pixel level, but the way they generate shifts in visual attention fully depends on winner-take-all (WTA) circuitry. WTA is implemented by the biological hardware in order to select a location with maximum saliency, towards which to direct overt attention. In this paper we propose a gravitational model to describe the attentional shifts. Every single feature acts as an attractor and the shifts are the result of the joint effects of the attractors. In the current framework, the assumption of a single, centralized saliency map is no longer necessary, though still plausible. Quantitative results on two large image datasets show that this model predicts shifts more accurately than winner-take-all.
Identifiants
pubmed: 33005008
doi: 10.1038/s41598-020-73494-2
pii: 10.1038/s41598-020-73494-2
pmc: PMC7530662
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
16335Références
Koch, K. et al. How much the eye tells the brain. Curr. Biol. 16, 1428–1434 (2006).
pubmed: 16860742
pmcid: 1564115
doi: 10.1016/j.cub.2006.05.056
Borji, A., Sihite, D. N. & Itti, L. Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Trans. Image Process. 22, 55–69 (2013).
pubmed: 22868572
doi: 10.1109/TIP.2012.2210727
Smith, P. L. & Ratcliff, R. An integrated theory of attention and decision making in visual signal detection. Psychol. Rev. 116, 283 (2009).
pubmed: 19348543
doi: 10.1037/a0015156
Hood, B. M., Willen, J. D. & Driver, J. Adult's eyes trigger shifts of visual attention in human infants. Psychol. Sci. 9, 131–134 (1998).
doi: 10.1111/1467-9280.00024
Duncan, J. Converging levels of analysis in the cognitive neuroscience of visual attention. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 353, 1307–1317 (1998).
doi: 10.1098/rstb.1998.0285
Martinez-Conde, S., Otero-Millan, J. & Macknik, S. L. The impact of microsaccades on vision: towards a unified theory of saccadic function. Nat. Rev. Neurosci. 14, 83 (2013).
pubmed: 23329159
doi: 10.1038/nrn3405
Itti, L., Koch, C. & Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
doi: 10.1109/34.730558
Bruce, N. & Tsotsos, J. Attention based on information maximization. J. Vis. 7, 950–950 (2007).
doi: 10.1167/7.9.950
Judd, T., Ehinger, K., Durand, F. & Torralba, A. Learning to predict where humans look. In IEEE 12th International Conference On Computer Vision 2106–2113 (2009).
Zanca, D. & Gori, M. Variational laws of visual attention for dynamic scenes. In Advances in Neural Information Processing Systems 3823–3832 (2017).
Cornia, M., Baraldi, L., Serra, G. & Cucchiara, R. A deep multi-level network for saliency prediction. In 2016 23rd International Conference on Pattern Recognition (ICPR) 3488–3493 (IEEE, 2016).
Borji, A. & Itti, L. State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
pubmed: 22487985
McMains, S. A. & Kastner, S. Visual Attention 4296–4302 (Springer, Berlin, 2009).
Itti, L. & Koch, C. Computational modelling of visual attention. Nat. Rev. Neurosci. 2, 194 (2001).
pubmed: 11256080
Connor, C. E., Egeth, H. E. & Yantis, S. Visual attention: bottom-up versus top-down. Curr. Biol. 14, R850–R852 (2004).
pubmed: 15458666
doi: 10.1016/j.cub.2004.09.041
Zanca, D., Gori, M. & Rufa, A. A unified computational framework for visual attention dynamics. Prog. Brain Res. https://doi.org/10.1016/bs.pbr.2019.01.001 (2019).
doi: 10.1016/bs.pbr.2019.01.001
pubmed: 31325977
Hankinson, G. The brand images of tourism destinations: a study of the saliency of organic images. J. Product Brand Manag. 13, 6–14 (2004).
doi: 10.1108/10610420410523803
Milosavljevic, M., Navalpakkam, V., Koch, C. & Rangel, A. Relative visual saliency differences induce sizable bias in consumer choice. J. Consum. Psychol. 22, 67–74 (2012).
doi: 10.1016/j.jcps.2011.10.002
Guo, C. & Zhang, L. A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19, 185–198 (2009).
Sitzmann, V. et al. Saliency in VR: how do people explore virtual environments?. IEEE Trans. Vis. Comput. Graph. 24, 1633–1642 (2018).
pubmed: 29553930
Womelsdorf, T., Anton-Erxleben, K., Pieper, F. & Treue, S. Dynamic shifts of visual receptive fields in cortical area MT by spatial attention. Nat. Neurosci. 9, 1156 (2006).
pubmed: 16906153
Corbetta, M. et al. A common network of functional areas for attention and eye movements. Neuron 21, 761–773 (1998).
pubmed: 9808463
Nobre, A. C. et al. Functional localization of the system for visuospatial attention using positron emission tomography. Brain J. Neurol. 120, 515–533 (1997).
Koch, C. & Ullman, S. Shifts in selective visual attention: towards the underlying neural circuitry. In Matters of Intelligence (ed. Vaina, L. M.) 115–141 (Springer, Dordrecht, 1987).
Duan, H. & Wang, X. Visual attention model based on statistical properties of neuron responses. Sci. Rep. 5, 8873 (2015).
pubmed: 25747859
pmcid: 4352866
Itti, L. Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis. Cogn. 12, 1093–1123 (2005).
Zhang, X., Zhaoping, L., Zhou, T. & Fang, F. Neural activities in v1 create a bottom-up saliency map. Neuron 73, 183–192 (2012).
pubmed: 22243756
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607 (1996).
pubmed: 8637596
Westerberg, J. A., Maier, A. & Schall, J. D. Priming of attentional selection in macaque visual cortex: feature-based facilitation and location-based inhibition of return. Eneuro 7, 1–15 (2020).
Burkhalter, A. & Bernardo, K. L. Organization of corticocortical connections in human visual cortex. Proc. Natl. Acad. Sci. 86, 1071–1075 (1989).
pubmed: 2464827
doi: 10.1073/pnas.86.3.1071
Jurafsky, D. & Martin, J. H. Speech and Language Processing Vol. 3 (Pearson, London, 2014).
Brandt, S. A. & Stark, L. W. Spontaneous eye movements during visual imagery reflect the content of the visual scene. J. Cogn. Neurosci. 9, 27–38 (1997).
pubmed: 23968178
doi: 10.1162/jocn.1997.9.1.27
Foulsham, T. & Underwood, G. What can saliency models predict about eye movements? Spatial and sequential aspects of fixations during encoding and recognition. J. vis. 8, 6–6 (2008).
pubmed: 18318632
doi: 10.1167/8.2.6
Zanca, D., Melacci, S. & Gori, M. Gravitational laws of focus of attention. In IEEE Transactions on Pattern Analysis and Machine Intelligence (2019).
Wang, W. et al. Simulating human saccadic scanpaths on natural images. In 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 441–448 (IEEE, 2011).
Zanca, D., Serchi, V., Piu, P., Rosini, F. & Rufa, A. Fixatons: a collection of human fixations datasets and metrics for scanpath similarity. ArXiv preprint, arXiv:1802.02534 (2018).
Bichot, N. P. & Schall, J. D. Priming in macaque frontal cortex during popout visual search: feature-based facilitation and location-based inhibition of return. J. Neurosci. 22, 4675–4685 (2002).
pubmed: 12040074
pmcid: 6758799
doi: 10.1523/JNEUROSCI.22-11-04675.2002
Posner, M. I., Rafal, R. D., Choate, L. S. & Vaughan, J. Inhibition of return: neural basis and function. Cogn. Neuropsychol. 2, 211–228 (1985).
doi: 10.1080/02643298508252866
Gibson, B. S. & Egeth, H. Inhibition and disinhibition of return: evidence from temporal order judgments. Percept. Psychophys. 56, 669–680 (1994).
pubmed: 7816537
doi: 10.3758/BF03208360
Pratt, J. & Abrams, R. A. Inhibition of return in discrimination tasks. J. Exp. Psychol. Hum. Percept. Perform. 25, 229 (1999).
pubmed: 10069033
doi: 10.1037/0096-1523.25.1.229
Milliken, B. & Tipper, S. P. Attention and inhibition. In Attention (ed. H. Pashler) 191–221 (Psychology Press, 1998).
Mondor, T. A., Breau, L. M. & Milliken, B. Inhibitory processes in auditory selective attention: evidence of location-based and frequency-based inhibition of return. Percept. Psychophys. 60, 296–302 (1998).
pubmed: 9529913
doi: 10.3758/BF03206038
Law, M. B., Pratt, J. & Abrams, R. A. Color-based inhibition of return. Percept. Psychophys. 57, 402–408 (1995).
pubmed: 7770330
doi: 10.3758/BF03213064
Houghton, G. & Tipper, S. P. A Model of Inhibitory Mechanisms in Selective Attention (Academic Press Ltd, London, 1984).
Milliken, B., Tipper, S. P., Houghton, G. & Lupiáñez, J. Attending, ignoring, and repetition: on the relation between negative priming and inhibition of return. Percept. Psychophys. 62, 1280–1296 (2000).
pubmed: 11019624
doi: 10.3758/BF03212130
Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97–136 (1980).
pubmed: 7351125
doi: 10.1016/0010-0285(80)90005-5
Bylinskii, Z. et al. Mit saliency benchmark. (Accessed 1 September 2019); http://saliency.mit.edu/ .
Le Meur, O. & Coutrot, A. Introducing context-dependent and spatially-variant viewing biases in saccadic models. Vis. Res. 121, 72–84 (2016).
pubmed: 26898752
doi: 10.1016/j.visres.2016.01.005
Renninger, L. W., Coughlan, J. M., Verghese, P. & Malik, J. An information maximization model of eye movements. In Advances in Neural Information Processing Systems 1121–1128 (2005).
Jiang, M. et al. Learning to predict sequences of human visual fixations. IEEE Trans. Neural Netw. Learn. Syst. 27, 1241–1252 (2016).
pubmed: 26761903
Kümmerer, M., Wallis, T. & Bethge, M. Deepgaze ii: Predicting fixations from deep features over time and tasks. In 17th Annual Meeting of the Vision Sciences Society (VSS 2017) 1147–1147 (2017).
Abarbanel, H. D., Carroll, T., Pecora, L., Sidorowich, J. & Tsimring, L. Predicting physical variables in time-delay embedding. Phys. Rev. E 49, 1840 (1994).
Henderson, J. M. & Hayes, T. R. Meaning guides attention in real-world scene images: evidence from eye movements and meaning maps. J. Vis. 18, 10. https://doi.org/10.1167/18.6.10 (2018).
doi: 10.1167/18.6.10
pubmed: 30029216
pmcid: 6012218
Vo, M.L.-H., Boettcher, S. E. & Draschkow, D. Reading scenes: how scene grammar guides attention and aids perception in real-world environments. Curr. Opin. Psychol. 29, 205–210 (2019).
pubmed: 31051430
Veneri, G., Federighi, P., Rosini, F., Federico, A. & Rufa, A. Spike removal through multiscale wavelet and entropy analysis of ocular motor noise: a case study in patients with cerebellar disease. J. Neurosci. Methods 196, 318–326 (2011).
pubmed: 21262262
Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019 (1999).
pubmed: 10526343
Carpenter, R. Movement control: moving the mental maps. Curr. Biol. 5, 1082–1084 (1995).
pubmed: 8548271
Anton-Erxleben, K. & Carrasco, M. Attentional enhancement of spatial resolution: linking behavioural and neurophysiological evidence. Nat. Rev. Neurosci. 14, 188 (2013).
pubmed: 23422910
pmcid: 3977878
Marr, D. & Poggio, T. From Understanding Computation to Understanding Neural Circuitry (MIT Press, Cambridge, 1976).
Briggs, F. & Usrey, W. M. A fast, reciprocal pathway between the lateral geniculate nucleus and visual cortex in the macaque monkey. J. Neurosci. 27, 5431–5436 (2007).
pubmed: 17507565
pmcid: 2888515
McAlonan, K., Cavanaugh, J. & Wurtz, R. H. Guarding the gateway to cortex with attention in visual thalamus. Nature 456, 391–394 (2008).
pubmed: 18849967
pmcid: 2713033
Lee, H. & Kang, I. S. Neural algorithm for solving differential equations. J. Comput. Phys. 91, 110–131 (1990).
Lagaris, I. E., Likas, A. & Fotiadis, D. I. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9, 987–1000 (1998).
pubmed: 18255782
Tsoulos, I. G., Gavrilis, D. & Glavas, E. Solving differential equations with constructed neural networks. Neurocomputing 72, 2385–2391 (2009).
Yadav, N., Yadav, A. & Kumar, M. Neural Network Methods for Solving Differential Equations 43–100 (Springer, Dordrecht, 2015).