Multifunctional human visual pathway-replicated hardware based on 2D materials.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
05 Oct 2024
05 Oct 2024
Historique:
received:
22
05
2024
accepted:
26
09
2024
medline:
6
10
2024
pubmed:
6
10
2024
entrez:
5
10
2024
Statut:
epublish
Résumé
Artificial visual system empowered by 2D materials-based hardware simulates the functionalities of the human visual system, leading the forefront of artificial intelligence vision. However, retina-mimicked hardware that has not yet fully emulated the neural circuits of visual pathways is restricted from realizing more complex and special functions. In this work, we proposed a human visual pathway-replicated hardware that consists of crossbar arrays with split floating gate 2D tungsten diselenide (WSe
Identifiants
pubmed: 39369011
doi: 10.1038/s41467-024-52982-3
pii: 10.1038/s41467-024-52982-3
doi:
Substances chimiques
Tungsten
V9306CXO6G
Selenium
H6241UJ22B
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
8650Subventions
Organisme : National Natural Science Foundation of China (National Science Foundation of China)
ID : 62222404, 62304084 and 92248304
Informations de copyright
© 2024. The Author(s).
Références
Luo, L. Architectures of neuronal circuits. Science 373, eabg7285 (2021).
doi: 10.1126/science.abg7285
Schuman, C. D. et al. Opportunities for neuromorphic computing algorithms and applications. Nat. Comput. Sci. 2, 10–19 (2022).
doi: 10.1038/s43588-021-00184-y
Lai, C., Tanaka, S., Harris, T. D. & Lee, A. K. Volitional activation of remote place representations with a hippocampal brain–machine interface. Science 382, 566–573 (2023).
doi: 10.1126/science.adh5206
Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).
doi: 10.1038/s41928-020-00501-9
Suárez, L. E., Richards, B. A., Lajoie, G. & Misic, B. Learning function from structure in neuromorphic networks. Nat. Mach. Intell. 3, 771–786 (2021).
doi: 10.1038/s42256-021-00376-1
Wang, T. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photonics 17, 408–415 (2023).
doi: 10.1038/s41566-023-01170-8
Langenegger, J. et al. In-memory factorization of holographic perceptual representations. Nat. Nanotechnol. 18, 479–485 (2023).
doi: 10.1038/s41565-023-01357-8
Leiserson, C. E. et al. There’s plenty of room at the top: What will drive computer performance after Moore’s law? Science 368, aam9744 (2020).
doi: 10.1126/science.aam9744
Indiveri, G. et al. Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 9202 (2011).
doi: 10.3389/fnins.2011.00073
Thakur, C. S. et al. Large-scale neuromorphic spiking array processors: A quest to mimic the brain. Front. Neurosci. 12, 353526 (2018).
doi: 10.3389/fnins.2018.00891
Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).
doi: 10.1038/s41565-019-0501-3
Choi, C. et al. Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system. Nat. Commun. 11, 1–9 (2020).
doi: 10.1038/s41467-020-19806-6
Zhang, H. T. et al. Reconfigurable perovskite nickelate electronics for artificial intelligence. Science 375, 533–539 (2022).
doi: 10.1126/science.abj7943
Lanza, M. et al. Memristive technologies for data storage, computation, encryption, and radio-frequency communication. Science 376, abj9979 (2022).
doi: 10.1126/science.abj9979
Li, X. et al. Power-efficient neural network with artificial dendrites. Nat. Nanotechnol. 15, 776–782 (2020).
doi: 10.1038/s41565-020-0722-5
Migliato Marega, G. et al. Logic-in-memory based on an atomically thin semiconductor. Nature 587, 72–77 (2020).
doi: 10.1038/s41586-020-2861-0
Xu, Z. et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384, 202–209 (2024).
doi: 10.1126/science.adl1203
Zhang, W. et al. Edge learning using a fully integrated neuro-inspired memristor chip. Science 381, 1205–1211 (2023).
doi: 10.1126/science.ade3483
Novoselov, K. S., Mishchenko, A., Carvalho, A. & Castro Neto, A. H. 2D materials and van der Waals heterostructures. Science 353, aac9439 (2016).
doi: 10.1126/science.aac9439
Luo, P. et al. Molybdenum disulfide transistors with enlarged van der Waals gaps at their dielectric interface via oxygen accumulation. Nat. Electron. 5, 849–858 (2022).
doi: 10.1038/s41928-022-00877-w
Tong, L. et al. 2D materials-based homogeneous transistor-memory architecture for neuromorphic hardware. Science 373, 1353–1358 (2021).
doi: 10.1126/science.abg3161
Lee, S., Peng, R., Wu, C. & Li, M. Programmable black phosphorus image sensor for broadband optoelectronic edge computing. Nat. Commun. 13, 1–8 (2022).
Zhou, Y. et al. Computational event-driven vision sensors for in-sensor spiking neural networks. Nat. Electron. 6, 870–878 (2023).
doi: 10.1038/s41928-023-01055-2
Liao, F. et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat. Electron. 5, 84–91 (2022).
doi: 10.1038/s41928-022-00713-1
Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).
doi: 10.1038/s41586-020-2038-x
Pan, X. et al. Parallel perception of visual motion using light-tunable memory matrix. Sci. Adv. 9, adi4083 (2023).
doi: 10.1126/sciadv.adi4083
Seo, S. et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat. Commun. 9, 1–8 (2018).
doi: 10.1038/s41467-018-07572-5
Zhang, G. X. et al. Broadband sensory networks with locally stored responsivities for neuromorphic machine vision. Sci. Adv. 9, adi5104 (2023).
doi: 10.1126/sciadv.adi5104
Pi, L. et al. Broadband convolutional processing using band-alignment-tunable heterostructures. Nat. Electron. 5, 248–254 (2022).
doi: 10.1038/s41928-022-00747-5
Wang, F. et al. A two-dimensional mid-infrared optoelectronic retina enabling simultaneous perception and encoding. Nat. Commun. 14, 1–8 (2023).
doi: 10.1038/s41467-023-43859-y
Wang, C. Y. et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci. Adv. 6, aba6173 (2020).
doi: 10.1126/sciadv.aba6173
Zhang, Z. et al. All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat. Nanotechnol. 17, 27–32 (2022).
doi: 10.1038/s41565-021-01003-1
Huang, P. Y. et al. Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction. Nat. Commun. 14, 1–9 (2023).
Lee, D. et al. In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing. Nat. Commun. 13, 1–9 (2022).
Kravitz, D. J., Saleem, K. S., Baker, C. I., Ungerleider, L. G. & Mishkin, M. The ventral visual pathway: An expanded neural framework for the processing of object quality. Trends Cogn. Sci. 17, 26–49 (2013).
doi: 10.1016/j.tics.2012.10.011
Tamietto, M. & Morrone, M. C. Visual plasticity: Blindsight bridges anatomy and function in the visual system. Curr. Biol. 26, R70–R73 (2016).
doi: 10.1016/j.cub.2015.11.026
Baden, T., Euler, T. & Berens, P. Understanding the retinal basis of vision across species. Nat. Rev. Neurosci. 21, 5–20 (2020).
doi: 10.1038/s41583-019-0242-1
Van Essen, D. C., Anderson, C. H. & Felleman, D. J. Information processing in the primate visual system: An integrated systems perspective. Science 255, 419–423 (1992).
doi: 10.1126/science.1734518
Joesch, M. & Meister, M. A neuronal circuit for colour vision based on rod-cone opponency. Nature 532, 236–239 (2016).
doi: 10.1038/nature17158
Solomon, S. G. & Lennie, P. The machinery of colour vision. Nat. Rev. Neurosci. 8, 276–286 (2007).
doi: 10.1038/nrn2094
Sun, W., Tan, Z., Mensh, B. D. & Ji, N. Thalamus provides layer 4 of primary visual cortex with orientation- and direction-tuned inputs. Nat. Neurosci. 19, 308–315 (2016).
doi: 10.1038/nn.4196
Chen, X., Wang, F., Fernandez, E. & Roelfsema, P. R. Shape perception via a high-channel-count neuroprosthesis in monkey visual cortex. Science 370, 1191–1196 (2020).
doi: 10.1126/science.abd7435
Ding, H., Smith, R. G., Poleg-Polsky, A., Diamond, J. S. & Briggman, K. L. Species-specific wiring for direction selectivity in the mammalian retina. Nature 535, 105–110 (2016).
doi: 10.1038/nature18609
Lien, A. D. & Scanziani, M. Cortical direction selectivity emerges at convergence of thalamic synapses. Nature 558, 80–86 (2018).
doi: 10.1038/s41586-018-0148-5
Liu, B., Hong, A., Rieke, F. & Manookin, M. B. Predictive encoding of motion begins in the primate retina. Nat. Neurosci. 24, 1280–1291 (2021).
doi: 10.1038/s41593-021-00899-1
Strauss, S. et al. Center-surround interactions underlie bipolar cell motion sensitivity in the mouse retina. Nat. Commun. 13, 1–18 (2022).
doi: 10.1038/s41467-022-32762-7
Brincat, S. L. & Connor, C. E. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat. Neurosci. 7, 880–886 (2004).
doi: 10.1038/nn1278
Priebe, N. J. & Ferster, D. Inhibition, spike threshold, and stimulus selectivity in primary visual cortex. Neuron 57, 482–497 (2008).
doi: 10.1016/j.neuron.2008.02.005
Jayachandran, D. et al. Three-dimensional integration of two-dimensional field-effect transistors. Nature 625, 276–281 (2024).
doi: 10.1038/s41586-023-06860-5
Wang, S. et al. Two-dimensional devices and integration towards the silicon lines. Nat. Mater. 21, 1225–1239 (2022).
doi: 10.1038/s41563-022-01383-2
Bellmund, J. L. S., Gärdenfors, P., Moser, E. I. & Doeller, C. F. Navigating cognition: Spatial codes for human thinking. Science 362, aat6766 (2018).
doi: 10.1126/science.aat6766
Pei, J. et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572, 106–111 (2019).
doi: 10.1038/s41586-019-1424-8
Gao, S. B., Yang, K. F., Li, C. Y. & Li, Y. J. Color constancy using double-opponency. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1973–1985 (2015).
doi: 10.1109/TPAMI.2015.2396053
Gijsenij, A., Gevers, T. & Van De Weijer, J. Computational color constancy: Survey and experiments. IEEE Trans. Image Process. 20, 2475–2489 (2011).
doi: 10.1109/TIP.2011.2118224
Luo, X. et al. A bionic self-driven retinomorphic eye with ionogel photosynaptic retina. Nat. Commun. 15, 1–9 (2024).
Yang, R. et al. Assessment of visual function in blind mice and monkeys with subretinally implanted nanowire arrays as artificial photoreceptors. Nat. Biomed. Eng. 1–22 https://doi.org/10.1038/s41551-023-01137-8 (2023).
Beauchamp, M. S. et al. Dynamic stimulation of visual cortex produces form vision in sighted and blind humans. Cell 181, 774–783 (2020).
doi: 10.1016/j.cell.2020.04.033
Peng, Z. Source data of figures in the article “Multifunctional human visual pathway-replicated hardware based on 2D materials”. figshare, https://doi.org/10.6084/m9.figshare.26234561.v2 (2024).