Hardware implementation of memristor-based artificial neural networks.

European Commission, Harnessing the economic benefits of Artificial Intelligence. Digital Transformation Monitor, no. November, 8, 2017.

Rattani, A. Reddy, N. and Derakhshani, R. “Multi-biometric Convolutional Neural Networks for Mobile User Authentication,” 2018 IEEE International Symposium on Technologies for Homeland Security, HST 2018, https://doi.org/10.1109/THS.2018.8574173 2018.

BBVA, Biometrics and machine learning: the accurate, secure way to access your bank Accessed: Jan. 21, 2024. [Online]. Available: https://www.bbva.com/en/biometrics-and-machine-learning-the-accurate-secure-way-to-access-your-bank/

Amerini, I., Li, C.-T. & Caldelli, R. Social network identification through image classification with CNN. IEEE Access 7, 35264–35273 (2019).

doi: 10.1109/ACCESS.2019.2903876

Ingle P. Y. and Kim, Y. G. “Real-time abnormal object detection for video surveillance in smart cities,” Sensors, 22, https://doi.org/10.3390/s22103862 2022.

Tan, X., Qin, T., F. Soong, and T.-Y. Liu, “A survey on neural speech synthesis,” https://doi.org/10.48550/arxiv.2106.15561 2021.

“ChatGPT: Optimizing language models for dialogue.” Accessed: Feb. 13, 2023. [Online]. Available: https://openai.com/blog/chatgpt/

Hong, T., Choi, J. A., Lim, K. & Kim, P. Enhancing personalized ads using interest category classification of SNS users based on deep neural networks. Sens. 2021, Vol. 21, Page 199, 21, 199 (2020).

McKee, S. A., Reflections on the memory wall in 2004 Computing Frontiers Conference, 162–167. https://doi.org/10.1145/977091.977115 2004.

Mehonic, A. & Kenyon, A. J. Brain-inspired computing needs a master plan. Nature 604, 255–260 (2022).

pubmed: 35418630 doi: 10.1038/s41586-021-04362-w

Zhang, C. et al. IMLBench: A machine learning benchmark suite for CPU-GPU integrated architectures. IEEE Trans. Parallel Distrib. Syst. 32, 1740–1752 (2021).

doi: 10.1109/TPDS.2020.3046870

Li, F., Ye, Y., Tian, Z. & Zhang, X. CPU versus GPU: which can perform matrix computation faster—performance comparison for basic linear algebra subprograms. Neural Comput. Appl. 31, 4353–4365 (2019).

doi: 10.1007/s00521-018-3354-z

Farabet, C. Poulet, C., Han, J. Y. and LeCun, Y. CNP: An FPGA-based processor for Convolutional Networks, FPL 09: 19th International Conference on Field Programmable Logic and Applications, 32–37, https://doi.org/10.1109/FPL.2009.5272559 2009.

Farabet, C. et al., NeuFlow: A runtime reconfigurable dataflow processor for vision, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 109–116, https://doi.org/10.1109/CVPRW.2011.5981829 2011.

Zhang, C. et al., Optimizing FPGA-based accelerator design for deep convolutional neural networks, FPGA 2015 - 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 161–170, https://doi.org/10.1145/2684746.2689060 2015.

Chakradhar, S., Sankaradas, M., Jakkula, V. and Cadambi, S. A dynamically configurable coprocessor for convolutional neural networks, Proc. Int. Symp. Comput. Archit., 247–257, https://doi.org/10.1145/1815961.1815993 2010.

Wei X. et al., Automated systolic array architecture synthesis for high throughput CNN Inference on FPGAs, Proc. Des. Autom. Conf., Part 128280, https://doi.org/10.1145/3061639.3062207 2017.

Guo, K. et al., Neural Network Accelerator Comparison. Accessed: Jan. 10, 2023. [Online]. Available: https://nicsefc.ee.tsinghua.edu.cn/projects/neural-network-accelerator.html

Jouppi, N. P. et al., In-datacenter performance analysis of a tensor processing unit. Proc. Int. Symp. Comput. Archit., Part F128643, 1–12, https://doi.org/10.1145/3079856.3080246 .2017,

AI Chip - Amazon Inferentia - AWS. Accessed: May 15, 2023. [Online]. Available: https://aws.amazon.com/machine-learning/inferentia/

Talpes, E. et al. Compute solution for Tesla’s full self-driving computer. IEEE Micro 40, 25–35 (2020).

doi: 10.1109/MM.2020.2975764

Reuther, A. et al, “AI and ML Accelerator Survey and Trends,” 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, https://doi.org/10.1109/HPEC55821.2022.9926331 .2022,

Fick, L., Skrzyniarz, S., Parikh, M., Henry, M. B. and Fick, D. “Analog matrix processor for edge AI real-time video analytics,” Dig. Tech. Pap. IEEE Int. Solid State Circuits Conf, 2022- 260–262, https://doi.org/10.1109/ISSCC42614.2022.9731773 .2022,

“Gyrfalcon Unveils Fourth AI Accelerator Chip - EE Times.” Accessed: May 16, 2023. [Online]. Available: https://www.eetimes.com/gyrfalcon-unveils-fourth-ai-accelerator-chip/

Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. and Eleftheriou, E. “Memory devices and applications for in-memory computing,” Nat. Nanotechnol. 2020 15:7, 15, 529–544, https://doi.org/10.1038/s41565-020-0655-z .

Zheng, N. and Mazumder, P. Learning in energy-efficient neuromorphic computing: algorithm and architecture co-design. Wiley-IEEE Press, Accessed: May 15, 2023. [Online]. Available: https://ieeexplore.ieee.org/book/8889858 2020.

Orchard, G. et al., “Efficient Neuromorphic Signal Processing with Loihi 2,” IEEE Workshop on Signal Processing Systems, SiPS: Design and Implementation, 2021-October, 254–259, https://doi.org/10.1109/SIPS52927.2021.00053 .2021,

“Microchips that mimic the human brain could make AI far more energy efficient | Science | AAAS.” Accessed: May 15, 2023. [Online]. Available: https://www.science.org/content/article/microchips-mimic-human-brain-could-make-ai-far-more-energy-efficient

Davies, M. et al., “Advancing neuromorphic computing with Loihi: A survey of results and outlook,” Proceedings of the IEEE, 109, 911–934, https://doi.org/10.1109/JPROC.2021.3067593 .2021,

Barnell, M., Raymond, C., Wilson, M., Isereau, D. and Cicotta, C. “Target classification in synthetic aperture radar and optical imagery using loihi neuromorphic hardware,” in 2020 IEEE High Performance Extreme Computing Conference (HPEC), IEEE, 1–6. https://doi.org/10.1109/HPEC43674.2020.9286246 .2020,

Viale, A., Marchisio, A., Martina, M., Masera, G., and Shafique, M. “CarSNN: An efficient spiking neural network for event-based autonomous cars on the Loihi Neuromorphic Research Processor,” 2021.

“Innatera Unveils Neuromorphic AI Chip to Accelerate Spiking Networks - EE Times.” Accessed: May 15, 2023. [Online]. Available: https://www.eetimes.com/innatera-unveils-neuromorphic-ai-chip-to-accelerate-spiking-networks/

Pei, J. et al. “Towards artificial general intelligence with hybrid Tianjic chip architecture,”. Nature 572, 106–111 (2019).

pubmed: 31367028 doi: 10.1038/s41586-019-1424-8

Merolla, P. A. et al. “A million spiking-neuron integrated circuit with a scalable communication network and interface,”. Science 345, 668–673 (2014).

pubmed: 25104385 doi: 10.1126/science.1254642

Adam, G. C., Khiat, A., and Prodromakis, T. “Challenges hindering memristive neuromorphic hardware from going mainstream,” Nat. Commun., 9, Nature Publishing Group, 1–4, https://doi.org/10.1038/s41467-018-07565-4 .2018.

Sung, C., Hwang, H. & Yoo, I. K. “Perspective: A review on memristive hardware for neuromorphic computation,”. J. Appl. Phys. 124, 15 (2018).

doi: 10.1063/1.5037835

Deng, L. et al. Energy consumption analysis for various memristive networks under different learning strategies,”. Phys. Lett. Sect. A: Gen. At. Solid State Phys. 380, 903–909 (2016).

doi: 10.1016/j.physleta.2015.12.024

Yu, S., Wu, Y., Jeyasingh, R., Kuzum, D. & Wong, H. S. P. “An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation,”. IEEE Trans. Electron Dev. 58, 2729–2737 (2011).

doi: 10.1109/TED.2011.2147791

Shulaker, M. M. et al. “Three-dimensional integration of nanotechnologies for computing and data storage on a single chip,”. Nature 547, 74–78 (2017).

pubmed: 28682331 doi: 10.1038/nature22994

Li, C. et al. Three-dimensional crossbar arrays of self-rectifying Si/SiO2/Si memristors. Nat. Commun. 2017 8:1 8, 1–9 (2017).

Yoon, J. H. et al. “Truly electroforming-free and low-energy memristors with preconditioned conductive tunneling paths,”. Adv. Funct. Mater. 27, 1702010 (2017).

doi: 10.1002/adfm.201702010

Choi, B. J. et al. “High-speed and low-energy nitride memristors,”. Adv. Funct. Mater. 26, 5290–5296 (2016).

doi: 10.1002/adfm.201600680

Strukov, D. B., Snider, G. S., Stewart, D. R. and Williams, R. S. “The missing memristor found,” Nature, 453, 80–83, https://doi.org/10.1038/nature06932 .

“FUJITSU SEMICONDUCTOR MEMORY SOLUTION.” Accessed: Nov. 16, 2022. [Online]. Available: https://www.fujitsu.com/jp/group/fsm/en/

“Everspin | The MRAM Company.” Accessed: Nov. 16, 2022. [Online]. Available: https://www.everspin.com/

“Yole Group.” Accessed: Nov. 16, 2022. [Online]. Available: https://www.yolegroup.com/?cn-reloaded=1

Stathopoulos, S. et al. “Multibit memory operation of metal-oxide Bi-layer memristors,”. Sci. Rep. 7, 1–7 (2017).

doi: 10.1038/s41598-017-17785-1

Wu, W. et al., “Demonstration of a multi-level μA-range bulk switching ReRAM and its application for keyword spotting,” Technical Digest - International Electron Devices Meeting, IEDM, 2022-December, 1841–1844, https://doi.org/10.1109/IEDM45625.2022.10019450 .2022,

Yang, J. et al., “Thousands of conductance levels in memristors monolithically integrated on CMOS,” https://doi.org/10.21203/RS.3.RS-1939455/V1 .2022,

Goux, L. et al., “Ultralow sub-500nA operating current high-performance TiN\Al 2O 3\HfO 2\Hf\TiN bipolar RRAM achieved through understanding-based stack-engineering,” Digest of Technical Papers - Symposium on VLSI Technology, 159–160, https://doi.org/10.1109/VLSIT.2012.6242510 2012

Li, H. et al. “Memristive crossbar arrays for storage and computing applications,”. Adv. Intell. Syst. 3, 2100017 (2021).

doi: 10.1002/aisy.202100017

Lin, P. et al. “Three-dimensional memristor circuits as complex neural networks,”. Nat. Electron. 3, 225–232 (2020).

doi: 10.1038/s41928-020-0397-9

Ishii, M. et al., “On-Chip Trainable 1.4M 6T2R PCM synaptic array with 1.6K Stochastic LIF neurons for spiking RBM,” Technical Digest - International Electron Devices Meeting, IEDM, 2019- 310–313, 2019, https://doi.org/10.1109/IEDM19573.2019.8993466 .

Li, C. et al. “Efficient and self-adaptive in-situ learning in multilayer memristor neural networks,”. Nat. Commun. 9, 1–8 (2018).

Yao, P. et al. “Fully hardware-implemented memristor convolutional neural network,”. Nature 577, 641–646 (2020).

pubmed: 31996818 doi: 10.1038/s41586-020-1942-4

Correll, J. M. et al., “An 8-bit 20.7 TOPS/W Multi-Level Cell ReRAM-based Compute Engine,” in 2022 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits), IEEE, 264–265. https://doi.org/10.1109/VLSITechnologyandCir46769.2022.9830490 .2022,

Cai, F. et al., “A fully integrated reprogrammable memristor-CMOS system for efficient multiply-accumulate operations,” Nat Electron, 2, no. July, 290–299, [Online]. Available: https://doi.org/10.1038/s41928-019-0270-x 2019.

Hung, J.-M., “An 8-Mb DC-Current-Free Binary-to-8b Precision ReRAM Nonvolatile Computing-in-Memory Macro using Time-Space-Readout with 1286.4-21.6TOPS/W for Edge-AI Devices,” in 2022 IEEE International Solid- State Circuits Conference (ISSCC), IEEE, 1–3. https://doi.org/10.1109/ISSCC42614.2022.9731715 .2022,

Xue, C.-X., “15.4 A 22nm 2Mb ReRAM Compute-in-Memory Macro with 121-28TOPS/W for Multibit MAC Computing for Tiny AI Edge Devices,” in 2020 IEEE International Solid- State Circuits Conference - (ISSCC), IEEE, 2020, 244–246.

Wan, W. et al. “A compute-in-memory chip based on resistive random-access memory,”. Nature 608, 504–512 (2022).

pubmed: 35978128 pmcid: 9385482 doi: 10.1038/s41586-022-04992-8

Yin, S., Sun, X., Yu, S. & Seo, J. S. “High-throughput in-memory computing for binary deep neural networks with monolithically integrated RRAM and 90-nm CMOS,”. IEEE Trans. Electron. Dev. 67, 4185–4192 (2020).

doi: 10.1109/TED.2020.3015178

Yan, X. et al. “Robust Ag/ZrO2/WS2/Pt Memristor for Neuromorphic Computing,”. ACS Appl Mater. Interfaces 11, 48029–48038 (2019).

pubmed: 31789034 doi: 10.1021/acsami.9b17160

Chen, Q. et al, “Improving the recognition accuracy of memristive neural networks via homogenized analog type conductance quantization,” Micromachines, 11, https://doi.org/10.3390/MI11040427 .2020,

Wang, Y. “High on/off ratio black phosphorus based memristor with ultra-thin phosphorus oxide layer,” Appl. Phys. Lett., 115, https://doi.org/10.1063/1.5115531 .2019,

Xue, F. et al. “Giant ferroelectric resistance switching controlled by a modulatory terminal for low-power neuromorphic in-memory computing,”. Adv. Mater. 33, 1–12 (2021).

Pan, W.-Q. et al. “Strategies to improve the accuracy of memristor-based convolutional neural networks,”. Trans. Electron. Dev., 67, 895–901 (2020).

doi: 10.1109/TED.2019.2963323

Seo, S. et al. “Artificial optic-neural synapse for colored and color-mixed pattern recognition,”. Nat. Commun. 9, 1–8 (2018).

doi: 10.1038/s41467-018-07572-5

Chandrasekaran, S., Simanjuntak, F. M., Saminathan, R., Panda, D. and Tseng, T. Y., “Improving linearity by introducing Al in HfO

Zhang, B. et al. “90% yield production of polymer nano-memristor for in-memory computing,”. Nat. Commun. 12, 1–11 (2021).

Feng, X. et al. “Self-selective multi-terminal memtransistor crossbar array for in-memory computing,”. ACS Nano 15, 1764–1774 (2021).

pubmed: 33443417 doi: 10.1021/acsnano.0c09441

Khaddam-Aljameh, R. et al. “HERMES-Core-A 1.59-TOPS/mm

doi: 10.1109/JSSC.2022.3140414

Narayanan, P. et al. “Fully on-chip MAC at 14 nm enabled by accurate row-wise programming of PCM-based weights and parallel vector-transport in duration-format,”. IEEE Trans. Electron Dev. 68, 6629–6636 (2021).

doi: 10.1109/TED.2021.3115993

Le Gallo, M. et al., “A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference,” 2022, Accessed: May 09, 2023. [Online]. Available: https://arxiv.org/abs/2212.02872v1

Murmann, B. “Mixed-signal computing for deep neural network inference,”. IEEE Trans. Very Large Scale Integr. VLSI Syst. 29, 3–13 (2021).

doi: 10.1109/TVLSI.2020.3020286

Yin, S., Jiang, Z., Seo, J. S. & Seok, M. “XNOR-SRAM: In-memory computing SRAM macro for binary/ternary deep neural networks,”. IEEE J. Solid-State Circuits 55, 1733–1743 (2020).

Biswas, A. & Chandrakasan, A. P. “CONV-SRAM: An energy-efficient SRAM with in-memory dot-product computation for low-power convolutional neural networks,”. IEEE J. Solid-State Circuits 54, 217–230 (2019).

doi: 10.1109/JSSC.2018.2880918

Valavi, H., Ramadge, P. J., Nestler, E. & Verma, N. “A 64-Tile 2.4-Mb In-memory-computing CNN accelerator employing charge-domain compute,”. IEEE J. Solid-State Circuits 54, 1789–1799 (2019).

doi: 10.1109/JSSC.2019.2899730

Khwa, W. S. et al. “A 65nm 4Kb algorithm-dependent computing-in-memory SRAM unit-macro with 2.3ns and 55.8TOPS/W fully parallel product-sum operation for binary DNN edge processors,”. Dig. Tech. Pap. IEEE Int. Solid State Circuits Conf. 61, 496–498 (2018).

Verma, N. et al. “In-memory computing: advances and prospects,”. IEEE Solid-State Circuits Mag. 11, 43–55 (2019).

doi: 10.1109/MSSC.2019.2922889

Diorio, C., Hasler, P., Minch, A. & Mead, C. A. “A single-transistor silicon synapse,”. IEEE Trans. Electron. Dev. 43, 1972–1980 (1996).

doi: 10.1109/16.543035

Merrikh-Bayat, F. et al. “High-performance mixed-signal neurocomputing with nanoscale floating-gate memory cell arrays,”. IEEE Trans. Neural Netw. Learn Syst. 29, 4782–4790 (2018).

pubmed: 29990267 doi: 10.1109/TNNLS.2017.2778940

Wang, P. et al. “Three-dimensional NAND flash for vector-matrix multiplication,”. IEEE Trans. Very Large Scale Integr. VLSI Syst. 27, 988–991 (2019).

doi: 10.1109/TVLSI.2018.2882194

Bavandpour, M., Sahay, S., Mahmoodi, M. R. & Strukov, D. B. “3D-aCortex: an ultra-compact energy-efficient neurocomputing platform based on commercial 3D-NAND flash memories,”. Neuromorph. Comput. Eng. 1, 014001 (2021).

doi: 10.1088/2634-4386/ac0775

Chu, M. et al. “Neuromorphic hardware system for visual pattern recognition with memristor array and CMOS neuron,”. IEEE Trans. Ind. Electron. 62, 2410–2419 (2015).

doi: 10.1109/TIE.2014.2356439

Yeo, I., Chu, M., Gi, S. G., Hwang, H. & Lee, B. G. “Stuck-at-fault tolerant schemes for memristor crossbar array-based neural networks,”. IEEE Trans. Electron Devices 66, 2937–2945 (2019).

doi: 10.1109/TED.2019.2914460

LeCun, Y., Cortes, C., and Burges, C. J. C., “MNIST handwritten digit database of handwritten digits.” Accessed: Nov. 21, 2019. [Online]. Available: http://yann.lecun.com/exdb/mnist/

Krizhevsky, A., Nair, V., and Hinton, G. “The CIFAR-10 dataset.” Accessed: Apr. 04, 2023. [Online]. Available: https://www.cs.toronto.edu/~kriz/cifar.html

Deng, J. et al., “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, 248–255.

Simonyan, K. and Zisserman, A. “Very deep convolutional networks for large-scale image recognition,” 2014.

He, K., Zhang, X., Ren, S. and Sun, J. “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 770–778. https://doi.org/10.1109/CVPR.2016.90 .2016,

Chen, P. Y., Peng, X. & Yu, S. “NeuroSim: A circuit-level macro model for benchmarking neuro-inspired architectures in online learning,”. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 37, 3067–3080 (2018).

doi: 10.1109/TCAD.2018.2789723

Wang, Q., Wang, X., Lee, S. H., Meng, F.-H. and Lu W. D., “A Deep Neural Network Accelerator Based on Tiled RRAM Architecture,” in 2019 IEEE International Electron Devices Meeting (IEDM), IEEE, 14.4.1-14.4.4. https://doi.org/10.1109/IEDM19573.2019.8993641 .2019,

Kim, H., Mahmoodi, M. R., Nili, H. & Strukov, D. B. “4K-memristor analog-grade passive crossbar circuit,”. Nat. Commun. 12, 1–11 (2021).

doi: 10.1038/s41467-021-25455-0

Inc. The Mathworks, “MATLAB.” Natick, Massachusetts, 2019.

Amirsoleimani, A. et al. “In-memory vector-matrix multiplication in monolithic complementary metal–oxide–semiconductor-memristor integrated circuits: design choices, challenges, and perspectives,”. Adv. Intell. Syst. 2, 2000115, https://doi.org/10.1002/AISY.202000115 (2020).

doi: 10.1002/AISY.202000115

Chakraborty, I. et al. “Resistive crossbars as approximate hardware building blocks for machine learning: opportunities and challenges,”. Proc. IEEE 108, 2276–2310 (2020).

doi: 10.1109/JPROC.2020.3003007

Jain, S. et al. “Neural network accelerator design with resistive crossbars: Opportunities and challenges,”. IBM J. Res Dev. 63, 6 (2019).

doi: 10.1147/JRD.2019.2947011

Ankit, A. et al. “PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM,”. IEEE Trans. Comput. 69, 1128–1142 (2020).

doi: 10.1109/TC.2020.2998456

Mochida, R. et al. “A 4M synapses integrated analog ReRAM based 66.5 TOPS/W neural-network processor with cell current controlled writing and flexible network architecture,” Digest of Technical Papers - Symposium on VLSI Technology, 175–176, Oct. 2018, 2018

Su, F. et al., “A 462GOPs/J RRAM-based nonvolatile intelligent processor for energy harvesting IoE system featuring nonvolatile logics and processing-in-memory,” Digest of Technical Papers - Symposium on VLSI Technology, C260–C261, https://doi.org/10.23919/VLSIT.2017.7998149 .2017,

Han J. and Orshansky, M. “Approximate computing: An emerging paradigm for energy-efficient design,” in 2013 18th IEEE European Test Symposium (ETS), IEEE, 1–6. https://doi.org/10.1109/ETS.2013.6569370 .2013,

Kiani, F., Yin, J., Wang, Z., Joshua Yang, J. & Xia, Q. “A fully hardware-based memristive multilayer neural network,”. Sci. Adv. 7, 4801 (2021).

doi: 10.1126/sciadv.abj4801

Gokmen, T. and Vlasov, Y. “Acceleration of deep neural network training with resistive cross-point devices: Design considerations,” Front. Neurosci., 10, no. JUL, https://doi.org/10.3389/fnins.2016.00333 .2016,

Fouda, M. E., Lee, S., Lee, J., Eltawil, A. & Kurdahi, F. “Mask technique for fast and efficient training of binary resistive crossbar arrays,”. IEEE Trans. Nanotechnol. 18, 704–716 (2019).

doi: 10.1109/TNANO.2019.2927493

Prezioso, M. et al. “Training and operation of an integrated neuromorphic network based on metal-oxide memristors,”. Nature 521, 61–64 (2015).

pubmed: 25951284 doi: 10.1038/nature14441

Hu, M. et al. “Memristor crossbar-based neuromorphic computing system: A case study,”. IEEE Trans. Neural Netw. Learn Syst. 25, 1864–1878 (2014).

pubmed: 25291739 doi: 10.1109/TNNLS.2013.2296777

Hu, M. et al., “Dot-product engine for neuromorphic computing,” in DAC ’16: Proceedings of the 53rd Annual Design Automation Conference, New York, NY, USA: Association for Computing Machinery, 1–6. https://doi.org/10.1145/2897937.2898010 .2016,

Liu, C., Hu, M., Strachan, J. P. and Li, H. H. “Rescuing memristor-based neuromorphic design with high defects,” in 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC), Institute of Electrical and Electronics Engineers Inc., https://doi.org/10.1145/3061639.3062310 .2017.

Romero-Zaliz, R., Pérez, E., Jiménez-Molinos, F., Wenger, C. & Roldán, J. B. “Study of quantized hardware deep neural networks based on resistive switching devices, conventional versus convolutional approaches,”. Electronics 10, 1–14 (2021).

doi: 10.3390/electronics10030346

Pérez, E. et al. “Advanced temperature dependent statistical analysis of forming voltage distributions for three different HfO2-based RRAM technologies,”. Solid State Electron. 176, 107961 (2021).

doi: 10.1016/j.sse.2021.107961

Pérez-Bosch Quesada, E. et al. “Toward reliable compact modeling of multilevel 1T-1R RRAM devices for neuromorphic systems,”. Electronics 10, 645 (2021).

doi: 10.3390/electronics10060645

Xia, L. et al. “Stuck-at Fault Tolerance in RRAM Computing Systems,”. IEEE J. Emerg. Sel. Top. Circuits Syst., 8, 102–115 (2018).

doi: 10.1109/JETCAS.2017.2776980

Li, C. et al., “CMOS-integrated nanoscale memristive crossbars for CNN and optimization acceleration,” 2020 IEEE International Memory Workshop, IMW 2020 - Proceedings, https://doi.org/10.1109/IMW48823.2020.9108112 .2020,

Pedretti, G. et al. “Redundancy and analog slicing for precise in-memory machine learning - Part I: Programming techniques,”. IEEE Trans. Electron. Dev. 68, 4373–4378 (2021).

doi: 10.1109/TED.2021.3095433

Pedretti, G. et al. “Redundancy and analog slicing for precise in-memory machine learning - Part II: Applications and benchmark,”. IEEE Trans. Electron. Dev. 68, 4379–4383 (2021).

doi: 10.1109/TED.2021.3095430

Wang, Z. et al. “Fully memristive neural networks for pattern classification with unsupervised learning,”. Nat. Electron. 1, 137–145 (2018).

doi: 10.1038/s41928-018-0023-2

T. Rabuske and J. Fernandes, “Charge-Sharing SAR ADCs for low-voltage low-power applications,” https://doi.org/10.1007/978-3-319-39624-8 .2017,

Kumar, P. et al. “Hybrid architecture based on two-dimensional memristor crossbar array and CMOS integrated circuit for edge computing,”. npj 2D Mater. Appl. 6, 1–10 (2022).

doi: 10.1038/s41699-021-00284-3

Krestinskaya, O., Salama, K. N. & James, A. P. “Learning in memristive neural network architectures using analog backpropagation circuits,”. IEEE Trans. Circuits Syst. I: Regul. Pap. 66, 719–732 (2019).

doi: 10.1109/TCSI.2018.2866510

Chua, L. O., Tetzlaff, R. and Slavova, A. Eds., Memristor Computing Systems. Springer International Publishing, https://doi.org/10.1007/978-3-030-90582-8 .2022.

Oh, S. et al. “Energy-efficient Mott activation neuron for full-hardware implementation of neural networks,”. Nat. Nanotechnol. 16, 680–687 (2021).

pubmed: 33737724 pmcid: 8627686 doi: 10.1038/s41565-021-00874-8

Ambrogio, S. et al. “Equivalent-accuracy accelerated neural-network training using analogue memory,”. Nature 558, 60–67 (2018).

pubmed: 29875487 doi: 10.1038/s41586-018-0180-5

Bocquet, M. et al., “In-memory and error-immune differential RRAM implementation of binarized deep neural networks,” Technical Digest - International Electron Devices Meeting, IEDM, 20.6.1-20.6.4, Jan. 2019, https://doi.org/10.1109/IEDM.2018.8614639 .2018,

Cheng, M. et al., “TIME: A Training-in-memory architecture for Memristor-based deep neural networks,” Proc. Des. Autom. Conf., Part 12828, 0–5, https://doi.org/10.1145/3061639.3062326 .2017,

Chi, P. et al., “PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory,” Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016, 27–39, https://doi.org/10.1109/ISCA.2016.13 .2016,

Krestinskaya, O., Choubey, B. & James, A. P. “Memristive GAN in Analog,”. Sci. Rep. 2020 10:1 10, 1–14 (2020).

Li, G. H. Y. et al., “All-optical ultrafast ReLU function for energy-efficient nanophotonic deep learning,” Nanophotonics, https://doi.org/10.1515/NANOPH-2022-0137/ASSET/GRAPHIC/J_NANOPH-2022-0137_FIG_007.JPG .2022,

Ando, K. et al. “BRein memory: a single-chip binary/ternary reconfigurable in-memory deep neural network accelerator achieving 1.4 TOPS at 0.6 W,”. IEEE J. Solid-State Circuits 53, 983–994 (2018).

doi: 10.1109/JSSC.2017.2778702

Price, M., Glass, J. & Chandrakasan, A. P. “A scalable speech recognizer with deep-neural-network acoustic models and voice-activated power gating,”. Dig. Tech. Pap. IEEE Int Solid State Circuits Conf. 60, 244–245 (2017).

Yin, S. et al., “A 1.06-to-5.09 TOPS/W reconfigurable hybrid-neural-network processor for deep learning applications,” IEEE Symposium on VLSI Circuits, Digest of Technical Papers, C26–C27, https://doi.org/10.23919/VLSIC.2017.8008534 .2017,

Chen, Y. H., Krishna, T., Emer, J. S. & Sze, V. “Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks,”. IEEE J. Solid-State Circuits 52, 127–138 (2017).

doi: 10.1109/JSSC.2016.2616357

Lazzaro, J., Ryckebusch, S. M., Mahowald, A. and Mead, C. A. “Winner-Take-All Networks of O(N) Complexity,” in Advances in Neural Information Processing Systems, D. Touretzky, Ed., Morgan-Kaufmann, 1988.

Andreou, A. G. et al. “Current-mode subthreshold MOS circuits for analog VLSI neural systems,”. IEEE Trans. Neural Netw. 2, 205–213 (1991).

pubmed: 18276373 doi: 10.1109/72.80331

Pouliquen, P. O., Andreou, A. G., Strohbehn, K. and Jenkins, R. E. “Associative memory integrated system for character recognition,” Midwest Symposium on Circuits and Systems, 1, 762–765, https://doi.org/10.1109/MWSCAS.1993.342935 .1993,

Starzyk, J. A. & Fang, X. “CMOS current mode winner-take-all circuit with both excitatory and inhibitory feedback,”. Electron. Lett. 29, 908–910 (1993).

doi: 10.1049/el:19930606

DeWeerth, S. P. & Morris, T. G. “CMOS current mode winner-take-all circuit with distributed hysteresis,”. Electron. Lett. 31, 1051–1053 (1995).

doi: 10.1049/el:19950729

Indiveri, G. “A current-mode hysteretic winner-take-all network, with excitatory and inhibitory coupling,”. Analog Integr. Circuits Signal Process 28, 279–291 (2001).

doi: 10.1023/A:1011208127849

Tan, B. P. & Wilson, D. M. “Semiparallel rank order filtering in analog VLSI,”. IEEE Trans. Circuits Syst. II: Analog Digit. Signal Process. 48, 198–205 (2001).

Serrano, T. & Linares-Barranco, B. “Modular current-mode high-precision winner-take-all circuit,”. Proc. - IEEE Int. Symp. Circuits Syst. 5, 557–560 (1994).

Meador, J. L. and Hylander, P. D. “Pulse Coded Winner-Take-All Networks,” Silicon Implementation of Pulse Coded Neural Networks, 79–99, https://doi.org/10.1007/978-1-4615-2680-3_5 .1994,

El-Masry, E. I., Yang, H. K. & Yakout, M. A. “Implementations of artificial neural networks using current-mode pulse width modulation technique,”. IEEE Trans. Neural Netw. 8, 532–548 (1997).

pubmed: 18255657 doi: 10.1109/72.572093

Choi, J. & Sheu, B. J. “A high-precision vlsi winner-take-all circuit for self-organizing neural networks,”. IEEE J. Solid-State Circuits 28, 576–584 (1993).

doi: 10.1109/4.229397

Yu, H. & Miyaoka, R. S. “A High-Speed and High-Precision Winner-Select-Output (WSO) ASIC,”. IEEE Trans. Nucl. Sci. 45, 772–776 (1998). PART 1.

doi: 10.1109/23.682634

Lau, K. T. and Lee, S. T. “A CMOS winner-takes-all circuit for self-organizing neural networks,” https://doi.org/10.1080/002072198134896 , 84, 131–136, 2010

He, Y. & Sánchez-Sinencio, E. “Min-net winner-take-all CMOS implementation,”. Electron Lett. 29, 1237–1239 (1993).

doi: 10.1049/el:19930827

Demosthenous, A., Smedley, S. & Taylor, J. “A CMOS analog winner-take-all network for large-scale applications,”. IEEE Trans. Circuits Syst. I: Fundam. Theory Appl. 45, 300–304 (1998).

doi: 10.1109/81.662705

Pouliquen, P. O., Andreou, A. G. & Strohbehn, K. “Winner-Takes-All associative memory: A hamming distance vector quantizer,”. Analog Integr. Circuits Signal Process. 1997 13:1 13, 211–222 (1997).

doi: 10.1023/A:1008204616413

Fish, A., Milrud, V. & Yadid-Pecht, O. “High-speed and high-precision current winner-take-all circuit,”. IEEE Trans. Circuits Syst. II: Express Briefs 52, 131–135 (2005).

Ohnhäuser, F. “Analog-Digital Converters for Industrial Applications Including an Introduction to Digital-Analog Converters,” 2015.

Pavan, S., Schreier, R.. and Temes, G. C. “Understanding Delta-Sigma Data Converters.”.

Walden, R. H. “Analog-to-digital converter survey and analysis,”. IEEE J. Sel. Areas Commun. 17, 539–550 (1999).

doi: 10.1109/49.761034

Harpe, P., Gao, H., Van Dommele, R., Cantatore, E. & Van Roermund, A. H. M. “A 0.20 mm2 3 nW signal acquisition IC for miniature sensor nodes in 65 nm CMOS. IEEE J. Solid-State Circuits 51, 240–248 (2016).

doi: 10.1109/JSSC.2015.2487270

Murmann, B. “ADC Performance Survey 1997-2022.” Accessed: Sep. 05, 2022. [Online]. Available: http://web.stanford.edu/~murmann/adcsurvey.html .

Ankit, A. et al., “PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference,” International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS, 715–731, https://doi.org/10.1145/3297858.3304049 .2019,

Ni, L. et al., “An energy-efficient matrix multiplication accelerator by distributed in-memory computing on binary RRAM crossbar,” Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC, 25-28-January-2016, 280–285, https://doi.org/10.1109/ASPDAC.2016.7428024 .2016,

Wang, X., Wu, Y. and Lu, W. D. “RRAM-enabled AI Accelerator Architecture,” in 2021 IEEE International Electron Devices Meeting (IEDM), IEEE, 12.2.1-12.2.4. https://doi.org/10.1109/IEDM19574.2021.9720543 .2021,

Xiao, T. P. et al. On the Accuracy of Analog Neural Network Inference Accelerators. [Feature],” IEEE Circuits Syst. Mag. 22, 26–48 (2022).

doi: 10.1109/MCAS.2022.3214409

Sun, X. et al, “XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks,” Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition, DATE 2018, 2018-January, 1423–1428, https://doi.org/10.23919/DATE.2018.8342235 .2018,

Zhang, W. et al. “Neuro-inspired computing chips,”. Nat. Electron. 2020 3:7 3, 371–382 (2020).

Shafiee, A. et al., “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars,” in Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016, Institute of Electrical and Electronics Engineers Inc., 14–26. https://doi.org/10.1109/ISCA.2016.12 .2016,

Fujiki, D., Mahlke, S. and Das, R. “In-memory data parallel processor,” in ACM SIGPLAN Notices, New York, NY, USA: ACM, 1–14. https://doi.org/10.1145/3173162.3173171 .2018,

Nourazar, M., Rashtchi, V., Azarpeyvand, A. & Merrikh-Bayat, F. “Memristor-based approximate matrix multiplier,”. Analog. Integr. Circuits Signal Process 93, 363–373 (2017).

doi: 10.1007/s10470-017-1029-9

Saberi, M., Lotfi, R., Mafinezhad, K. & Serdijn, W. A. “Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation ADCs,”. IEEE Trans. Circuits Syst. I: Regul. Pap. 58, 1736–1748 (2011).

doi: 10.1109/TCSI.2011.2107214

Kull, L. et al. “A 3.1 mW 8b 1.2 GS/s single-Channel asynchronous SAR ADC with alternate comparators for enhanced speed in 32 nm digital SOI CMOS,”. IEEE J. Solid-State Circuits 48, 3049–3058 (2013).

doi: 10.1109/JSSC.2013.2279571

Hagan, M., Demuth, H., Beale, M. and De Jesús, O. Neural Network Design, 2nd ed. Stillwater, OK, USA: Oklahoma State University, 2014.

Choi, S., Sheridan, P. & Lu, W. D. “Data clustering using memristor networks,”. Sci. Rep. 5, 1–10 (2015).

Khaddam-Aljameh, R. et al., “HERMES Core: A 14nm CMOS and PCM-based In-Memory Compute Core using an array of 300ps/LSB Linearized CCO-based ADCs and local digital processing,” in 2021 Symposium on VLSI Technology, Kyoto, Japan: IEEE, 978–982. Accessed: Jan. 21, 2024. [Online]. Available: https://ieeexplore.ieee.org/document/9508706

Kennedy, J. and Eberhart, R. “Particle swarm optimization,” Proceedings of ICNN’95 - International Conference on Neural Networks, 4, https://doi.org/10.1109/ICNN.1995.488968 .1942–1948,

Goldberg, D. E. & Holland, J. H. “Genetic Algorithms and machine learning. Mach. Learn. 3, 95–99 (1988).

doi: 10.1023/A:1022602019183

Kirkpatrick, S., Gelatt, C. D. & Vecchi, M. P. “Optimization by simulated annealing,”. Science 220, 671–680 (1983).

pubmed: 17813860 doi: 10.1126/science.220.4598.671

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. “Learning representations by back-propagating errors,”. Nature 323, 533–536 (1986).

doi: 10.1038/323533a0

Dennis, J. E. and Schnabel, R. B. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Society for Industrial and Applied Mathematics, https://doi.org/10.1137/1.9781611971200 .1996.

Møller, M. F. “A scaled conjugate gradient algorithm for fast supervised learning,”. Neural Netw. 6, 525–533 (1993).

doi: 10.1016/S0893-6080(05)80056-5

Powell, M. J. D. “Restart procedures for the conjugate gradient method,”. Math. Program. 12, 241–254 (1977).

doi: 10.1007/BF01593790

Fletcher, R. “Function minimization by conjugate gradients,”. Comput. J. 7, 149–154 (1964).

doi: 10.1093/comjnl/7.2.149

Marquardt, D. W. “An algorithm for least-squares estimation of nonlinear parameters,”. J. Soc. Ind. Appl. Math. 11, 431–441 (1963).

doi: 10.1137/0111030

Riedmiller, M. and Braun, H. “Direct adaptive method for faster backpropagation learning: The RPROP algorithm,” in 1993 IEEE International Conference on Neural Networks, Publ by IEEE, 586–591. https://doi.org/10.1109/icnn.1993.298623 .1993,

Battiti, R. “First- and second-order methods for learning: between steepest descent and Newton’s Method,”. Neural Comput. 4, 141–166 (1992).

doi: 10.1162/neco.1992.4.2.141

Bottou, L. “Stochastic gradient descent tricks,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7700 LECTURE NO, 421–436, https://doi.org/10.1007/978-3-642-35289-8_25/COVER .2012,

Li, M., Zhang, T., Chen, Y. and Smola, A. J. “Efficient mini-batch training for stochastic optimization,” Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 661–670, https://doi.org/10.1145/2623330.2623612 .2014,

Zamanidoost, E., Bayat, F. M., Strukov, D. and Kataeva, I. “Manhattan rule training for memristive crossbar circuit pattern classifiers,” WISP 2015 - IEEE International Symposium on Intelligent Signal Processing, Proceedings, https://doi.org/10.1109/WISP.2015.7139171 .2015,

Duchi, J., Hazan, E. & Singer, Y. “Adaptive subgradient methods for online learning and stochastic optimization,”. J. Mach. Learn. Res. 12, 2121–2159 (2011).

“Neural Networks for Machine Learning — Geoffrey Hinton – C. Cui’s Blog.” Accessed: Nov. 21, 2022. [Online]. Available: https://cuicaihao.com/neural-networks-for-machine-learning-geoffrey-hinton/

Kingma, D. P. and Ba, J. L. “Adam: A Method for Stochastic Optimization,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2014, https://doi.org/10.48550/arxiv.1412.6980 .

Zeiler, M. D. “ADADELTA: An adaptive learning rate method,” Dec. 2012, https://doi.org/10.48550/arxiv.1212.5701 .

Xiong, X. et al. “Reconfigurable logic-in-memory and multilingual artificial synapses based on 2D heterostructures,”. Adv. Funct. Mater. 30, 2–7 (2020).

doi: 10.1002/adfm.201909645

Zoppo, G., Marrone, F. & Corinto, F. “Equilibrium propagation for memristor-based recurrent neural networks,”. Front Neurosci. 14, 1–8 (2020).

doi: 10.3389/fnins.2020.00240

Alibart, F., Zamanidoost, E. & Strukov, D. B. “Pattern classification by memristive crossbar circuits using ex situ and in situ training,”. Nat. Commun. 4, 1–7 (2013).

doi: 10.1038/ncomms3072

Joshi, V. et al., “Accurate deep neural network inference using computational phase-change memory,” Nat Commun, 11, https://doi.org/10.1038/s41467-020-16108-9 .2020,

Rasch, M. J. et al., “Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators,” 2023.

Huang, H.-M., Wang, Z., Wang, T., Xiao, Y. & Guo, X. “Artificial neural networks based on memristive devices: from device to system,”. Adv. Intell. Syst. 2, 2000149 (2020).

doi: 10.1002/aisy.202000149

Nandakumar, S. R. et al., “Mixed-precision deep learning based on computational memory,” Front. Neurosci., 14, https://doi.org/10.3389/fnins.2020.00406 .2020,

Le Gallo, M. et al. “Mixed-precision in-memory computing,”. Nat. Electron. 1, 246–253 (2018).

doi: 10.1038/s41928-018-0054-8

Yao, P. et al., “Face classification using electronic synapses,” Nat. Commun., 8, May, 1–8, https://doi.org/10.1038/ncomms15199 .2017,

Papandreou, N. et al., “Programming algorithms for multilevel phase-change memory,” Proceedings - IEEE International Symposium on Circuits and Systems, 329–332, https://doi.org/10.1109/ISCAS.2011.5937569 .2011,

Milo, V. et al., “Multilevel HfO2-based RRAM devices for low-power neuromorphic networks,” APL Mater, 7, https://doi.org/10.1063/1.5108650 .2019,

Yu, S. et al., “Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect,” in Technical Digest - International Electron Devices Meeting, IEDM, Institute of Electrical and Electronics Engineers Inc., 17.3.1-17.3.4. https://doi.org/10.1109/IEDM.2015.7409718 .2015,

Woo, J. et al. “Improved synaptic behavior under identical pulses using AlOx/HfO2 bilayer RRAM array for neuromorphic systems,”. IEEE Electron. Device Lett. 37, 994–997 (2016).

doi: 10.1109/LED.2016.2582859

Xiao, S. et al. “GST-memristor-based online learning neural networks,”. Neurocomputing 272, 677–682 (2018).

doi: 10.1016/j.neucom.2017.08.014

Tian, H. et al. “A novel artificial synapse with dual modes using bilayer graphene as the bottom electrode,”. Nanoscale 9, 9275–9283 (2017).

pubmed: 28657078 doi: 10.1039/C7NR03106H

Shi, T., Yin, X. B., Yang, R. & Guo, X. “Pt/WO3/FTO memristive devices with recoverable pseudo-electroforming for time-delay switches in neuromorphic computing,”. Phys. Chem. Chem. Phys. 18, 9338–9343 (2016).

pubmed: 26996120 doi: 10.1039/C5CP07675G

Menzel, S. et al. “Origin of the ultra-nonlinear switching kinetics in oxide-based resistive switches,”. Adv. Funct. Mater. 21, 4487–4492 (2011).

doi: 10.1002/adfm.201101117

Buscarino, A., Fortuna, L., Frasca, M., Gambuzza, L. V. and Sciuto, G., “Memristive chaotic circuits based on cellular nonlinear networks,” https://doi.org/10.1142/S0218127412500708 , 22,3, 2012

Li, Y. & Ang, K.-W. “Hardware implementation of neuromorphic computing using large-scale memristor crossbar arrays,”. Adv. Intell. Syst. 3, 2000137 (2021).

doi: 10.1002/aisy.202000137

Zhu, J., Zhang, T., Yang, Y. & Huang, R. “A comprehensive review on emerging artificial neuromorphic devices,”. Appl Phys. Rev. 7, 011312 (2020).

doi: 10.1063/1.5118217

Wang, Z. et al. “Engineering incremental resistive switching in TaOx based memristors for brain-inspired computing,”. Nanoscale 8, 14015–14022 (2016).

pubmed: 27143476 doi: 10.1039/C6NR00476H

Park, S. M. et al. “Improvement of conductance modulation linearity in a Cu

pubmed: 31820625 doi: 10.1021/acsami.9b18794

Slesazeck, S. & Mikolajick, T. “Nanoscale resistive switching memory devices: a review,”. Nanotechnology 30, 352003 (2019).

pubmed: 31071689 doi: 10.1088/1361-6528/ab2084

Waser, R., Dittmann, R., Staikov, C. & Szot, K. “Redox-based resistive switching memories nanoionic mechanisms, prospects, and challenges,”. Adv. Mater. 21, 2632–2663 (2009).

pubmed: 36751064 doi: 10.1002/adma.200900375

Ielmini, D. and Waser, R. Resistive Switching. Weinheim, Germany: Wiley-VCH Verlag GmbH & Co. KGaA, 2016.

Wouters, D. J., Waser, R. & Wuttig, M. “Phase-change and redox-based resistive switching memories,”. Proc. IEEE 103, 1274–1288 (2015).

doi: 10.1109/JPROC.2015.2433311

Pan, F., Gao, S., Chen, C., Song, C. & Zeng, F. “Recent progress in resistive random access memories: Materials, switching mechanisms, and performance,”. Mater. Sci. Eng. R: Rep. 83, 1–59 (2014).

doi: 10.1016/j.mser.2014.06.002

Kim, S. et al. “Analog synaptic behavior of a silicon nitride memristor,”. ACS Appl Mater. Interfaces 9, 40420–40427 (2017).

pubmed: 29086551 doi: 10.1021/acsami.7b11191

Li, W., Sun, X., Huang, S., Jiang, H. & Yu, S. “A 40-nm MLC-RRAM compute-in-memory macro with sparsity control, On-Chip Write-verify, and temperature-independent ADC references,”. IEEE J. Solid-State Circuits 57, 2868–2877 (2022).

doi: 10.1109/JSSC.2022.3163197

Buchel, J. et al., “Gradient descent-based programming of analog in-memory computing cores,” Technical Digest - International Electron Devices Meeting, IEDM, 3311–3314, 2022, https://doi.org/10.1109/IEDM45625.2022.10019486 .2022,

Prezioso, M. et al. “Spike-timing-dependent plasticity learning of coincidence detection with passively integrated memristive circuits,”. Nat. Commun. 9, 1–8 (2018).

doi: 10.1038/s41467-018-07757-y

Park, S. et al., “Electronic system with memristive synapses for pattern recognition,” Sci. Rep., 5, https://doi.org/10.1038/srep10123 .2015,

Yu, S. et al., “Binary neural network with 16 Mb RRAM macro chip for classification and online training,” in Technical Digest - International Electron Devices Meeting, IEDM, Institute of Electrical and Electronics Engineers Inc., 16.2.1-16.2.4. https://doi.org/10.1109/IEDM.2016.7838429 .2017,

Chen, W. H. et al. “CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors,”. Nat. Electron. 2, 420–428 (2019).

doi: 10.1038/s41928-019-0288-0

Chen, W. H. et al., “A 16Mb dual-mode ReRAM macro with sub-14ns computing-in-memory and memory functions enabled by self-write termination scheme,” Technical Digest - International Electron Devices Meeting, IEDM, 28.2.1-28.2.4, 2018,

Hu, M. et al., “Memristor-based analog computation and neural network classification with a dot product engine,” Adv. Mater., 30, https://doi.org/10.1002/adma.201705914 .2018,

Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron 1, 52–59 (2018).

doi: 10.1038/s41928-017-0002-z

Paszke A. et al., “Automatic differentiation in PyTorch”.

Abadi, M. et al., “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems,” 2015.

Stimberg, M., Brette, R. and Goodman, D. F. M. “Brian 2, an intuitive and efficient neural simulator,” Elife, 8, https://doi.org/10.7554/ELIFE.47314 .2019,

Spreizer, S. et al., “NEST 3.3,” Mar. 2022, https://doi.org/10.5281/ZENODO.6368024 .

Hazan, H. et al. “BindsNET: A machine learning-oriented spiking neural networks library in python,”. Front Neuroinform 12, 89 (2018).

pubmed: 30631269 pmcid: 6315182 doi: 10.3389/fninf.2018.00089

M. Y. Lin et al., “DL-RSIM: A simulation framework to enable reliable ReRAM-based accelerators for deep learning,” IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD, https://doi.org/10.1145/3240765.3240800 .2018,

Sun, X. & Yu, S. “Impact of non-ideal characteristics of resistive synaptic devices on implementing convolutional neural networks,”. IEEE J. Emerg. Sel. Top. Circuits Syst. 9, 570–579 (2019).

doi: 10.1109/JETCAS.2019.2933148

Ma, X. et al., “Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation,” Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC, 2020-Janua, 301–306, https://doi.org/10.1109/ASP-DAC47756.2020.9045658 .2020,

Yuan, G. et al., “An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM,” Proceedings of the International Symposium on Low Power Electronics and Design, 2019, https://doi.org/10.1109/ISLPED.2019.8824944 .2019.

Rasch, M. J. et al., “A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays,” 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems, AICAS 2021, https://doi.org/10.48550/arxiv.2104.02184 .2021,

Grötker, T., “System design with SystemC,” 217, 2002.

Gajski, D. D. “SpecC: specification language and methodology,” 313, 2000.

Lee, M. K. F. et al. “A system-level simulator for RRAM-based neuromorphic computing chips,”. ACM Trans. Archit. Code Optim. (TACO) 15, 4 (2019).

BanaGozar, A. et al. “System simulation of memristor based computation in memory platforms,”. Lect. Notes Comput. Sci. (Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinforma.) 12471, 152–168 (2020).

Gai, L. and Gajski, D. “Transaction level modeling: an overview,” Hardware/Software Codesign - Proceedings of the International Workshop, 19–24, https://doi.org/10.1109/CODESS.2003.1275250 .2003,

Poremba, M. and Xie, Y. “NVMain: An architectural-level main memory simulator for emerging non-volatile memories,” Proceedings - 2012 IEEE Computer Society Annual Symposium on VLSI, ISVLSI 2012, 392–397, https://doi.org/10.1109/ISVLSI.2012.82 .2012,

Poremba, M., Zhang, T. & Xie, Y. “NVMain 2.0: A user-friendly memory simulator to model (non-)volatile memory systems,”. IEEE Comput. Archit. Lett. 14, 140–143 (2015).

doi: 10.1109/LCA.2015.2402435

Xia, L. et al. “MNSIM: Simulation platform for memristor-based neuromorphic computing system,”. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 37, 1009–1022 (2018).

Zhu, Z. et al., “MNSIM 2.0: A behavior-level modeling tool for memristor-based neuromorphic computing systems,” in Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI, Association for Computing Machinery, 83–88. https://doi.org/10.1145/3386263.3407647 .2020,

Banagozar, A. et al., “CIM-SIM: Computation in Memory SIMuIator,” in Proceedings of the 22nd International Workshop on Software and Compilers for Embedded Systems, SCOPES 2019, Association for Computing Machinery, Inc, 1–4. https://doi.org/10.1145/3323439.3323989 .2019,

Fei, X., Zhang, Y. & Zheng, W. “XB-SIM: A simulation framework for modeling and exploration of ReRAM-based CNN acceleration design,”. Tsinghua Sci. Technol. 26, 322–334 (2021).

doi: 10.26599/TST.2019.9010070

Zahedi, M. et al. “MNEMOSENE: Tile architecture and simulator for memristor-based computation-in-memory,”. ACM J. Emerg. Technol. Comput. Syst. 18, 1–24 (2022).

doi: 10.1145/3485824

Dong, X., Xu, C., Xie, Y. & Jouppi, N. P. “NVSim: A circuit-level performance, energy, and area model for emerging nonvolatile memory,”. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 31, 994–1007 (2012).

doi: 10.1109/TCAD.2012.2185930

Song, L., Qian, X., Li, H. and Chen, Y. “PipeLayer: A Pipelined ReRAM-based accelerator for deep learning,” Proceedings - International Symposium on High-Performance Computer Architecture, 541–552, https://doi.org/10.1109/HPCA.2017.55 .2017,

Imani, M. et al., “RAPIDNN: In-Memory Deep Neural Network Acceleration Framework,” 2018.

Chen, A. “A comprehensive crossbar array model with solutions for line resistance and nonlinear device characteristics,”. IEEE Trans. Electron Devices 60, 1318–1326 (2013).

doi: 10.1109/TED.2013.2246791

Aguirre, F. L. et al., “Line resistance impact in memristor-based multi layer perceptron for pattern recognition,” in 2021 IEEE 12th Latin American Symposium on Circuits and Systems, LASCAS 2021, Institute of Electrical and Electronics Engineers Inc., Feb. https://doi.org/10.1109/LASCAS51355.2021.9667132 .2021.

Aguirre, F. L. et al. “Minimization of the line resistance impact on memdiode-based simulations of multilayer perceptron arrays applied to pattern recognition,”. J. Low. Power Electron. Appl. 11, 9 (2021).

doi: 10.3390/jlpea11010009

Lee, Y. K. et al. “Matrix mapping on crossbar memory arrays with resistive interconnects and its use in in-memory compression of biosignals,”. Micromachines 10, 306 (2019).

pubmed: 31067708 pmcid: 6562796 doi: 10.3390/mi10050306

Fei, W., Yu, H., Zhang, W. & Yeo, K. S. “Design exploration of hybrid CMOS and memristor circuit by new modified nodal analysis,”. IEEE Trans. Very Large Scale Integr. VLSI Syst. 20, 1012–1025 (2012).

doi: 10.1109/TVLSI.2011.2136443

Aguirre, F. L., Pazos, S. M., Palumbo, F., Suñé, J. & Miranda, E. “Application of the quasi-static memdiode model in cross-point arrays for large dataset pattern recognition,”. IEEE Access 8, 1–1 (2020).

doi: 10.1109/ACCESS.2020.3035638

Aguirre, F. L., Pazos, S. M., Palumbo, F., Suñé, J. & Miranda, E. “SPICE simulation of RRAM-based crosspoint arrays using the dynamic memdiode model,”. Front Phys. 9, 548 (2021).

doi: 10.3389/fphy.2021.735021

Aguirre, F. L. et al. “Assessment and improvement of the pattern recognition performance of memdiode-based cross-point arrays with randomly distributed stuck-at-faults,”. Electron. 10, 2427 (2021).

doi: 10.3390/electronics10192427

Fritscher, M., Knodtel, J., Reichenbach, M. and Fey, D. “Simulating memristive systems in mixed-signal mode using commercial design tools,” 2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019, 225–228, https://doi.org/10.1109/ICECS46596.2019.8964856 .2019,

Applied Materials, “Ginestra

“TCAD - Technology Computer Aided Design (TCAD) | Synopsys.” Accessed: Jan. 20, 2023. [Online]. Available: https://www.synopsys.com/silicon/tcad.html

Krestinskaya, O., Salama, K. N. & James, A. P. “Automating analogue AI chip design with genetic search,”. Adv. Intell. Syst. 2, 2000075 (2020).

doi: 10.1002/aisy.202000075

Krestinskaya, O., Salama, K. and James, A. P. “Towards hardware optimal neural network selection with multi-objective genetic search,” Proceedings - IEEE International Symposium on Circuits and Systems, 2020, 2020, https://doi.org/10.1109/ISCAS45731.2020.9180514/VIDEO .

Guan, Z. et al., “A hardware-aware neural architecture search pareto front exploration for in-memory computing,” in 2022 IEEE 16th International Conference on Solid-State & Integrated Circuit Technology (ICSICT), IEEE, 1–4. https://doi.org/10.1109/ICSICT55466.2022.9963263 .2022,

Li, G., Mandal, S. K., Ogras, U. Y. and Marculescu, R. “FLASH: Fast neural architecture search with hardware optimization,” ACM Trans. Embed. Compu. Syst., 20, https://doi.org/10.1145/3476994 .2021,

Yuan, Z. et al. “NAS4RRAM: neural network architecture search for inference on RRAM-based accelerators,”. Sci. China Inf. Sci. 64, 160407 (2021).

doi: 10.1007/s11432-020-3245-7

Yan, Z., Juan, D.-C., Hu, X. S. and Shi, Y. “Uncertainty modeling of emerging device based computing-in-memory neural accelerators with application to neural architecture search,” in Proceedings of the 26th Asia and South Pacific Design Automation Conference, New York, NY, USA: ACM, 859–864. https://doi.org/10.1145/3394885.3431635 .2021,

Sun H. et al., “Gibbon: Efficient co-exploration of NN model and processing-in-memory architecture,” in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), IEEE, 867–872. https://doi.org/10.23919/DATE54114.2022.9774605 .2022,

Jiang, W. et al. Device-circuit-architecture co-exploration for computing-in-memory neural accelerators. IEEE Trans. Comput. 70, 595–605 (2021).

doi: 10.1109/TC.2020.2991575

Burr, G. W. et al. Experimental demonstration and tolerancing of a large-scale neural network (165 000 Synapses) using phase-change memory as the synaptic weight element. IEEE Trans. Electron Devices 62, 3498–3507 (2015).

doi: 10.1109/TED.2015.2439635

Dong, Z. et al. “Convolutional neural networks based on RRAM devices for image recognition and online learning tasks,”. IEEE Trans. Electron. Dev. 66, 793–801 (2019).

doi: 10.1109/TED.2018.2882779

Querlioz, D., Bichler, O., Dollfus, P. & Gamrat, C. “Immunity to device variations in a spiking neural network with memristive nanodevices,”. IEEE Trans. Nanotechnol. 12, 288–295 (2013).

doi: 10.1109/TNANO.2013.2250995

Guan, X., Yu, S. & Wong, H. S. P. “A SPICE compact model of metal oxide resistive switching memory with variations,”. IEEE Electron. Device Lett. 33, 1405–1407 (2012).

doi: 10.1109/LED.2012.2210856

Liang, J., Yeh, S., Simon Wong, S. & Philip Wong, H. S. “Effect of wordline/bitline scaling on the performance, energy consumption, and reliability of cross-point memory array,”. ACM J. Emerg. Technol. Comput. Syst. 9, 1–14 (2013).

doi: 10.1145/2422094.2422103

Hirtzlin, T. et al. “Digital biologically plausible implementation of binarized neural networks with differential hafnium oxide resistive memory arrays,”. Front Neurosci. 13, 1383 (2020).

pubmed: 31998059 pmcid: 6962102 doi: 10.3389/fnins.2019.01383

Xue, C. X. et al., “A 1Mb Multibit ReRAM computing-in-memory macro with 14.6ns Parallel MAC computing time for CNN based AI Edge processors,” Dig Tech Pap IEEE Int Solid State Circuits Conf, 2019-February, 388–390, https://doi.org/10.1109/ISSCC.2019.8662395 .2019,

Wu, T. F. et al., “A 43pJ/Cycle Non-Volatile Microcontroller with 4.7μs Shutdown/Wake-up Integrating 2.3-bit/Cell Resistive RAM and Resilience Techniques,” Dig Tech Pap IEEE Int Solid State Circuits Conf, 2019-February, 226–228, https://doi.org/10.1109/ISSCC.2019.8662402 .2019,

Liu, Q. et al., “A Fully Integrated Analog ReRAM based 78.4TOPS/W compute-in-memory chip with fully parallel MAC computing,” Dig. Tech. Pap. IEEE Int. Solid State Circuits Conf, 2020-February, 500–502, https://doi.org/10.1109/ISSCC19947.2020.9062953 .2020,

Xiao, T. P., Bennett, C. H., Feinberg, B., Agarwal, S. and Marinella, M. J. “Analog architectures for neural network acceleration based on non-volatile memory,” Applied Physics Reviews, 7, American Institute of Physics Inc., https://doi.org/10.1063/1.5143815 .2020.

“NVIDIA Data Center Deep Learning Product Performance | NVIDIA Developer.” Accessed: Nov. 28, 2022. [Online]. Available: https://developer.nvidia.com/deep-learning-performance-training-inference

Habana L., “Goya

Chen Y. et al., “DaDianNao: A Machine-Learning Supercomputer,” Proceedings of the Annual International Symposium on Microarchitecture, MICRO, 2015-January, no. January, 609–622, https://doi.org/10.1109/MICRO.2014.58 .2015,

Lee, J. et al. “UNPU: An energy-efficient deep neural network accelerator with fully variable weight bit precision,”. IEEE J. Solid-State Circuits 54, 173–185 (2019).

doi: 10.1109/JSSC.2018.2865489

Bankman, D., Yang, L., Moons, B., Verhelst, M. & Murmann, B. “An always-on 3.8μJ/86% CIFAR-10 mixed-signal binary CNN processor with all memory on chip in 28nm CMOS,”. Dig. Tech. Pap. IEEE Int Solid State Circuits Conf. 61, 222–224 (2018).

Nag, A. et al. “Newton: Gravitating towards the physical limits of crossbar acceleration,”. IEEE Micro 38, 41–49 (2018).

doi: 10.1109/MM.2018.053631140

Bojnordi M. N. and Ipek, E. “Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning,” Proceedings - International Symposium on High-Performance Computer Architecture, 2016-April, 1–13, https://doi.org/10.1109/HPCA.2016.7446049 .2016,

Jain, S. et al. “A heterogeneous and programmable compute-in-memory accelerator architecture for analog-AI using dense 2-D Mesh,”. IEEE Trans. Very Large Scale Integr. VLSI Syst. 31, 114–127 (2023).

doi: 10.1109/TVLSI.2022.3221390

Carnevale N. T. and Hines, M. L. “The NEURON book,” The NEURON Book, 1–457, https://doi.org/10.1017/CBO9780511541612 .2006,

Lammie, C., Xiang, W., Linares-Barranco, B. and Azghadi, M. R. “MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems,” 1–14, 2020.

Xiao, T. P., Bennett, C. H., Feinberg, B., Marinella, M. J. and Agarwal, S. “CrossSim: accuracy simulation of analog in-memory computing,” https://github.com/sandialabs/cross-sim . Accessed: Sep. 06, 2022. [Online]. Available: https://github.com/sandialabs/cross-sim

Mehonic, A., Joksas, D., Ng, W. H., Buckwell, M. & Kenyon, A. J. “Simulation of inference accuracy using realistic rram devices,”. Front. Neurosci. 13, 1–15 (2019).

doi: 10.3389/fnins.2019.00593

Zhang, Q. et al. “Sign backpropagation: An on-chip learning algorithm for analog RRAM neuromorphic computing systems,”. Neural Netw. 108, 217–223 (2018).

pubmed: 30216871 doi: 10.1016/j.neunet.2018.08.012

Yamaoka, M. “Low-power SRAM,” in Green Computing with Emerging Memory: Low-Power Computation for Social Innovation, 9781461408123, Springer New York, 59–85. https://doi.org/10.1007/978-1-4614-0812-3_4/TABLES/4 .2013,

Starzyk, J. A. and Jan, Y. W. “Voltage based winner takes all circuit for analog neural networks,” Midwest Symposium on Circuits and Systems, 1, 501–504, https://doi.org/10.1109/mwscas.1996.594211 , 1996

Hardware implementation of memristor-based artificial neural networks.

Journal

Informations de publication

Résumé

Identifiants

Types de publication

Langues

Sous-ensembles de citation

Pagination

Informations de copyright

Références

Auteurs

Classifications MeSH