Header

UZH-Logo

Maintenance Infos

Synaptic metaplasticity with multi-level memristive devices


D’Agostino, S; Moro, F; Hirtzlin, T; Arcamone, J; Castellani, N; Querlioz, D; Payvand, Melika; Vianello, Elisa (2023). Synaptic metaplasticity with multi-level memristive devices. In: 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), Hangzhou, China, 11 June 2023 - 13 June 2023, Institute of Electrical and Electronics Engineers.

Abstract

Deep learning has made remarkable progress in various tasks, surpassing human performance in some cases. However, one drawback of neural networks is catastrophic forgetting, where a network trained on one task forgets the solution when learning a new one. To address this issue, recent works have proposed solutions based on Binarized Neural Networks (BNNs) incorporating metaplasticity. In this work, we extend this solution to quantized neural networks (QNNs) and present a memristor-based hardware solution for implementing metaplasticity during both inference and training. We propose a hardware architecture that integrates quantized weights in memristor devices programmed in an analog multi-level fashion with a digital processing unit for high-precision metaplastic storage. We validated our approach using a combined software framework and memristor based crossbar array for in-memory computing fabricated in 130 nm CMOS technology. Our experimental results show that a two-layer perceptron achieves 97% and 86% accuracy on consecutive training of MNIST and Fashion-MNIST, equal to software baseline. This result demonstrates immunity to catastrophic forgetting and the resilience to analog device imperfections of the proposed solution. Moreover, our architecture is compatible with the memristor limited endurance and has a 15× reduction in memory footprint compared to the binarized neural network case.

Abstract

Deep learning has made remarkable progress in various tasks, surpassing human performance in some cases. However, one drawback of neural networks is catastrophic forgetting, where a network trained on one task forgets the solution when learning a new one. To address this issue, recent works have proposed solutions based on Binarized Neural Networks (BNNs) incorporating metaplasticity. In this work, we extend this solution to quantized neural networks (QNNs) and present a memristor-based hardware solution for implementing metaplasticity during both inference and training. We propose a hardware architecture that integrates quantized weights in memristor devices programmed in an analog multi-level fashion with a digital processing unit for high-precision metaplastic storage. We validated our approach using a combined software framework and memristor based crossbar array for in-memory computing fabricated in 130 nm CMOS technology. Our experimental results show that a two-layer perceptron achieves 97% and 86% accuracy on consecutive training of MNIST and Fashion-MNIST, equal to software baseline. This result demonstrates immunity to catastrophic forgetting and the resilience to analog device imperfections of the proposed solution. Moreover, our architecture is compatible with the memristor limited endurance and has a 15× reduction in memory footprint compared to the binarized neural network case.

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

2 downloads since deposited on 31 Jan 2024
2 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Neuroinformatics
Dewey Decimal Classification:570 Life sciences; biology
Scopus Subject Areas:Physical Sciences > Artificial Intelligence
Physical Sciences > Computer Vision and Pattern Recognition
Physical Sciences > Hardware and Architecture
Physical Sciences > Information Systems
Physical Sciences > Electrical and Electronic Engineering
Language:English
Event End Date:13 June 2023
Deposited On:31 Jan 2024 10:02
Last Modified:01 Feb 2024 22:27
Publisher:Institute of Electrical and Electronics Engineers
Series Name:IEEE International Conference on Artificial Intelligence Circuits and Systems
ISSN:2834-9830
OA Status:Green
Publisher DOI:https://doi.org/10.1109/aicas57966.2023.10168563
Project Information:
  • : FunderEuropean Research Council
  • : Grant ID
  • : Project Title
  • : FunderHorizon Europe
  • : Grant ID
  • : Project Title
  • Content: Submitted Version
  • Language: English