Energy-Efficient Recurrent and Fully-Connected Neural Network Training with Bio-Inspired Temporal Sparsity

Abstract

The past decade has seen a resurgence of Deep Learning (DL) driven by the rapid advancement of computational power and the explosion of data. The massive parallel processing capacities of the Graphics Processing Units (GPU) and Application Specific Integrated Circuit (ASIC) clusters on the cloud have enabled training of large-scale Deep Neural Network (DNN) models, but they consume a considerable amount of power and risk leaking private data. Local learning on edge devices is becoming increasingly important in privacy-sensitive applic Show more