Header

UZH-Logo

Maintenance Infos

Event-Driven Deep Neural Network Hardware System for Sensor Fusion


Kiselev, Ilya; Neil, Daniel; Liu, Shih-Chii (2016). Event-Driven Deep Neural Network Hardware System for Sensor Fusion. In: IEEE International Symposium on Circuits and Systems (ISCAS) 2016, Montreal, Canada, 22 May 2016 - 25 May 2016.

Abstract

This paper presents a real-time multi-modal spiking Deep Neural Network (DNN) implemented on an FPGA platform. The hardware DNN system, called n-Minitaur, demonstrates a 4-fold improvement in computational speed over the previous DNN FPGA system. The proposed system directly interfaces two different event-based sensors: a Dynamic Vision Sensor (DVS) and a Dynamic Audio Sensor (DAS). The DNN for this bimodal hardware system is trained on the MNIST digit dataset and a set of unique audio tones for each digit. When tested on the spikes produced by each sensor alone, the classification accuracy is around 70% for DVS spikes generated in response to displayed MNIST images, and 60% for DAS spikes generated in response to noisy tones. The accuracy increases to 98% when spikes from both modalities are provided simultaneously. In addition, the system shows a fast latency response of only 5ms.

Abstract

This paper presents a real-time multi-modal spiking Deep Neural Network (DNN) implemented on an FPGA platform. The hardware DNN system, called n-Minitaur, demonstrates a 4-fold improvement in computational speed over the previous DNN FPGA system. The proposed system directly interfaces two different event-based sensors: a Dynamic Vision Sensor (DVS) and a Dynamic Audio Sensor (DAS). The DNN for this bimodal hardware system is trained on the MNIST digit dataset and a set of unique audio tones for each digit. When tested on the spikes produced by each sensor alone, the classification accuracy is around 70% for DVS spikes generated in response to displayed MNIST images, and 60% for DAS spikes generated in response to noisy tones. The accuracy increases to 98% when spikes from both modalities are provided simultaneously. In addition, the system shows a fast latency response of only 5ms.

Statistics

Altmetrics

Downloads

0 downloads since deposited on 27 Jan 2017
0 downloads since 12 months

Additional indexing

Item Type:Conference or Workshop Item (Speech), refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Neuroinformatics
Dewey Decimal Classification:570 Life sciences; biology
Language:English
Event End Date:25 May 2016
Deposited On:27 Jan 2017 08:36
Last Modified:30 Aug 2017 23:19
Publisher:Proceedings of 2016 IEEE International Symposium on Circuits and Systems (ISCAS)
Series Name:IEEE Int. Symposium on Circuits and Systems (ISCAS)
Publisher DOI:https://doi.org/10.1109/ISCAS.2016.7539099

Download