Header

UZH-Logo

Maintenance Infos

Incremental Learning of Hand Symbols Using Event-Based Cameras


Lungu, Iulia Alexandra; Liu, Shih-Chii; Delbruck, Tobi (2019). Incremental Learning of Hand Symbols Using Event-Based Cameras. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(4):690-696.

Abstract

Conventional cameras create redundant output especially when the frame rate is high. Dynamic vision sensors (DVSs), on the other hand, generate asynchronous and sparse brightness change events only when an object in the field of view is in motion. Such event-based output can be processed as a 1D time sequence, or it can be converted to 2D frames that resemble conventional camera frames. Frames created, e.g., by accumulating a fixed number of events, can be used as input for conventional deep learning algorithms, thus upgrading existing computer vision pipelines through low-power, low-redundancy sensors. This paper describes a hand symbol recognition system that can quickly be trained to incrementally learn new symbols recorded with an event-based camera, without forgetting previously learned classes. By using the iCaRL incremental learning algorithm, we show that we can learn up to 16 new symbols using only 4000 samples for each symbol and achieving a final symbol accuracy of over 80%. The system achieves latency of under 0.5s and training requires 3 minutes for 5 epochs on an NVIDIA 1080TI GPU.

Abstract

Conventional cameras create redundant output especially when the frame rate is high. Dynamic vision sensors (DVSs), on the other hand, generate asynchronous and sparse brightness change events only when an object in the field of view is in motion. Such event-based output can be processed as a 1D time sequence, or it can be converted to 2D frames that resemble conventional camera frames. Frames created, e.g., by accumulating a fixed number of events, can be used as input for conventional deep learning algorithms, thus upgrading existing computer vision pipelines through low-power, low-redundancy sensors. This paper describes a hand symbol recognition system that can quickly be trained to incrementally learn new symbols recorded with an event-based camera, without forgetting previously learned classes. By using the iCaRL incremental learning algorithm, we show that we can learn up to 16 new symbols using only 4000 samples for each symbol and achieving a final symbol accuracy of over 80%. The system achieves latency of under 0.5s and training requires 3 minutes for 5 epochs on an NVIDIA 1080TI GPU.

Statistics

Citations

Altmetrics

Downloads

0 downloads since deposited on 14 Feb 2020
0 downloads since 12 months

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Neuroinformatics
Dewey Decimal Classification:570 Life sciences; biology
Scopus Subject Areas:Physical Sciences > Electrical and Electronic Engineering
Uncontrolled Keywords:Electrical and Electronic Engineering
Language:English
Date:1 December 2019
Deposited On:14 Feb 2020 10:00
Last Modified:29 Jul 2020 14:15
Publisher:Institute of Electrical and Electronics Engineers
ISSN:2156-3357
OA Status:Closed
Publisher DOI:https://doi.org/10.1109/jetcas.2019.2951062

Download

Closed Access: Download allowed only for UZH members