Header

UZH-Logo

Maintenance Infos

End-to-End Learning of Representations for Asynchronous Event-Based Data


Gehrig, Daniel; Loquercio, Antonio; Derpanis, Konstantinos; Scaramuzza, Davide (2019). End-to-End Learning of Representations for Asynchronous Event-Based Data. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 27 November 2019 - 2 December 2019, 5632-5642.

Abstract

Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events”. They have appealing advantages over frame based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatio-temporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations by means of strictly differentiable operations. Our framework comes with two main advantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.

Abstract

Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events”. They have appealing advantages over frame based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatio-temporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations by means of strictly differentiable operations. Our framework comes with two main advantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.

Statistics

Citations

Dimensions.ai Metrics
5 citations in Web of Science®
16 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

3 downloads since deposited on 26 Jan 2021
3 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Scopus Subject Areas:Physical Sciences > Software
Physical Sciences > Computer Vision and Pattern Recognition
Language:English
Event End Date:2 December 2019
Deposited On:26 Jan 2021 10:07
Last Modified:27 Jan 2021 21:01
Publisher:IEEE
ISBN:978-1-7281-4803-8
OA Status:Green
Publisher DOI:https://doi.org/10.1109/iccv.2019.00573
Other Identification Number:merlin-id:20298

Download

Green Open Access

Download PDF  'End-to-End Learning of Representations for Asynchronous Event-Based Data'.
Preview
Content: Accepted Version
Filetype: PDF
Size: 790kB
View at publisher