Header

UZH-Logo

Maintenance Infos

PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasing


Moeys, Diederik Paul; Neil, Daniel; Corradi, Federico; Kerr, Emmett; Vance, Philip; Das, Gautham; Coleman, Sonya A; McGinnity, Thomas M; Kerr, Dermot; Delbruck, Tobi (2018). PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasing. In: 4th International Conference on Event-Based Control, Communication and Signal Processing (EBCCSP) 2018, Perpignan, 27 June 2018 - 29 June 2018.

Abstract

Machine vision systems using convolutional neural networks (CNNs) for robotic applications are increasingly being developed. Conventional vision CNNs are driven by camera frames at constant sample rate, thus achieving a fixed latency and power consumption tradeoff. This paper describes further work on the first experiments of a closed-loop robotic system integrating a CNN together with a Dynamic and Active Pixel Vision Sensor (DAVIS) in a predator/prey scenario. The DAVIS, mounted on the predator Summit XL robot, produces frames at a fixed 15 Hz frame-rate and Dynamic Vision Sensor (DVS) histograms containing 5k ON and OFF events at a variable frame-rate ranging from 15-500 Hz depending on the robot speeds. In contrast to conventional frame-based systems, the latency and processing cost depends on the rate of change of the image. The CNN is trained offline on the 1.25h labeled dataset to recognize the position and size of the prey robot, in the field of view of the predator. During inference, combining the ten output classes of the CNN allows extracting the analog position vector of the prey relative to the predator with a mean 8.7% error in angular estimation. The system is compatible with conventional deep learning technology, but achieves a variable latency-power tradeoff that adapts automatically to the dynamics. Finally, investigations on the robustness of the algorithm, a human performance comparison and a deconvolution analysis are also explored.

Abstract

Machine vision systems using convolutional neural networks (CNNs) for robotic applications are increasingly being developed. Conventional vision CNNs are driven by camera frames at constant sample rate, thus achieving a fixed latency and power consumption tradeoff. This paper describes further work on the first experiments of a closed-loop robotic system integrating a CNN together with a Dynamic and Active Pixel Vision Sensor (DAVIS) in a predator/prey scenario. The DAVIS, mounted on the predator Summit XL robot, produces frames at a fixed 15 Hz frame-rate and Dynamic Vision Sensor (DVS) histograms containing 5k ON and OFF events at a variable frame-rate ranging from 15-500 Hz depending on the robot speeds. In contrast to conventional frame-based systems, the latency and processing cost depends on the rate of change of the image. The CNN is trained offline on the 1.25h labeled dataset to recognize the position and size of the prey robot, in the field of view of the predator. During inference, combining the ten output classes of the CNN allows extracting the analog position vector of the prey relative to the predator with a mean 8.7% error in angular estimation. The system is compatible with conventional deep learning technology, but achieves a variable latency-power tradeoff that adapts automatically to the dynamics. Finally, investigations on the robustness of the algorithm, a human performance comparison and a deconvolution analysis are also explored.

Statistics

Downloads

2 downloads since deposited on 12 Mar 2019
2 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Speech), not_refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Neuroinformatics
Dewey Decimal Classification:570 Life sciences; biology
Language:English
Event End Date:29 June 2018
Deposited On:12 Mar 2019 13:36
Last Modified:25 Sep 2019 00:27
Publisher:arxiv
OA Status:Green
Free access at:Publisher DOI. An embargo period may apply.
Official URL:https://arxiv.org/abs/1807.03128

Download

Green Open Access

Download PDF  'PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasing'.
Preview
Content: Published Version
Filetype: PDF
Size: 598kB