Header

UZH-Logo

Maintenance Infos

Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition


Moryossef, Amit; Tsochantaridis, Ioannis; Dinn, Joe; Camgöz, Necati Cihan; Bowden, Richard; Jiang, Tao; Rios, Annette; Müller, Mathias; Ebling, Sarah (2021). Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Nashville, TN, USA, 20 June 2021 - 25 June 2021.

Abstract

Signed languages are visual languages produced by the movement of the hands, face, and body. In this paper, we evaluate representations based on skeleton poses, as these are explainable, person-independent, privacy-preserving, low-dimensional representations. Basically, skeletal representations generalize over an individual's appearance and background, allowing us to focus on the recognition of motion. But how much information is lost by the skeletal representation? We perform two independent studies using two state-of-the-art pose estimation systems. We analyze the applicability of the pose estimation systems to sign language recognition by evaluating the failure cases of the recognition models. Importantly, this allows us to characterize the current limitations of skeletal pose estimation approaches in sign language recognition.

Abstract

Signed languages are visual languages produced by the movement of the hands, face, and body. In this paper, we evaluate representations based on skeleton poses, as these are explainable, person-independent, privacy-preserving, low-dimensional representations. Basically, skeletal representations generalize over an individual's appearance and background, allowing us to focus on the recognition of motion. But how much information is lost by the skeletal representation? We perform two independent studies using two state-of-the-art pose estimation systems. We analyze the applicability of the pose estimation systems to sign language recognition by evaluating the failure cases of the recognition models. Importantly, this allows us to characterize the current limitations of skeletal pose estimation approaches in sign language recognition.

Statistics

Citations

Dimensions.ai Metrics
16 citations in Web of Science®
26 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

36 downloads since deposited on 25 May 2021
10 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Speech), not_refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
06 Faculty of Arts > Zurich Center for Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:25 June 2021
Deposited On:25 May 2021 14:53
Last Modified:13 Oct 2023 13:36
Number:10166v1
Additional Information:2021 ChaLearn Looking at People Sign Language Recognition in the Wild Workshop at CVPR
OA Status:Green
Publisher DOI:https://doi.org/10.1109/CVPRW53098.2021.00382
Official URL:https://openaccess.thecvf.com/content/CVPR2021W/ChaLearn/papers/Moryossef_Evaluating_the_Immediate_Applicability_of_Pose_Estimation_for_Sign_Language_CVPRW_2021_paper.pdf
Related URLs:https://ieeexplore.ieee.org/xpl/conhome/9577055/proceeding (Organisation)
Project Information:
  • Content: Published Version
  • Language: English
  • Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)