Header

UZH-Logo

Maintenance Infos

Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting


Jiang, Zifan; Moryossef, Amit; Müller, Mathias; Ebling, Sarah (2023). Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting. In: Findings of the Association for Computational Linguistics: EACL 2023, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics, 1706-1724.

Abstract

This paper presents work on novel machine translation (MT) systems between spoken and signed languages, where signed languages are represented in SignWriting, a sign language writing system. Our work seeks to address the lack of out-of-the-box support for signed languages in current MT systems and is based on the SignBank dataset, which contains pairs of spoken language text and SignWriting content. We introduce novel methods to parse, factorize, decode, and evaluate SignWriting, leveraging ideas from neural factored MT. In a bilingual setup—-translating from American Sign Language to (American) English—-our method achieves over 30 BLEU, while in two multilingual setups—-translating in both directions between spoken languages and signed languages—-we achieve over 20 BLEU. We find that common MT techniques used to improve spoken language translation similarly affect the performance of sign language translation. These findings validate our use of an intermediate text representation for signed languages to include them in natural language processing research.

Abstract

This paper presents work on novel machine translation (MT) systems between spoken and signed languages, where signed languages are represented in SignWriting, a sign language writing system. Our work seeks to address the lack of out-of-the-box support for signed languages in current MT systems and is based on the SignBank dataset, which contains pairs of spoken language text and SignWriting content. We introduce novel methods to parse, factorize, decode, and evaluate SignWriting, leveraging ideas from neural factored MT. In a bilingual setup—-translating from American Sign Language to (American) English—-our method achieves over 30 BLEU, while in two multilingual setups—-translating in both directions between spoken languages and signed languages—-we achieve over 20 BLEU. We find that common MT techniques used to improve spoken language translation similarly affect the performance of sign language translation. These findings validate our use of an intermediate text representation for signed languages to include them in natural language processing research.

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

16 downloads since deposited on 30 Oct 2023
16 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
06 Faculty of Arts > Zurich Center for Linguistics
08 Research Priority Programs > Digital Society Initiative
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Scopus Subject Areas:Physical Sciences > Computational Theory and Mathematics
Physical Sciences > Software
Social Sciences & Humanities > Linguistics and Language
Language:English
Event End Date:May 2023
Deposited On:30 Oct 2023 13:25
Last Modified:30 Mar 2024 04:46
Publisher:Association for Computational Linguistics
OA Status:Green
Publisher DOI:https://doi.org/10.18653/v1/2023.findings-eacl.127
  • Content: Published Version
  • Language: English