Navigation auf zora.uzh.ch

Search ZORA

ZORA (Zurich Open Repository and Archive)

Revisiting End-to-End Speech-to-Text Translation From Scratch

Zhang, Biao; Haddow, Barry; Sennrich, Rico (2022). Revisiting End-to-End Speech-to-Text Translation From Scratch. In: Proceedings of the 39th International Conference on Machine Learning, Baltimore, USA, 17 July 2022 - 23 July 2022. PMLR, 26193-26205.

Abstract

End-to-end (E2E) speech-to-text translation (ST) often depends on pretraining its encoder and/or decoder using source transcripts via speech recognition or text translation tasks, without which translation performance drops substantially. However, transcripts are not always available, and how significant such pretraining is for E2E ST has rarely been studied in the literature. In this paper, we revisit this question and explore the extent to which the quality of E2E ST trained on speech-translation pairs alone can be improved. We reexamine several techniques proven beneficial to ST previously, and offer a set of best practices that biases a Transformer-based E2E ST system toward training from scratch. Besides, we propose parameterized distance penalty to facilitate the modeling of locality in the self-attention model for speech. On four benchmarks covering 23 languages, our experiments show that, without using any transcripts or pretraining, the proposed system reaches and even outperforms previous studies adopting pretraining, although the gap remains in (extremely) low-resource settings. Finally, we discuss neural acoustic feature modeling, where a neural model is designed to extract acoustic features from raw speech signals directly, with the goal to simplify inductive biases and add freedom to the model in describing speech. For the first time, we demonstrate its feasibility and show encouraging results on ST tasks.

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:23 July 2022
Deposited On:12 Dec 2022 15:13
Last Modified:27 Sep 2023 07:15
Publisher:PMLR
OA Status:Green
Free access at:Official URL. An embargo period may apply.
Official URL:https://proceedings.mlr.press/v162/zhang22i.html
Download PDF  'Revisiting End-to-End Speech-to-Text Translation From Scratch'.
Preview
  • Content: Published Version
  • Language: English
  • Publisher License

Metadata Export

Statistics

Citations

Downloads

15 downloads since deposited on 12 Dec 2022
7 downloads since 12 months
Detailed statistics

Authors, Affiliations, Collaborations

Similar Publications