Header

UZH-Logo

Maintenance Infos

Subword segmentation and a single bridge language affect zero-shot neural machine translation


Rios, Annette; Müller, Mathias; Sennrich, Rico (2020). Subword segmentation and a single bridge language affect zero-shot neural machine translation. In: 5th Conference on Machine Translation (WMT), Virtual, 19 November 2020 - 20 November 2020, 526-535.

Abstract

Zero-shot neural machine translation is an attractive goal because of the high cost of obtaining data and building translation systems for new translation directions.
However, previous papers have reported mixed success in zero-shot translation. It is hard to predict in which settings it will be effective, and what limits performance compared to a fully supervised system.
In this paper, we investigate zero-shot performance of a multilingual EN<->{FR,CS,DE,FI} system trained on WMT data.
We find that zero-shot performance is highly unstable and can vary by more than 6 BLEU between training runs, making it difficult to reliably track improvements.
We observe a bias towards copying the source in zero-shot translation, and investigate how the choice of subword segmentation affects this bias. We find that language-specific subword segmentation results in less subword copying at training time, and leads to better zero-shot performance compared to jointly trained segmentation.
A recent trend in multilingual models is to not train on parallel data between all language pairs, but have a single {\em bridge} language, e.g.\ English.
We find that this negatively affects zero-shot translation and leads to a failure mode where the model ignores the language tag and instead produces English output in zero-shot directions.
We show that this bias towards English can be effectively reduced with even a small amount of parallel data in some of the non-English pairs.

Abstract

Zero-shot neural machine translation is an attractive goal because of the high cost of obtaining data and building translation systems for new translation directions.
However, previous papers have reported mixed success in zero-shot translation. It is hard to predict in which settings it will be effective, and what limits performance compared to a fully supervised system.
In this paper, we investigate zero-shot performance of a multilingual EN<->{FR,CS,DE,FI} system trained on WMT data.
We find that zero-shot performance is highly unstable and can vary by more than 6 BLEU between training runs, making it difficult to reliably track improvements.
We observe a bias towards copying the source in zero-shot translation, and investigate how the choice of subword segmentation affects this bias. We find that language-specific subword segmentation results in less subword copying at training time, and leads to better zero-shot performance compared to jointly trained segmentation.
A recent trend in multilingual models is to not train on parallel data between all language pairs, but have a single {\em bridge} language, e.g.\ English.
We find that this negatively affects zero-shot translation and leads to a failure mode where the model ignores the language tag and instead produces English output in zero-shot directions.
We show that this bias towards English can be effectively reduced with even a small amount of parallel data in some of the non-English pairs.

Statistics

Downloads

7 downloads since deposited on 08 Dec 2020
7 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Speech), refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:20 November 2020
Deposited On:08 Dec 2020 16:40
Last Modified:08 Dec 2020 22:34
Publisher:Association for Computational Linguistics
OA Status:Green
Free access at:Official URL. An embargo period may apply.
Official URL:http://www.statmt.org/wmt20/pdf/2020.wmt-1.64.pdf
Project Information:

Download

Green Open Access

Download PDF  'Subword segmentation and a single bridge language affect zero-shot neural machine translation'.
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 262kB
Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)