Header

UZH-Logo

Maintenance Infos

Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks


Emelin, Denis; Titov, Ivan; Sennrich, Rico (2020). Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, 16 November 2020 - 20 November 2020, 7635-7653.

Abstract

Word sense disambiguation is a well-known source of translation errors in NMT. We posit that some of the incorrect disambiguation choices are due to models' over-reliance on dataset artifacts found in training data, specifically superficial word co-occurrences, rather than a deeper understanding of the source text. We introduce a method for the prediction of disambiguation errors based on statistical data properties, demonstrating its effectiveness across several domains and model types. Moreover, we develop a simple adversarial attack strategy that minimally perturbs sentences in order to elicit disambiguation errors to further probe the robustness of translation models. Our findings indicate that disambiguation robustness varies substantially between domains and that different models trained on the same data are vulnerable to different attacks.

Abstract

Word sense disambiguation is a well-known source of translation errors in NMT. We posit that some of the incorrect disambiguation choices are due to models' over-reliance on dataset artifacts found in training data, specifically superficial word co-occurrences, rather than a deeper understanding of the source text. We introduce a method for the prediction of disambiguation errors based on statistical data properties, demonstrating its effectiveness across several domains and model types. Moreover, we develop a simple adversarial attack strategy that minimally perturbs sentences in order to elicit disambiguation errors to further probe the robustness of translation models. Our findings indicate that disambiguation robustness varies substantially between domains and that different models trained on the same data are vulnerable to different attacks.

Statistics

Downloads

10 downloads since deposited on 10 Nov 2020
10 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:20 November 2020
Deposited On:10 Nov 2020 10:53
Last Modified:27 Nov 2020 07:34
Publisher:Association for Computational Linguistics
OA Status:Green
Official URL:https://www.aclweb.org/anthology/2020.emnlp-main.616
Project Information:
  • : FunderSNSF
  • : Grant IDPP00P1_176727
  • : Project TitleMulti-Task Learning with Multilingual Resources for Better Natural Language Understanding

Download

Green Open Access

Download PDF  'Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks'.
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 424kB
Licence: Creative Commons: Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)