Header

UZH-Logo

Maintenance Infos

Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data


Goldzycher, Janis; Preisig, Moritz; Amrhein, Chantal; Schneider, Gerold (2023). Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data. In: The 7th Workshop on Online Abuse and Harms (WOAH), Toronto, Canada, 13 July 2023. Association for Computational Linguistics, 187-201.

Abstract

Most research on hate speech detection has focused on English where a sizeable amount of labeled training data is available. However, to expand hate speech detection into more languages, approaches that require minimal training data are needed. In this paper, we test whether natural language inference (NLI) models which perform well in zero- and few-shot settings can benefit hate speech detection performance in scenarios where only a limited amount of labeled data is available in the target language. Our evaluation on five languages demonstrates large performance improvements of NLI fine-tuning over direct fine-tuning in the target language. However, the effectiveness of previous work that proposed intermediate fine-tuning on English data is hard to match. Only in settings where the English training data does not match the test domain, can our customised NLI-formulation outperform intermediate fine-tuning on English. Based on our extensive experiments, we propose a set of recommendations for hate speech detection in languages where minimal labeled training data is available.

Abstract

Most research on hate speech detection has focused on English where a sizeable amount of labeled training data is available. However, to expand hate speech detection into more languages, approaches that require minimal training data are needed. In this paper, we test whether natural language inference (NLI) models which perform well in zero- and few-shot settings can benefit hate speech detection performance in scenarios where only a limited amount of labeled data is available in the target language. Our evaluation on five languages demonstrates large performance improvements of NLI fine-tuning over direct fine-tuning in the target language. However, the effectiveness of previous work that proposed intermediate fine-tuning on English data is hard to match. Only in settings where the English training data does not match the test domain, can our customised NLI-formulation outperform intermediate fine-tuning on English. Based on our extensive experiments, we propose a set of recommendations for hate speech detection in languages where minimal labeled training data is available.

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

2 downloads since deposited on 11 Dec 2023
2 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
06 Faculty of Arts > Zurich Center for Linguistics
06 Faculty of Arts > Linguistic Research Infrastructure (LiRI)
Dewey Decimal Classification:410 Linguistics
000 Computer science, knowledge & systems
Scopus Subject Areas:Physical Sciences > Computer Science Applications
Social Sciences & Humanities > Linguistics and Language
Social Sciences & Humanities > Language and Linguistics
Language:English
Event End Date:13 July 2023
Deposited On:11 Dec 2023 11:40
Last Modified:31 Mar 2024 03:35
Publisher:Association for Computational Linguistics
OA Status:Closed
Free access at:Official URL. An embargo period may apply.
Publisher DOI:https://doi.org/10.18653/v1/2023.woah-1.19
Official URL:https://aclanthology.org/2023.woah-1.19