Navigation auf zora.uzh.ch

Search ZORA

ZORA (Zurich Open Repository and Archive)

Towards Holistic Human Evaluation of Automatic Text Simplification

Carrer, Luisa; Säuberli, Andreas; Kappus, Martin; Ebling, Sarah (2024). Towards Holistic Human Evaluation of Automatic Text Simplification. In: Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024, Turin, Italy, 21 May 2024. ELRA and ICCL, 71-80.

Abstract

Text simplification refers to the process of rewording within a single language, moving from a standard form into an easy-to-understand one. Easy Language and Plain Language are two examples of simplified varieties aimed at improving readability and understanding for a wide-ranging audience. Human evaluation of automatic text simplification is usually done by employing experts or crowdworkers to rate the generated texts. However, this approach does not include the target readers of simplified texts and does not reflect actual comprehensibility. In this paper, we explore different ways of measuring the quality of automatically simplified texts. We conducted a multi-faceted evaluation study involving end users, post-editors, and Easy Language experts and applied a variety of qualitative and quantitative methods. We found differences in the perception and actual comprehension of the texts by different user groups. In addition, qualitative surveys and behavioral observations proved to be essential in interpreting the results.

Additional indexing

Item Type:Conference or Workshop Item (Paper), original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:21 May 2024
Deposited On:05 Jun 2024 18:13
Last Modified:05 Aug 2024 14:48
Publisher:ELRA and ICCL
OA Status:Green
Official URL:https://aclanthology.org/2024.humeval-1.7
Download PDF  'Towards Holistic Human Evaluation of Automatic Text Simplification'.
Preview
  • Content: Published Version
  • Language: English
  • Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)
Download PDF  'Towards Holistic Human Evaluation of Automatic Text Simplification'.
Preview
  • Content: Published Version
  • Language: German
  • Description: Zusammenfassung in leicht verständlichem Deutsch
  • Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)

Metadata Export

Statistics

Downloads

10 downloads since deposited on 05 Jun 2024
10 downloads since 12 months
Detailed statistics

Authors, Affiliations, Collaborations

Similar Publications