Header

UZH-Logo

Maintenance Infos

A method for in-depth comparative evaluation: How (dis)similar are outputs of POS taggers, dependency parsers and coreference resolvers really?


Tuggener, Don (2017). A method for in-depth comparative evaluation: How (dis)similar are outputs of POS taggers, dependency parsers and coreference resolvers really? In: 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, 3 April 2017 - 7 April 2017, 188-198.

Abstract

This paper proposes a generic method for the comparative evaluation of system outputs. The approach is able to quantify the pairwise differences between two outputs and to unravel in detail what the differences consist of. We apply our approach to three tasks in Computational Linguistics, i.e. POS tagging, dependency parsing, and coreference resolution. We find that system outputs are more distinct than the (often) small differences in evaluation scores seem to suggest.

Abstract

This paper proposes a generic method for the comparative evaluation of system outputs. The approach is able to quantify the pairwise differences between two outputs and to unravel in detail what the differences consist of. We apply our approach to three tasks in Computational Linguistics, i.e. POS tagging, dependency parsing, and coreference resolution. We find that system outputs are more distinct than the (often) small differences in evaluation scores seem to suggest.

Statistics

Downloads

28 downloads since deposited on 21 Apr 2017
28 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:7 April 2017
Deposited On:21 Apr 2017 08:04
Last Modified:30 Aug 2017 08:33
Publisher:Association for Computational Linguistics
Free access at:Official URL. An embargo period may apply.
Official URL:https://aclweb.org/anthology/E/E17/E17-1018.pdf

Download

Preview Icon on Download
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 197kB