UZH-Logo

Evaluating semantic search systems to identify future directions of research


Elbedweihy, Khadija; Wrigley, Stuart N; Ciravegna, Fabio; Reinhard, Dorothee; Bernstein, Abraham (2012). Evaluating semantic search systems to identify future directions of research. In: Second International Workshop on Evaluation of Semantic Technologies, Heraklion, Greece, 28 May 2012 - 28 May 2012, 25-36.

Abstract

Recent work on searching the Semantic Web has yielded a wide range of approaches with respect to the style of input, the under- lying search mechanisms and the manner in which results are presented. Each approach has an impact upon the quality of the information re- trieved and the user’s experience of the search process. This highlights the need for formalised and consistent evaluation to benchmark the cov- erage, applicability and usability of existing tools and provide indications of future directions for advancement of the state-of-the-art. In this paper, we describe a comprehensive evaluation methodology which addresses both the underlying performance and the subjective usability of a tool. We present the key outcomes of a recently completed international evalu- ation campaign which adopted this approach and thus identify a number of new requirements for semantic search tools from both the perspective of the underlying technology as well as the user experience.

Recent work on searching the Semantic Web has yielded a wide range of approaches with respect to the style of input, the under- lying search mechanisms and the manner in which results are presented. Each approach has an impact upon the quality of the information re- trieved and the user’s experience of the search process. This highlights the need for formalised and consistent evaluation to benchmark the cov- erage, applicability and usability of existing tools and provide indications of future directions for advancement of the state-of-the-art. In this paper, we describe a comprehensive evaluation methodology which addresses both the underlying performance and the subjective usability of a tool. We present the key outcomes of a recently completed international evalu- ation campaign which adopted this approach and thus identify a number of new requirements for semantic search tools from both the perspective of the underlying technology as well as the user experience.

Citations

Downloads

34 downloads since deposited on 10 Jul 2012
6 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:28 May 2012
Deposited On:10 Jul 2012 15:08
Last Modified:05 Apr 2016 15:52
Series Name:CEUR Workshop Proceedings
Number:843
ISSN:1613-0073
Official URL:http://ceur-ws.org/Vol-843/paper3.pdf
Related URLs:http://ceur-ws.org/Vol-843/
Other Identification Number:merlin-id:6971
Permanent URL: http://doi.org/10.5167/uzh-63315

Download

[img]
Preview
Content: Published Version
Filetype: PDF
Size: 1MB

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations