Header

UZH-Logo

Maintenance Infos

User evaluation of automatically generated keywords and toponyms for geo-referenced images


Ostermann, Frank O; Tomko, Martin; Purves, Ross S (2013). User evaluation of automatically generated keywords and toponyms for geo-referenced images. Journal of the American Society for Information Science and Technology, 64(3):480-499.

Abstract

This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.

Abstract

This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.

Statistics

Citations

5 citations in Web of Science®
6 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

1 download since deposited on 19 Dec 2013
0 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, not refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Geography
08 University Research Priority Programs > Language and Space
Dewey Decimal Classification:910 Geography & travel
Language:English
Date:2013
Deposited On:19 Dec 2013 15:35
Last Modified:22 Nov 2017 22:29
Publisher:Wiley-Blackwell
ISSN:0002-8231
Publisher DOI:https://doi.org/10.1002/asi.22738

Download