UZH-Logo

Maintenance Infos

Behavior-Based Quality Assurance in Crowdsourcing Markets


Feldman, Michael; Bernstein, Abraham (2014). Behavior-Based Quality Assurance in Crowdsourcing Markets. In: Conference on Human Computation & Crowdsourcing 2014, Pittsburgh, USA, 2 November 2014 - 4 November 2014.

Abstract

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterised by relying on time-series data. Behavioural traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioural traces for their ability to recognise the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signalling a drop in work quality.

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterised by relying on time-series data. Behavioural traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioural traces for their ability to recognise the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signalling a drop in work quality.

Altmetrics

Downloads

111 downloads since deposited on 23 Sep 2014
24 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:4 November 2014
Deposited On:23 Sep 2014 15:49
Last Modified:03 Jun 2016 03:10
Publisher:s.n.
Other Identification Number:merlin-id:10078
Permanent URL: https://doi.org/10.5167/uzh-98779

Download

[img]
Preview
Filetype: PDF
Size: 408kB

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations