Header

UZH-Logo

Maintenance Infos

Behavior-Based Quality Assurance in Crowdsourcing Markets


Feldman, Michael; Bernstein, Abraham (2014). Behavior-Based Quality Assurance in Crowdsourcing Markets. In: Conference on Human Computation & Crowdsourcing 2014, Pittsburgh, USA, 2 November 2014 - 4 November 2014.

Abstract

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterised by relying on time-series data. Behavioural traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioural traces for their ability to recognise the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signalling a drop in work quality.

Abstract

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterised by relying on time-series data. Behavioural traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioural traces for their ability to recognise the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signalling a drop in work quality.

Statistics

Altmetrics

Downloads

133 downloads since deposited on 23 Sep 2014
19 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:4 November 2014
Deposited On:23 Sep 2014 15:49
Last Modified:21 Nov 2017 17:27
Publisher:s.n.
Other Identification Number:merlin-id:10078

Download

Download PDF  'Behavior-Based Quality Assurance in Crowdsourcing Markets'.
Preview
Filetype: PDF
Size: 408kB