Permanent URL to this publication: http://dx.doi.org/10.5167/uzh-25869
Bernstein, A; Li, J (2009). From active towards itive learning: using consideration information to improve labeling correctness. In: Human Computation Workshop, Paris, France, June 2009 - June 2009.
Data mining techniques have become central to many applications. Most of those applications rely on so called supervised learning algorithms, which learn from given examples in the form of data with predefined labels (e.g., classes such as spam, not spam). Labeling, however, is oftentimes expensive, as it typically requires manual work by human experts. Active learning systems reduce the human effort by choosing the most informative instances for labeling. Unfortunately, research in psychology has shown conclusively that human decisions are inaccurate, easily biased by circumstances, and far from the oracle decision making assumed in active learning research. Based on these findings we show experimentally that (human) mistakes in labeling can significantly deteriorate the performance of active learning systems. To solve this problem, we introduce consideration information – a concept from marketing – into an active learning system to bias and improve the human’s labeling performance. Results (with simulated and human labelers) show that consideration information can indeed be used to exert a bias. Furthermore, we find that the choice of appropriate consideration information can be used to positively bias an expert and thereby improving the overall performance of the learning setting.
26 downloads since deposited on 04 Feb 2010
9 downloads since 12 months
|Item Type:||Conference or Workshop Item (Paper), refereed, original work|
|Communities & Collections:||03 Faculty of Economics > Department of Informatics|
|Dewey Decimal Classification:||000 Computer science, knowledge & systems|
|Event End Date:||June 2009|
|Deposited On:||04 Feb 2010 11:50|
|Last Modified:||09 Jul 2012 04:04|
Users (please log in): suggest update or correction for this item
Repository Staff Only: item control page