Permanent URL to this publication: http://dx.doi.org/10.5167/uzh-63243
Minder, Patrick; Bernstein, Abraham (2012). How to translate a book within an hour - Towards general purpose programmable human computers with CrowdLang. In: Web Science 2012, Evanston, Illinois, USA., 22 June 2012 - 24 June 2012.
In this paper we present the programming language and framework CrowdLang for engineering complex computation systems incorporating large numbers of networked humans and machines agents. We evaluate CrowdLang by developing a text translation program incorporating human and machine agents. The evaluation shows that we are able to simply explore a large design space of possible problem solving programs with the simple variation of the used abstractions. Furthermore, an experiment, involving 1918 different human actors, shows that the developed mixed human-machine translation program significantly outperforms a pure machine translation in terms of adequacy and fluency whilst translating more than 30 pages per hour and that the program approximates the professional translated gold-standard to 75% using the automatic evaluation metric METEOR. Last but not least, our evaluation illustrates that our new human computation pattern staged-contest with pruning outperforms all other refinements in the translation task.
164 downloads since deposited on 10 Jul 2012
19 downloads since 12 months
|Item Type:||Conference or Workshop Item (Paper), refereed, original work|
|Communities & Collections:||03 Faculty of Economics > Department of Informatics|
|Dewey Decimal Classification:||000 Computer science, knowledge & systems|
|Event End Date:||24 June 2012|
|Deposited On:||10 Jul 2012 08:56|
|Last Modified:||05 Dec 2013 10:15|
|Other Identification Number:||merlin-id:6988|
Users (please log in): suggest update or correction for this item
Repository Staff Only: item control page