Header

UZH-Logo

Maintenance Infos

Using psycho-physiological measures to assess task difficulty in software development


Fritz, Thomas; Begel, Andrew; Müller, Sebastian; Yigitt-Elliott, Serap; Züger, Manuela (2014). Using psycho-physiological measures to assess task difficulty in software development. In: International Conference on Software Engineering (ICSE), Hyderabad, 31 May 2014 - 7 June 2014, 402-413.

Abstract

Software developers make programming mistakes that cause serious bugs for their customers. Existing work to detect problematic software focuses mainly on post hoc identification of correlations between bug fixes and code. We propose a new approach to address this problem --- detect when software developers are experiencing difficulty while they work on their programming tasks, and stop them before they can introduce bugs into the code.
In this paper, we investigate a novel approach to classify the difficulty of code comprehension tasks using data from psycho-physiological sensors. We present the results of a study we conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity sensor, and an electroencephalography sensor could be used to predict whether developers would find a task to be difficult. We can predict nominal task difficulty (easy/difficult) for a new developer with 64.99% precision and 64.58% recall, and for a new task with 84.38% precision and 69.79% recall. We can improve the Naive Bayes classifier's performance if we trained it on just the eye-tracking data over the entire dataset, or by using a sliding window data collection schema with a 55 second time window. Our work brings the community closer to a viable and reliable measure of task difficulty that could power the next generation of programming support tools.

Abstract

Software developers make programming mistakes that cause serious bugs for their customers. Existing work to detect problematic software focuses mainly on post hoc identification of correlations between bug fixes and code. We propose a new approach to address this problem --- detect when software developers are experiencing difficulty while they work on their programming tasks, and stop them before they can introduce bugs into the code.
In this paper, we investigate a novel approach to classify the difficulty of code comprehension tasks using data from psycho-physiological sensors. We present the results of a study we conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity sensor, and an electroencephalography sensor could be used to predict whether developers would find a task to be difficult. We can predict nominal task difficulty (easy/difficult) for a new developer with 64.99% precision and 64.58% recall, and for a new task with 84.38% precision and 69.79% recall. We can improve the Naive Bayes classifier's performance if we trained it on just the eye-tracking data over the entire dataset, or by using a sliding window data collection schema with a 55 second time window. Our work brings the community closer to a viable and reliable measure of task difficulty that could power the next generation of programming support tools.

Statistics

Citations

13 citations in Web of Science®
40 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

460 downloads since deposited on 10 Mar 2014
73 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:7 June 2014
Deposited On:10 Mar 2014 13:38
Last Modified:07 Dec 2017 08:29
Publisher:ACM
ISBN:978-1-4503-2756-5
Additional Information:Proceedings of the 36th International Conference on Software Engineering
Publisher DOI:https://doi.org/10.1145/2568225.2568266
Related URLs:http://dl.acm.org/citation.cfm?id=2568225.2568266&coll=DL&dl=ACM&CFID=630165474&CFTOKEN=15248028
http://2014.icse-conferences.org/
Other Identification Number:merlin-id:9045

Download

Download PDF  'Using psycho-physiological measures to assess task difficulty in software development'.
Preview
Content: Published Version
Filetype: PDF
Size: 342kB
View at publisher