Header

UZH-Logo

Maintenance Infos

When testing meets code review: why and how developers review tests


Spadini, Davide; Aniche, Maurício; Storey, Margaret-Anne; Bruntink, Magiel; Bacchelli, Alberto (2018). When testing meets code review: why and how developers review tests. In: 40th International Conference on Software Engineering (ICSE), Gothenburg, Sweden, 27 June 2018 - 3 July 2018, 677-687.

Abstract

Automated testing is considered an essential process for ensuring software quality. However, writing and maintaining high-quality test code is challenging and frequently considered of secondary importance. For production code, many open source and industrial software projects employ code review, a well-established software quality practice, but the question remains whether and how code review is also used for ensuring the quality of test code. The aim of this research is to answer this question and to increase our understanding of what developers think and do when it comes to reviewing test code. We conducted both quantitative and qualitative methods to analyze more than 300,000 code reviews, and interviewed 12 developers about how they review test files. This work resulted in an overview of current code reviewing practices, a set of identified obstacles limiting the review of test code, and a set of issues that developers would like to see improved in code review tools. The study reveals that reviewing test files is very different from reviewing production files, and that the navigation within the review itself is one of the main issues developers currently face. Based on our findings, we propose a series of recommendations and suggestions for the design of tools and future research.

Abstract

Automated testing is considered an essential process for ensuring software quality. However, writing and maintaining high-quality test code is challenging and frequently considered of secondary importance. For production code, many open source and industrial software projects employ code review, a well-established software quality practice, but the question remains whether and how code review is also used for ensuring the quality of test code. The aim of this research is to answer this question and to increase our understanding of what developers think and do when it comes to reviewing test code. We conducted both quantitative and qualitative methods to analyze more than 300,000 code reviews, and interviewed 12 developers about how they review test files. This work resulted in an overview of current code reviewing practices, a set of identified obstacles limiting the review of test code, and a set of issues that developers would like to see improved in code review tools. The study reveals that reviewing test files is very different from reviewing production files, and that the navigation within the review itself is one of the main issues developers currently face. Based on our findings, we propose a series of recommendations and suggestions for the design of tools and future research.

Statistics

Citations

Dimensions.ai Metrics
3 citations in Web of Science®
5 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

21 downloads since deposited on 24 Aug 2018
21 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:3 July 2018
Deposited On:24 Aug 2018 13:33
Last Modified:13 Dec 2018 04:29
Publisher:Association for Computing Machinery
Series Name:Proceedings of the International Conference on Software Engineering
ISBN:9781450356381
OA Status:Green
Publisher DOI:https://doi.org/10.1145/3180155.3180192
Related URLs:https://doi.org/10.5281/zenodo.1172419 (Author)
https://www.icse2018.org/ (Organisation)
Other Identification Number:merlin-id:16641

Download

Download PDF  'When testing meets code review: why and how developers review tests'.
Preview
Content: Published Version
Filetype: PDF
Size: 605kB
View at publisher