Header

UZH-Logo

Maintenance Infos

On the Effectiveness of Manual and Automatic Unit Test Generation: Ten Years Later


Serra, Domenico; Grano, Giovanni; Palomba, Fabio; Ferrucci, Filomena; Gall, Harald C; Bacchelli, Alberto (2019). On the Effectiveness of Manual and Automatic Unit Test Generation: Ten Years Later. In: Proceedings of the 16th International Conference on Mining Software Repositories, Montreal, Quebec, Canada, 1 January 2019 - 1 January 2019, 121-125.

Abstract

Good unit tests play a paramount role when it comes to foster and evaluate software quality. However, writing effective tests is an extremely costly and time consuming practice. To reduce such a burden for developers, researchers devised ingenious techniques to automatically generate test suite for existing code bases. Nevertheless, how automatically generated test cases fare against manually written ones is an open research question. In 2008, Bacchelli et al. conducted an initial case study comparing automatic and manually generated test suites. Since in the last ten years we have witnessed a huge amount of work on novel approaches and tools for automatic test generation, in this paper we revise their study using current tools as well as complementing their research method by evaluating these tools' ability in finding regressions. Preprint [https://doi.org/10.5281/zenodo. 2595232], dataset [https://doi.org/10.6084/m9.figshare.7628642].

Abstract

Good unit tests play a paramount role when it comes to foster and evaluate software quality. However, writing effective tests is an extremely costly and time consuming practice. To reduce such a burden for developers, researchers devised ingenious techniques to automatically generate test suite for existing code bases. Nevertheless, how automatically generated test cases fare against manually written ones is an open research question. In 2008, Bacchelli et al. conducted an initial case study comparing automatic and manually generated test suites. Since in the last ten years we have witnessed a huge amount of work on novel approaches and tools for automatic test generation, in this paper we revise their study using current tools as well as complementing their research method by evaluating these tools' ability in finding regressions. Preprint [https://doi.org/10.5281/zenodo. 2595232], dataset [https://doi.org/10.6084/m9.figshare.7628642].

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

17 downloads since deposited on 02 Jul 2019
17 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Language:English
Event End Date:1 January 2019
Deposited On:02 Jul 2019 12:29
Last Modified:25 Sep 2019 00:37
Publisher:IEEE Press
Series Name:MSR '19
OA Status:Green
Publisher DOI:https://doi.org/10.1109/MSR.2019.00028
Official URL:https://doi.org/10.1109/MSR.2019.00028
Other Identification Number:merlin-id:17894

Download

Download PDF  'On the Effectiveness of Manual and Automatic Unit Test Generation: Ten Years Later'.
Preview
Content: Published Version
Filetype: PDF
Size: 163kB
View at publisher