Header

UZH-Logo

Maintenance Infos

The multiple facets of test case quality: analyzing effectiveness and going beyond


Grano, Giovanni. The multiple facets of test case quality: analyzing effectiveness and going beyond. 2021, University of Zurich, Faculty of Economics.

Abstract

Nowadays, software pervades our life. Being software so deeply rooted into our society, software failures can cause enormous consequences. Unit test cases represent the first line of defense against the introduction of software bugs and a pillar of any software development pipeline.
Higher is their quality, the better they can fulfill their role. This research aims at supporting developers in measuring and optimizing test suite quality. To fulfill this goal, we fist characterized the test code quality aspects deemed important by practitioners. We learned that test quality does not have an exact definition and includes a variety of different facets. We also discovered that, while developers value test effectiveness, they believe it is not sufficient to achieve test quality since non-functional aspects also play a crucial role in it.
These insights motivated us to devise novel approaches to measure and optimize test effectiveness and non-functional quality aspects both in the context of manually written and automatically generated tests.
While mutation testing is widely used to measure effectiveness, its computational cost hinders its practical usage. We tackled the problem by exploiting machine learning (ML) models trained on source code features to estimate test effectiveness.
We relied on similar features to tackle the problem of code coverage prediction in the context of test case generation (TCG).
The ML models we proposed are able to suggest developers whether TCG is able to produce satisfactory result for their software projects.
To optimize non-functional aspects along with code coverage in TCG, we proposed an adaptive search-based algorithm suitable to arbitrary secondary objectives. We instantiated it to focus on test resource demands, obtaining more parsimonious tests at equal levels of code coverage.

Abstract

Nowadays, software pervades our life. Being software so deeply rooted into our society, software failures can cause enormous consequences. Unit test cases represent the first line of defense against the introduction of software bugs and a pillar of any software development pipeline.
Higher is their quality, the better they can fulfill their role. This research aims at supporting developers in measuring and optimizing test suite quality. To fulfill this goal, we fist characterized the test code quality aspects deemed important by practitioners. We learned that test quality does not have an exact definition and includes a variety of different facets. We also discovered that, while developers value test effectiveness, they believe it is not sufficient to achieve test quality since non-functional aspects also play a crucial role in it.
These insights motivated us to devise novel approaches to measure and optimize test effectiveness and non-functional quality aspects both in the context of manually written and automatically generated tests.
While mutation testing is widely used to measure effectiveness, its computational cost hinders its practical usage. We tackled the problem by exploiting machine learning (ML) models trained on source code features to estimate test effectiveness.
We relied on similar features to tackle the problem of code coverage prediction in the context of test case generation (TCG).
The ML models we proposed are able to suggest developers whether TCG is able to produce satisfactory result for their software projects.
To optimize non-functional aspects along with code coverage in TCG, we proposed an adaptive search-based algorithm suitable to arbitrary secondary objectives. We instantiated it to focus on test resource demands, obtaining more parsimonious tests at equal levels of code coverage.

Statistics

Downloads

202 downloads since deposited on 07 May 2021
43 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Dissertation (monographical)
Referees:Gall Harald, Fraser Gordon, Bacchelli Alberto, Proksch Sebastian
Communities & Collections:03 Faculty of Economics > Department of Informatics
UZH Dissertations
Dewey Decimal Classification:000 Computer science, knowledge & systems
Scope:Discipline-based scholarship (basic research)
Language:English
Place of Publication:Zürich
Date:April 2021
Deposited On:07 May 2021 05:38
Last Modified:06 Mar 2024 14:35
Number of Pages:230
OA Status:Green
Other Identification Number:merlin-id:21050
  • Content: Published Version