Navigation auf zora.uzh.ch

Search ZORA

ZORA (Zurich Open Repository and Archive)

Migration von ZORA auf die Software DSpace

ZORA will change to a new software on 8th September 2025. Please note: deadline for new submissions is 21th July 2025!

Information & dates for training courses can be found here: Information on Software Migration.

Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators

Grano, Giovanni; Palomba, Fabio; Gall, Harald (2021). Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):758-774.

Abstract

Test cases are crucial to help developers preventing the introduction of software faults. Unfortunately, not all the tests are properly designed or can effectively capture faults in production code. Some measures have been defined to assess test-case effectiveness: the most relevant one is the mutation score, which highlights the quality of a test by generating the so-called mutants, ie variations of the production code that make it faulty and that the test is supposed to identify. However, previous studies revealed that mutation analysis is extremely costly and hard to use in practice. The approaches proposed by researchers so far have not been able to provide practical gains in terms of mutation testing efficiency. This leaves the problem of efficiently assessing test-case effectiveness as still open. In this paper, we investigate a novel, orthogonal, and lightweight methodology to assess test-case effectiveness: in particular, we study the feasibility to exploit production and test-code-quality indicators to estimate the mutation score of a test case. We firstly select a set of 67 factors and study their relation with test-case effectiveness. Then, we devise a mutation score estimation model exploiting such factors and investigate its performance as well as its most relevant features. The key results of the study reveal that our estimation model only based on static features has 86% of both F-Measure and AUC-ROC. This means that we can estimate the test-case effectiveness, using source-code-quality indicators, with high accuracy and without executing the tests. As a consequence, we can provide a practical approach that is beyond the typical limitations of current mutation testing techniques.

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Scope:Discipline-based scholarship (basic research)
Language:English
Date:1 April 2021
Deposited On:15 Mar 2019 09:39
Last Modified:20 Jul 2025 01:37
Publisher:Institute of Electrical and Electronics Engineers
ISSN:0098-5589
Additional Information:© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
OA Status:Green
Publisher DOI:https://doi.org/10.1109/TSE.2019.2903057
Related URLs:https://ieeexplore.ieee.org/document/8658120 (Publisher)
Other Identification Number:merlin-id:17661
Download PDF  'Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators'.
Preview
  • Content: Accepted Version

Metadata Export

Statistics

Citations

Dimensions.ai Metrics
33 citations in Web of Science®
26 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

358 downloads since deposited on 15 Mar 2019
49 downloads since 12 months
Detailed statistics

Authors, Affiliations, Collaborations

Similar Publications