Header

UZH-Logo

Maintenance Infos

Less is more: Supporting developers in vulnerability detection during code review


Braz, Larissa; Aeberhard, Christian; Çalikli, Gül; Bacchelli, Alberto (2022). Less is more: Supporting developers in vulnerability detection during code review. In: ICSE '22: 44th International Conference on Software Engineering, Pittsburgh Pennsylvania, 21 June 2022 - 29 June 2022. ACM, 1317-1329.

Abstract

Reviewing source code from a security perspective has proven to be a difficult task. Indeed, previous research has shown that developers often miss even popular and easy-to-detect vulnerabilities during code review. Initial evidence suggests that a significant cause may lie in the reviewers' mental attitude and common practices.

In this study, we investigate whether and how explicitly asking developers to focus on security during a code review affects the detection of vulnerabilities. Furthermore, we evaluate the effect of providing a security checklist to guide the security review. To this aim, we conduct an online experiment with 150 participants, of which 71% report to have three or more years of professional development experience. Our results show that simply asking reviewers to focus on security during the code review increases eight times the probability of vulnerability detection. The presence of a security checklist does not significantly improve the outcome further, even when the checklist is tailored to the change under review and the existing vulnerabilities in the change. These results provide evidence supporting the mental attitude hypothesis and call for further work on security checklists' effectiveness and design.

Abstract

Reviewing source code from a security perspective has proven to be a difficult task. Indeed, previous research has shown that developers often miss even popular and easy-to-detect vulnerabilities during code review. Initial evidence suggests that a significant cause may lie in the reviewers' mental attitude and common practices.

In this study, we investigate whether and how explicitly asking developers to focus on security during a code review affects the detection of vulnerabilities. Furthermore, we evaluate the effect of providing a security checklist to guide the security review. To this aim, we conduct an online experiment with 150 participants, of which 71% report to have three or more years of professional development experience. Our results show that simply asking reviewers to focus on security during the code review increases eight times the probability of vulnerability detection. The presence of a security checklist does not significantly improve the outcome further, even when the checklist is tailored to the change under review and the existing vulnerabilities in the change. These results provide evidence supporting the mental attitude hypothesis and call for further work on security checklists' effectiveness and design.

Statistics

Citations

Dimensions.ai Metrics
8 citations in Web of Science®
12 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

21 downloads since deposited on 09 Mar 2023
17 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not_refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Informatics
Dewey Decimal Classification:000 Computer science, knowledge & systems
Scopus Subject Areas:Physical Sciences > Software
Scope:Discipline-based scholarship (basic research)
Language:English
Event End Date:29 June 2022
Deposited On:09 Mar 2023 10:01
Last Modified:06 Mar 2024 14:39
Publisher:ACM
ISBN:9781450392211
OA Status:Hybrid
Publisher DOI:https://doi.org/10.1145/3510003.3511560
Other Identification Number:merlin-id:23369
  • Content: Published Version
  • Language: English