Header

UZH-Logo

Maintenance Infos

Reliable and valid measures of threat detection performance in x-ray screening


Hofer, Franziska; Schwaninger, Adrian (2004). Reliable and valid measures of threat detection performance in x-ray screening. In: Proceedings of the 38th Carnahan Conference on Security Technology, Albuquerque, NM, 2004 - 2004, 303-308.

Abstract

Over the last decades, airport security technology has evolved remarkably. This is especially evident when state-of-the-art detection systems are concerned. However, such systems are only as effective as the personnel who operate them. Reliable and valid measures of screener detection performance are important for risk analysis, screener certification and competency assessment, as well as for measuring quality performance and effectiveness of training systems. In many of these applications the hit rate is used in order to measure detection performance. However, measures based on signal detection theory have gained popularity in recent years, for example in the analysis of data from threat image projection (TIP) or computer based training (CBT) systems. In this study, computer-based tests were used to measure detection performance for improvised explosive devices (IEDs). These tests were conducted before and after training with an individually adaptive CBT system. The following measures were calculated: pHit, d', Δm, Az, A' p(c)max. All measures correlated well, but ROC curve analysis suggests that "nonparametric" measures are more valid to measure detection performance for IEDs. More specifically, we found systematic deviations in the ROC curves that are consistent with two-state low threshold theory of R.D. Luce (1963). These results have to be further studied and the question rises if similar results could be obtained for other X-ray screening data. In any case, it is recommended to use A' in addition to d' in practical applications such as certification, threat image projection and CBT rather than the hit rate alone.

Abstract

Over the last decades, airport security technology has evolved remarkably. This is especially evident when state-of-the-art detection systems are concerned. However, such systems are only as effective as the personnel who operate them. Reliable and valid measures of screener detection performance are important for risk analysis, screener certification and competency assessment, as well as for measuring quality performance and effectiveness of training systems. In many of these applications the hit rate is used in order to measure detection performance. However, measures based on signal detection theory have gained popularity in recent years, for example in the analysis of data from threat image projection (TIP) or computer based training (CBT) systems. In this study, computer-based tests were used to measure detection performance for improvised explosive devices (IEDs). These tests were conducted before and after training with an individually adaptive CBT system. The following measures were calculated: pHit, d', Δm, Az, A' p(c)max. All measures correlated well, but ROC curve analysis suggests that "nonparametric" measures are more valid to measure detection performance for IEDs. More specifically, we found systematic deviations in the ROC curves that are consistent with two-state low threshold theory of R.D. Luce (1963). These results have to be further studied and the question rises if similar results could be obtained for other X-ray screening data. In any case, it is recommended to use A' in addition to d' in practical applications such as certification, threat image projection and CBT rather than the hit rate alone.

Statistics

Citations

3 citations in Web of Science®
24 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

1 download since deposited on 12 Aug 2014
0 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Psychology
Dewey Decimal Classification:150 Psychology
Language:English
Event End Date:2004
Deposited On:12 Aug 2014 15:25
Last Modified:08 Dec 2017 06:47
Publisher:O E S Publications
Series Name:International Carnahan Conference of Security Technology. Proceedings
ISSN:1071-6572
Publisher DOI:https://doi.org/10.1109/CCST.2004.1405409

Download