Careless responding in questionnaire measures: Detection, impact, and remedies

Abstract

Undetected carelessly given responses in survey data diminish the credibility of study findings. We therefore examined two pressing issues: the accuracy of popular screening indices, and the impact of careless responding on the psychometric properties of constructs. In an experiment in Study 1, induced response sets were used to examine the effectiveness of seven indices in detecting careless responding. Response time per item, personal reliability, psychometric synonyms, psychometric antonyms, and Mahalanobis distance were effective. However, the longstring and intra-individual response variability were ineffective. In Study 2, the effects of careless responding were examined under normal study conditions. In this sample, 33% of the participants were identified as careless responders. Careless responding inflated item variances, biased item means towards the scale midpoint, increased residual variances of construct indicators, and reduced the within-group agreement on consensus-based constructs. To enhance the credibility of findings, therefore, screenings for careless responding should be applied regularly.