Testing for measurement equivalence of human values across online and paper-and-pencil surveys

Abstract

The following study investigates the measurement quivalence of an online and paper-and-pencil (PAP) survey of human values. For this purpose, a total of 250 respondents completed the 21-item version of the Portrait Value Questionnaire (PVQ) either online (n = 125) or by PAP (n = 125). This questionnaire was developed by Shalom Schwartz and has been included in the European Social Survey (ESS) since 2002 to test his theory of basic human values (Schwartz, 1992). Measurement invariance was tested via a multiple group confirmatory factor analysis (MGCFA). The assessment of invariance included the three levels of configural, metric, and scalar invariance, and the latent means of the values between both samples were compared. Results of this study show that the measurements are invariant at the three levels (configural, metric, and scalar), but there are latent mean differences between the values across the surveys. These differences may be partly explained by age and level of education differences between the two samples. Based on these findings we conclude that the methods of measurement are essentially invariant.