Header

UZH-Logo

Maintenance Infos

When experts disagree: response aggregation and its consequences in expert surveys


Lindstädt, René; Proksch, Sven-Oliver; Slapin, Jonathan B (2020). When experts disagree: response aggregation and its consequences in expert surveys. Political Science Research and Methods, 8(3):580-588.

Abstract

Political scientists use expert surveys to assess the latent features of political actors. Experts, though, are unlikely to be equally informed and assess all actors equally well. The literature acknowledges variance in measurement quality but pays little attention to the implications of uncertainty for aggregating responses. We discuss the nature of the measurement problem in expert surveys. We then propose methods to assess the ability of experts to judge where actors stand and to aggregate expert responses. We examine the effects of aggregation for a prominent survey in the literature on party politics and EU integration. Using a Monte Carlo simulation, we demonstrate that it is better to aggregate expert responses using the median or modal response, rather than the mean.

Abstract

Political scientists use expert surveys to assess the latent features of political actors. Experts, though, are unlikely to be equally informed and assess all actors equally well. The literature acknowledges variance in measurement quality but pays little attention to the implications of uncertainty for aggregating responses. We discuss the nature of the measurement problem in expert surveys. We then propose methods to assess the ability of experts to judge where actors stand and to aggregate expert responses. We examine the effects of aggregation for a prominent survey in the literature on party politics and EU integration. Using a Monte Carlo simulation, we demonstrate that it is better to aggregate expert responses using the median or modal response, rather than the mean.

Statistics

Citations

Dimensions.ai Metrics
15 citations in Web of Science®
16 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

3 downloads since deposited on 29 Dec 2020
0 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Political Science
Dewey Decimal Classification:320 Political science
Scopus Subject Areas:Social Sciences & Humanities > Sociology and Political Science
Social Sciences & Humanities > Political Science and International Relations
Language:English
Date:July 2020
Deposited On:29 Dec 2020 12:10
Last Modified:24 Apr 2024 01:49
Publisher:Cambridge University Press
ISSN:2049-8470
OA Status:Closed
Publisher DOI:https://doi.org/10.1017/psrm.2018.52