Header

UZH-Logo

Maintenance Infos

Confidence Intervals for Intraclass Correlation in Inter-Rater Reliability


Rousson, V; Gasser, Theo; Seifert, Burkhardt (2003). Confidence Intervals for Intraclass Correlation in Inter-Rater Reliability. Scandinavian Journal of Statistics, 30(3):617-624.

Abstract

AbstractCalculation of a confidence interval for intraclass correlation to assess inter-rater reliability is problematic when the number of raters is small and the rater effect is not negligible. Intervals produced by existing methods are uninformative: the lower bound is often close to zero, even in cases where the reliability is good and the sample size is large. In this paper, we show that this problem is unavoidable without extra assumptions and we propose two new approaches. The first approach assumes that the raters are sufficiently trained and is related to a sensitivity analysis. The second approach is based on a model with fixed rater effect. Using either approach, we obtain conservative and informative confidence intervals even from samples with only two raters. We illustrate our point with data on the development of neuromotor functions in children and adolescents.

Abstract

AbstractCalculation of a confidence interval for intraclass correlation to assess inter-rater reliability is problematic when the number of raters is small and the rater effect is not negligible. Intervals produced by existing methods are uninformative: the lower bound is often close to zero, even in cases where the reliability is good and the sample size is large. In this paper, we show that this problem is unavoidable without extra assumptions and we propose two new approaches. The first approach assumes that the raters are sufficiently trained and is related to a sensitivity analysis. The second approach is based on a model with fixed rater effect. Using either approach, we obtain conservative and informative confidence intervals even from samples with only two raters. We illustrate our point with data on the development of neuromotor functions in children and adolescents.

Statistics

Citations

9 citations in Web of Science®
9 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

87 downloads since deposited on 27 Jun 2009
4 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:04 Faculty of Medicine > Epidemiology, Biostatistics and Prevention Institute (EBPI)
07 Faculty of Science > Institute of Mathematics
Dewey Decimal Classification:610 Medicine & health
510 Mathematics
Date:2003
Deposited On:27 Jun 2009 17:43
Last Modified:05 Apr 2016 13:16
Publisher:Wiley-Blackwell
ISSN:0303-6898
Additional Information:The definitive version is available at www.blackwell-synergy.com
Publisher DOI:https://doi.org/10.1111/1467-9469.00353

Download

Preview Icon on Download
Preview
Filetype: PDF
Size: 1MB
View at publisher