Title
Estimating rater agreement in 2x2 tables: Correction for chance and intraclass correlation
Abstract
Many estimators of the measure of agreement
between two dichotomous ratings of a person have
been proposed. The results of Fleiss (1975) are
extended, and it is shown that four estimators-
Scott’s (1955) π coefficient, Cohen’s (1960) kˆ,
Maxwell & Pilliner’s (1968) r₁₁, and Mak’s (1988)
p˜-are interpretable both as chance-corrected
measures of agreement and as intraclass correlation
coefficients for different ANOVA models. Relationships
among these estimators are established
for finite samples. Under Kraemer’s (1979) model,
it is shown that these estimators are equivalent in
large samples, and that the equations for their
large sample variances are equivalent. Index
terms: index of agreement, interrater reliability,
intraclass correlation, kappa statistic.
Identifiers
other: doi:10.1177/014662169301700302
Previously Published Citation
Blackman, Nicole J.-M & Koval, John J. (1993). Estimating rater agreement in 2x2 tables: Correction for chance and intraclass correlation. Applied Psychological Measurement, 17, 211-223. doi:10.1177/014662169301700302
Suggested Citation
Blackman, Nicole J.-M.; Koval, John J..
(1993).
Estimating rater agreement in 2x2 tables: Correction for chance and intraclass correlation.
Retrieved from the University of Minnesota Digital Conservancy,
https://hdl.handle.net/11299/116366.