Reproducibility of multiple‐observer scoring of radiologic abnormalities in the hands and wrists of patients with rheumatoid arthritis

Abstract

Forty-one coded radiographic films from 16 patients with rheumatoid arthritis were read by 13 observers, using 4 different methods for scoring abnormalities. Although absolute scores differed widely among individual observers, correlation coefficients were greater than 0.850 for approximately 2 of 3 comparisons. When films were ranked, using the median rank of all readers, 72% of individual ranks were within 10% of the median ranks. Among serial films on individual patients, 92% of comparisons between early and late films were interpreted as demonstrating progression of abnormalities when mean standardized scores showed an increase in scores of 15 units or greater. Films with lesser changes were interpreted inconsistently. This study shows good general agreement among readers in scoring radiologic abnormalities of hands and wrists, when applied to a film set showing a broad spectrum of severity, and defines the sensitivity of radiologic detection of disease progression.