Reliability of a rating scale used to distinguish direction of eye movement using infrared/video ENG recordings during repositioning maneuvers

The purpose of this study was to determine the validity of the nystagmus rating scale (NRS) and to assess inter- and intra-rater reliability of audiologists and experts using the scale. Face and content validity was established by eliciting feedback from two neurotologists and one neurologist. A training tape was developed to describe the rating scale and provide practice with patterns of nystagmus in benign paroxysmal positional vertigo (BPPV). Eye movements of 34 patients, ages 33 to 82 years, were videotaped using infrared/video ENG during repositioning maneuvers. Six randomly paired audiologists and six experts viewed the videotape recordings and completed the NRS for each pattern. Cohen's kappa coefficients were calculated to determine inter-rater reliability. The kappa values were 0.31 with 41% agreement for audiologists, and 0.48 with 59% agreement for experts. Intra-rater reliability for a subgroup of audiologists was 0.55 with 64% agreement, and for experts was 0.75 with 81% agreement. In summary, even in this experienced population, additional training in viewing nystagmus patterns is needed to improve reliability among clinicians during diagnosis and treatment.