The Reliability of the AO Spine Upper Cervical Classification System in Children: Results of a Multi-Center Study

Background: There is no uniform classification system for traumatic upper cervical spine injuries in children. This study assesses the reliability and reproducibility of the AO Upper Cervical Spine Classification System (UCCS), which was developed and validated in adults, to children. Methods: Twenty-six patients under 18 years old with operative and nonoperative upper cervical injuries, defined as from the occipital condyle to the C2-C3 joint, were identified from 2000 to 2018. Inclusion criteria included the availability of computed tomography and magnetic resonance imaging at the time of injury. Patients with significant comorbidities were excluded. Each case was reviewed by a single senior surgeon to determine eligibility. Educational videos, schematics describing the UCCS, and imaging from 26 cases were sent to 9 pediatric orthopaedic surgeons. The surgeons classified each case into 3 categories: A, B, and C. Inter-rater reliability was assessed for the initial reading across all 9 raters by Fleiss's kappa coefficient (kF) along with 95% confidence intervals. One month later, the surgeons repeated the classification, and intra-rater reliability was calculated. All images were de-identified and randomized for each read independently. Intra-rater reproducibility across both reads was assessed using Fleiss's kappa. Interpretations for reliability estimates were based on Landis and Koch (1977): 0 to 0.2, slight; 0.2 to 0.4, fair; 0.4 to 0.6, moderate; 0.6 to 0.8, substantial; and >0.8, almost perfect agreement. Results: Twenty-six cases were read by 9 raters twice. Sub-classification agreement was moderate to substantial with ακ estimates from 0.55 for the first read and 0.70 for the second read. Inter-rater agreement was moderate (kF 0.56 to 0.58) with respect to fracture location and fair (kF 0.24 to 0.3) with respect to primary classification (A, B, and C). Krippendorff's alpha for intra-rater reliability overall sub-classifications ranged from 0.41 to 0.88, with 0.75 overall raters. Conclusion: Traumatic upper cervical injuries are rare in the pediatric population. A uniform classification system can be vital to guide diagnosis and treatment. This study is the first to evaluate the use of the UCCS in the pediatric population. While moderate to substantial agreement was found, limitations to applying the UCCS to the pediatric population exist, and thus the UCCS can be considered a starting point for developing a pediatric classification.