Improvement of interobserver reproducibility of adhesion scoring systems

Objective: To compare the interobserver reproducibility of two adhesion scoring methods, a more comprehensive adhesion scoring method and the American Fertility Society (AFS) adhesion scoring method. Design: Eleven endoscopic surgeons independently evaluated and scored 13 surgical video recordings using both systems. Material and Methods: The standardized AFS adhesion scoring method and the more comprehensive adhesion scoring method were utilized to assess abdominal adhesions. The more comprehensive adhesion scoring method scored 23 individual locations in the abdominal cavity for severity (0, none; 1, filmy, avascular; 2, vascular and/or dense; 3, cohesive) and extent of total area or length (0, none; 1, ≤25%; 2, 26% to 50%; 3, >50%). Because the best method of representing a composite adhesion score is unclear, the more comprehensive adhesion scoring method employed two independent methods of determining total score based on the severity and extent at each location, either adding severity plus extent or multiplying severity times extent before summing all 23 locations. Results: For each scoring method, a correlation coefficient was calculated for each of the 55 pairs of surgeons. Significant correlations were identified for all methods (AFS adhesion scoring method: 35 of 55, 64%; more comprehensive adhesion scoring method: severity plus extent, 49 of 55, 89%; and more comprehensive adhesion scoring method: severity times extent, 53 of 55, 96%). When the more comprehensive adhesion scoring method is limited to 13 areas in the lower pelvis corresponding to the locations represented in the AFS adhesion scoring method, the proportion of significant correlation coefficients become 54 of 55 and 50 of 55 for severity plus extent and severity times extent, respectively. Conclusion: Although the AFS adhesion scoring method generated significant agreement between pairs of surgeons, less than a third correlated at the 0.7 level. Use of the more comprehensive adhesion scoring method specifically demonstrating locations, severity, and extent of adhesions produced a marked increase in reproducibility between surgeon pairs in scoring pelvic adhesions.