Suboptimal Agreement Among Cytopathologists in Diagnosis of Malignancy Based on Endoscopic Ultrasound Needle Aspirates of Solid Pancreatic Lesions: A Validation Study

Abstract

Background & aims: Despite the widespread use of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) to sample pancreatic lesions and the standardization of pancreaticobiliary cytopathologic nomenclature, there are few data on inter-observer agreement among cytopathologists evaluating pancreatic cytologic specimens obtained by EUS-FNA. We developed a scoring system to assess agreement among cytopathologists in overall diagnosis and quantitative and qualitative parameters, and evaluated factors associated with agreement.

Methods: We performed a prospective study to validate results from our pilot study that demonstrated moderate to substantial inter-observer agreement among cytopathologists for the final cytologic diagnosis. In the first phase, 3 cytopathologists refined criteria for assessment of quantity and quality measures. During phase 2, EUS-FNA specimens of solid pancreatic lesions from 46 patients were evaluated by 11 cytopathologists at 5 tertiary care centers using a standardized scoring tool. Individual quantitative and qualitative measures were scored and an overall cytologic diagnosis was determined. Clinical and EUS parameters were assessed as predictors of unanimous agreement. Inter-observer agreement (IOA) was calculated using multi-rater kappa (κ) statistics and a logistic regression model was created to identify factors associated with unanimous agreement.

Results: The IOA for final diagnoses, based on cytologic analysis, was moderate (κ = 0.56; 95% CI, 0.43-0.70). Kappa values did not increase when categories of suspicious for malignancy, malignant, and neoplasm were combined. IOA was slight to moderate for individual quantitative (κ = 0.007; 95% CI, -0.03 to -0.04) and qualitative parameters (κ = 0.5; 95% CI, 0.47-0.53). Jaundice was the only factor associated with agreement among all cytopathologists on multivariate analysis (odds ratio for unanimous agreement, 5.3; 95% CI, 1.1-26.89).

Conclusions: There is a suboptimal level of agreement among cytopathologists in the diagnosis of malignancy based on analysis of EUS-FNA specimens obtained from solid pancreatic masses. Strategies are needed to refine the cytologic criteria for diagnosis of malignancy and enhance tissue acquisition techniques to improve diagnostic reproducibility among cytopathologists.

Keywords: Cytopathology; EUS-FNA; Interobserver Agreement; Pancreas.