The early and definitive diagnosis of malignant bile duct stenoses is essential for a timely and adequate therapy. However, tissue sampling with transpapillary brush cytology (BC) or forceps biopsy (FB) remains challenging. With this study, we aimed to compare the effectiveness and safety of different tissue sampling modalities (BC, FB without/after previous balloon dilatation). Standardized database research identified all patients, who underwent endoscopic retrograde cholangiography with BC and/or FB for indeterminate bile duct stenosis between January 2010 and April 2018 and with a definitive diagnosis. 218 patients were enrolled (149 cases with malignant and 69 with benign disease). FB had a significant higher sensitivity than BC (43% vs. 16%, p < 0.01). Prior balloon dilatation of the stenosis improved the sensitivity of FB from 41 to 71% (p = 0.03), the NPV from 36 to 81% (p < 0.01) and the accuracy from 55 to 87% (p < 0.01). The complication rates did not differ significantly between the modalities. In our center FB turned out to be the diagnostically more effective procedure. Balloon dilatation of the stenosis before FB had a significant diagnostic benefit and was not associated with a higher complication rate.