Epidemiologic studies of histologic types of breast cancer including mucinous, medullary, and tubular carcinomas have primarily relied on International Classification of Diseases-Oncology (ICD-O) codes assigned by local pathologists to define histology. Using data from the Breast Cancer Family Registry (BCFR), we compared histologic agreement between centralized BCFR pathology review and ICD-O codes available from local tumor registries among 3,260 breast cancer cases. Agreement was low to moderate for less common histologies; for example, only 55 and 26 % of cases classified as mucinous and medullary, respectively, by centralized review were similarly classified using ICD-O coding. We then evaluated risk factors for each histologic subtype by comparing each histologic case group defined by centralized review with a common set of 2,997 population-based controls using polytomous logistic regression. Parity [odds ratio (OR) = 0.4, 95 % confidence interval (95 % CI): 0.2-0.9, for parous vs. nulliparous], age at menarche (OR = 0.5, 95 % CI: 0.3-0.9, for age ≥13 vs. ≤11), and use of oral contraceptives (OCs) (OR = 0.5, 95 % CI: 0.2-0.8, OC use >5 years vs. never) were associated with mucinous carcinoma (N = 92 cases). Body mass index (BMI) (OR = 1.05, 95 % CI: 1.0-1.1, per unit of BMI) and high parity (OR = 2.6, 95 % CI: 1.1-6.0 for ≥3 live births vs. nulliparous) were associated with medullary carcinoma (N = 90 cases). We did not find any associations between breast cancer risk factors and tubular carcinoma (N = 86 cases). Relative risk estimates from analyses using ICD-O classifications of histology, rather than centralized review, resulted in attenuated, and/or more imprecise, associations. These findings suggest risk factor heterogeneity across breast cancer tumor histologies, and demonstrate the value of centralized pathology review for classifying rarer tumor types.