Evidence has shown that polymorphisms of various genes known to be involved in estrogen biosynthesis and function are associated with estrogen-dependent diseases (EDDs). These genes include CYP17A1, estrogen receptor 1 (ESR1), and 2 (ESR2). Phthalates are considered estrogenic endocrine disruptors, and recent research has suggested that they may act as a risk factor for EDDs. However, extremely few studies have assessed the effects of gene-environment interaction on these diseases. We recruited 44 patients with endometriosis or adenomyosis, 36 patients with leiomyoma, and 69 healthy controls from a medical center in Taiwan between 2005 and 2007. Urine samples were collected and analyzed for seven phthalate metabolites using liquid chromatography tandem mass spectrometry. Peripheral lymphocytes were used for DNA extraction to determine the genotype of CYP17A1, ESR1, and ESR2. Compared to controls, patients with leiomyoma had significantly higher levels of total urinary mono-ethylhexyl phthalate (ΣMEHP) (52.1 vs. 29.6 μg/g creatinine, p = 0.040), mono-n-butyl phthalate (MnBP) (75.4 vs. 51.3 μg/g creatinine, p = 0.019), and monoethyl phthalate (MEP) (103.7 vs. 59.3 μg/g creatinine, p = 0.031). In contrast, patients with endometriosis or adenomyosis showed a marginally increased level of urinary MEHP only. Subjects who were homozygous for both the ESR1 C allele (rs2234693) and CYP17A1 C allele (rs743572) showed a significantly increased risk for leiomyoma (OR = 19.8; 95 % CI, 1.70; 231.5; p = 0.017) relative to subjects with other genotypes of ESR1 and CYP17A1. These results were obtained after adjusting for age, cigarette smoking, MEHP level, GSTM1 genotype and other covariates. Our results suggested that both CYP17A1 and ESR1 polymorphisms may modulate the effects of phthalate exposure on the development of leiomyoma.