Prospective studies have shown an association between elevated plasma levels of insulin-like growth factor-I (IGF-I) and/or decreased levels of its major circulating carrier protein insulin-like growth factor binding protein-3 (IGFBP-3) and increased risk of major cancers. Identifying the factors which affect these biomarkers is of particular interest as subjects at increased risk could benefit from lifestyle changes, and/or chemoprevention intervention. We evaluated the association between constitutional, hormonal and clinical factors and IGF-I and IGFBP-3 in 740 women, including 376 unaffected women and 364 women with intraepithelial neoplasia (IEN) or early invasive breast cancer enrolled in breast cancer chemoprevention trials, conducted at a single institution. Age, body mass index (BMI), height, waist to hip girth ratio (WHR), parity, menopausal status, age at menarche, number of affected first degree relatives, number of biopsies and breast cancer status were considered in the analysis. Women with early breast cancer had 21% higher IGF-I levels (p = 0.033) and 19% higher IGF-I/IGFBP-3 molar ratio (p = 0.047) than unaffected women. In unaffected women, age was negatively associated with IGF-I (p = 0.002) and IGF-I/IGFBP-3 (p = 0.001), while age at menarche was negatively associated with IGFBP-3 levels (p = 0.043). In women with IEN or early breast cancer, IGF-I levels were negatively associated with age (p < 0.001), and positively associated with prior biopsies for benign disease (p = 0.013), while age, parity and menopausal status were significant predictors of IGF-I/IGFBP-3 molar ratio. We conclude that circulating IGF-I levels are higher in women with prior breast cancer compared to unaffected women, and that IGF-I and/or IGFBP-3 levels are influenced by age and by reproductive and hormonal factors. These findings support their putative role as breast cancer risk biomarker.