We investigated how volumes of cerebrospinal fluid (CSF), grey matter (GM) and white matter (WM) varied with age, sex, small vessel disease and cardiovascular risk factors in the Rotterdam Scan Study. Participants (n=490; 60-90 years) were non-demented and 51.0% had hypertension, 4.9% had diabetes mellitus, 17.8% were current smoker and 54.0% were former smoker. We segmented brain MR-images into GM, normal WM, white matter lesion (WML) and CSF. Brain infarcts were rated visually. Volumes were expressed as percentage of intra-cranial volume. With increasing age, volumes of total brain, normal WM and total WM decreased; that of GM remained unchanged; and that of WML increased, in both men and women. Excluding persons with infarcts did not alter these results. Persons with larger load of small vessel disease had smaller brain volume, especially normal WM volume. Diastolic blood pressure, diabetes mellitus and current smoking were also related to smaller brain volume. In the elderly, higher age, small vessel disease and cardiovascular risk factors are associated with smaller brain volume, especially WM volume.