The impact of body weight on all-cause mortality is subject to ongoing debate. We assessed the relation between body mass index (BMI) and all-cause mortality in a cohort of 8043 male employees in the German construction industry who underwent detailed occupational health examinations at ages 25-64 and who were followed for all cause mortality over an average period of 4.5 years. Overall, there was a negative, graded relation between BMI and all-cause mortality, which persisted after controlling for multiple covariates including age and cigarette smoking, and after excluding the initial two years of follow-up. There was a strong positive cross-sectional relationship between BMI and a medical diagnosis of diabetes, hypertension, and ischemic heart disease at the baseline examination. While BMI showed a strong negative relation with all-cause mortality among men with such diseases, the association was much weaker and non-monotonic for mean free of these diseases. Our results underline the importance of preexisting diseases for the prognostic value of body weight.