Background We compared the development of adverse effects and psychosocial measures from baseline to 36-month follow-up in patients with prostate cancer (T1-3 M0) referred to our department for definitive radiotherapy encompassing the prostate and pelvic lymph nodes (RAD + IMRT) or radiotherapy to the prostatic gland only (RAD), applied with standard adjuvant androgen deprivation (AD) in all patients. Few studies have explored the impact of fatigue on patients' reported quality of life (QoL) after combined therapy for prostate cancer. Material and methods The 206 consecutive eligible men (RAD + IMRT = 64 and RAD = 142) completed the UCLA-PCI questionnaire for adverse effects at baseline, 12, 24, and 36 months. QoL, anxiety and depression, and fatigue were rated at the same time points. Between-group and longitudinal within-group changes at different time points were reported. At 36 months variables associated with fatigue were analyzed with regression analyses. Results Our main novel finding is the long-term high level of fatigue and high prevalence of chronic fatigue, affecting patients receiving radiotherapy combined with long-term AD. Except for urinary bother in the RAD + IMRT group all functions and the other bothers mean scores were significantly worse at 36 months compared to baseline. In multivariable analyses only physical QoL remained significantly associated with fatigue at 36-months follow-up. Conclusions Fatigue and impaired QoL in patients considered to curative irradiation with long-term AD should be addressed when counseling men to combined treatment.