The progressive patterns of neurological disability in multiple sclerosis (MS) and neuromyelitis optica spectrum disorders (NMOSD) and the significance of clinical relapses to the progressions of neurological disability in these diseases have not been fully elucidated. In this study, to elucidate the impact of relapses to the progression of accumulated neurological disability and to identify the factors to affect the progression of neurological disability in MS and NMOSD, we followed 62 consecutive MS patients and 33 consecutive NMOSD patients for more than 5 years with the clinical symptoms, relapse occurrence, and Expanded Disability Status Scale (EDSS) in the chronic phase. All enrolled MS patients were confirmed to be negative for serum anti-myelin oligodendrocyte glycoprotein antibody. As a result, patients with NMOSD showed significantly severer neurological disability at 5 years from onset than MS patients. Progression in EDSS score was almost exclusively seen after clinical attacks in NMOSD, whereas progression could be observed apart from relapses in MS. Neurological disability did not change without attacks in NMOSD, whereas it sometimes spontaneously improved or deteriorated apart from relapses in MS (p < 0.001). In patients with MS, those with responsible lesions primarily in spinal cord were more likely to show such spontaneous improvement. In conclusion, clinical deterioration in NMOSD patients is irreversible and almost exclusively takes place at the timing of clinical attacks with stepwise accumulation of neurological disability. Meanwhile, changes in EDSS score can be seen apart from relapses in MS patients. Neurological disability in MS patients is partly reversible, and the patients with disease modifying drugs sometimes present spontaneous improvement of the neurological disability.