Worldwide, HIV care is becoming increasingly decentralized. For patients in care at centralized facilities, this requires down-referral to local clinics for their HIV care. Information on the real-world experience and predictors of retention in care at the time of down-referral is lacking. We sought to evaluate the effect of patient-level factors on retention in care surrounding a period of down-referral to new clinics for patients with and without virologic failure (VF) on their first-line ART. We conducted a secondary analysis of a case-control study of people living with HIV attending the Sinikethemba (SKT) Clinic at McCord Hospital in Durban, South Africa. Cases (VF) and controls (no VF) responded to a questionnaire focused on individual-level factors. Subsequently, participants self-reported either changing service provider (retained in care), were unable to be reached, died or reported not attending a new provider visit (not retained in care). Multivariate logistic regression was conducted with factors associated with not being retained in care in a univariate analysis. In all, 458 patients were enrolled in the parent study (158 cases and 300 controls) with a median age of 38 years old and with 65% women. A total of 436 (95%) participants successfully established care at the down-referral clinic. In the multivariate analysis, not being pleased with the clinic (SKT), lower adherence scores, and shorter duration of ART predicted failure of down-referral. Down-referral was successful even for patients with VF. Individual-level factors could act as predictors for patients at increased risk for poor retention during the down-referral process to a local clinic.