HIV prevalence varies markedly throughout Africa, and it is often presumed areas of higher HIV prevalence (i.e., hotspots) serve as sources of infection to neighboring areas of lower prevalence. However, the small-scale geography of migration networks and movement of HIV-positive individuals between communities is poorly understood. Here, we use population-based data from ~22,000 persons of known HIV status to characterize migratory patterns and their relationship to HIV among 38 communities in Rakai, Uganda with HIV prevalence ranging from 9 to 43%. We find that migrants moving into hotspots had significantly higher HIV prevalence than migrants moving elsewhere, but out-migration from hotspots was geographically dispersed, contributing minimally to HIV burden in destination locations. Our results challenge the assumption that high prevalence hotspots are drivers of transmission in regional epidemics, instead suggesting that migrants with high HIV prevalence, particularly women, selectively migrate to these areas.