Chromatin architecture has been implicated in cell type-specific gene regulatory programs, yet how chromatin remodels during development remains to be fully elucidated. Here, by interrogating chromatin reorganization during human pluripotent stem cell (hPSC) differentiation, we discover a role for the primate-specific endogenous retrotransposon human endogenous retrovirus subfamily H (HERV-H) in creating topologically associating domains (TADs) in hPSCs. Deleting these HERV-H elements eliminates their corresponding TAD boundaries and reduces the transcription of upstream genes, while de novo insertion of HERV-H elements can introduce new TAD boundaries. The ability of HERV-H to create TAD boundaries depends on high transcription, as transcriptional repression of HERV-H elements prevents the formation of boundaries. This ability is not limited to hPSCs, as these actively transcribed HERV-H elements and their corresponding TAD boundaries also appear in pluripotent stem cells from other hominids but not in more distantly related species lacking HERV-H elements. Overall, our results provide direct evidence for retrotransposons in actively shaping cell type- and species-specific chromatin architecture.