Autobiographical memories are more imbued with affect when one adopts a first-person or field perspective during event retrieval, rather than a third-person or observer perspective. We combined fMRI, event narratives, and subjective ratings to identify the neural networks engaged with field versus observer memories for real-world events. Our results revealed significant decreases in bilateral insula and left somato-motor activity during the recall of observer memories, paired with a small relative increase in right posterior amygdala activity coincident with the recall of field memories. Notably, these regions showed no overlap with those areas mediating the narrative content and subjective emotionality of the remembered events. Our findings suggest that the emotionality of field relative to observer memories is not simply driven by increased limbic activation when one adopts a first-person retrieval perspective. Rather, there is also a significant reduction in one's cortical representations of the physical, embodied self when a third-person--or disembodied--perspective is taken at retrieval.