Episodic-like memory is thought to be supported by attractor dynamics in the hippocampus. A possible neural substrate for this memory mechanism is rate remapping, in which the spatial map of place cells encodes contextual information through firing rate variability. To test whether memories are stored as multimodal attractors in populations of place cells, recent experiments morphed one familiar context into another while observing the responses of CA3 cell ensembles. Average population activity in CA3 was reported to transition gradually rather than abruptly from one familiar context to the next, suggesting a lack of attractive forces associated with the two stored representations. On the other hand, individual CA3 cells showed a mix of gradual and abrupt transitions at different points along the morph sequence, and some displayed hysteresis which is a signature of attractor dynamics. To understand whether these seemingly conflicting results are commensurate with attractor network theory, we developed a neural network model of the CA3 with attractors for both position and discrete contexts. We found that for memories stored in overlapping neural ensembles within a single spatial map, position-dependent context attractors made transitions at different points along the morph sequence. Smooth transition curves arose from averaging across the population, while a heterogeneous set of responses was observed on the single unit level. In contrast, orthogonal memories led to abrupt and coherent transitions on both population and single unit levels as experimentally observed when remapping between two independent spatial maps. Strong recurrent feedback entailed a hysteretic effect on the network which diminished with the amount of overlap in the stored memories. These results suggest that context-dependent memory can be supported by overlapping local attractors within a spatial map of CA3 place cells. Similar mechanisms for context-dependent memory may also be found in other regions of the cerebral cortex.