Accumulating evidence from electrophysiology and neuroimaging studies suggests that spatial working memory is subserved by a network of frontal and parietal regions. In the present study, we parametrically varied the memory set size (one to four spatial locations) of a delayed-response task and applied time-resolved fMRI to study the influence of memory load upon the spatial working memory circuit. Our behavioral results showed that performance deteriorates (lower accuracy and longer reaction time) as memory load increases. Memory load influenced cortical activity during the cue, delay, and response phases of the delayed-response task. Although delay-related activity in many regions increased with increasing memory load, it also was significantly reduced in the middle frontal gyrus and frontal eye fields and leveled off in the parietal areas when memory load increased further. Delay-related activity in the left posterior parietal cortex was also lower during the error trials, in comparison with the correct trials. Our findings indicate that the delay period activity in the spatial working memory circuit is load sensitive and that the attenuation of this signal is the neural manifestation of performance limitation in the face of excessive memory load.