We used functional magnetic resonance imaging (fMRI) to assess cortical activations associated with increasing task difficulty (TD) in a visuospatial paired associates learning task. Encoding and retrieval were examined when 100% successful retrieval of three, four, or six object-location pairs had been attained (thus ensuring that performance was matched across subjects). As memory load increased, in general, the number of attempts taken to achieve 100% successful retrieval increased, while the number of trials correctly completed on the first attempt decreased. By modelling parametric variations in working memory load with BOLD signal changes we were able to identify brain regions displaying linear and nonlinear responses to increasing load. During encoding, load-independent activations were found in occipitoparietal cortices (excluding the precuneus for which linear load dependency was demonstrated), anterior cingulate, and cerebellum, while linear load-dependent activations in these same regions were found during retrieval. Nonlinear load-dependent responses, as identified by categorical contrasts between levels of load, were found in the right DLPFC and left inferior frontal gyrus. The cortical response to increasing cognitive demands or TD appears to involve the same, rather than an additional, network of brain regions "working harder."