Colors distinguishable with trichromatic vision can be defined by a 3D color space, such as red-green-blue or hue-saturation-lightness (HSL) space, but it remains unclear how the cortex represents colors along these dimensions. Using intrinsic optical imaging and electrophysiology, and systematically choosing color stimuli from HSL coordinates, we examined how perceptual colors are mapped in visual area V4 in behaving macaques. We show that any color activates 1-4 separate cortical patches within "globs," millimeter-sized color-preferring modules. Most patches belong to different hue or lightness clusters, in which sequential representations follow the color order in HSL space. Some patches overlap greatly with those of related colors, forming stacks, possibly representing invariable features, whereas few seem positioned irregularly. However, for any color, saturation increases the activity of all its patches. These results reveal how the color map in V4 is organized along the framework of the perceptual HSL space, whereupon different multipatch activity patterns represent different colors. We propose that such distributed and combinatorial representations may expand the encodable color space of small cortical maps and facilitate binding color information to other image features.