We demonstrate the existence of quantized "bulk" plasmons in ultrathin magnesium films on Si(111) by analyzing plasmon-loss satellites in core-level photoemission spectra, recorded as a function of the film thickness d. Remarkably, the plasmon energy is shown to vary as 1/d2 all the way down to three atomic layers. The loss spectra are dominated by the n=1 and n=2 normal modes, consistent with the excitation of plasmons involving quantized electronic subbands. With decreasing film thickness, spectral weight is gradually transferred from the plasmon modes to the low-energy single-particle excitations. These results represent striking manifestations of the role of quantum confinement on plasmon resonances in precisely controlled nanostructures.