We have used coherent, resonant, x-ray magnetic speckle patterns to measure the statistical evolution of the microscopic magnetic domains in perpendicular magnetic films as a function of the applied magnetic field. Our work constitutes the first direct, ensemble-averaged study of microscopic magnetic return-point memory, and demonstrates the profound impact of interfacial roughness on this phenomenon. At low fields, the microscopic magnetic domains forget their past history with an exponential field dependence.