Kelvin probe microscopy in ultrahigh vacuum is used to image the local electrostatic potential fluctuations above hexagonal boron nitride (h-BN) and SiO2, common substrates for graphene. Results are compared to a model of randomly distributed charges in a two-dimensional (2D) plane. For SiO2, the results are well modeled by 2D charge densities ranging from 0.24 to 2.7 × 10(11) cm(-2), while h-BN displays potential fluctuations 1-2 orders of magnitude lower than SiO2, consistent with the improvement in charge carrier mobility for graphene on h-BN compared to SiO2. Electron beam exposure of SiO2 increases the charge density fluctuations, creating long-lived metastable charge populations of ~2 × 10(11) cm(-2) at room temperature, which can be reversed by heating.