We report on the intrinsic optoelectronic response of high-quality dual-gated monolayer and bilayer graphene p-n junction devices. Local laser excitation (of wavelength 850 nanometers) at the p-n interface leads to striking six-fold photovoltage patterns as a function of bottom- and top-gate voltages. These patterns, together with the measured spatial and density dependence of the photoresponse, provide strong evidence that nonlocal hot carrier transport, rather than the photovoltaic effect, dominates the intrinsic photoresponse in graphene. This regime, which features a long-lived and spatially distributed hot carrier population, may offer a path to hot carrier-assisted thermoelectric technologies for efficient solar energy harvesting.