We report the first measurement of the average of the electron-proton and positron-proton elastic scattering cross sections. This lepton charge-averaged cross section is insensitive to the leading effects of hard two-photon exchange, giving more robust access to the proton's electromagnetic form factors. The cross section was extracted from data taken by the OLYMPUS experiment at DESY, in which alternating stored electron and positron beams were scattered from a windowless gaseous hydrogen target. Elastic scattering events were identified from the coincident detection of the scattered lepton and recoil proton in a large-acceptance toroidal spectrometer. The luminosity was determined from the rates of Møller, Bhabha, and elastic scattering in forward electromagnetic calorimeters. The data provide some selectivity between existing form factor global fits and will provide valuable constraints to future fits.