We have measured the beam-normal single-spin asymmetries in elastic scattering of transversely polarized electrons from the proton, and performed the first measurement in quasielastic scattering on the deuteron, at backward angles (lab scattering angle of 108°) for Q² = 0.22 GeV²/c² and 0.63 GeV²/c² at beam energies of 362 and 687 MeV, respectively. The asymmetry arises due to the imaginary part of the interference of the two-photon exchange amplitude with that of single-photon exchange. Results for the proton are consistent with a model calculation which includes inelastic intermediate hadronic (πN) states. An estimate of the beam-normal single-spin asymmetry for the scattering from the neutron is made using a quasistatic deuterium approximation, and is also in agreement with theory.