In nanoscale metal wires, electrical current can cause structural changes through electromigration, in which the momentum of electrons biases atomic motion, but the microscopic details are complex. Using in situ scanning tunneling microscopy, we examined the effects of thermally excited defects on the current-biased displacement of monatomic islands of radius 2 to 50 nanometers on single-crystal Ag(111). The islands move opposite to the current direction, with velocity varying inversely with radius. The force is thus in the same direction as electron flow and acts on atomic defect sites at the island edge. The unexpectedly large force on the boundary atoms can be decreased by over a factor of 10 by adding a mildly electron-withdrawing adsorbate, C60, which also modifies the step geometry. The low coordination of the identified scattering sites is the likely origin of the large force.