The oxygen evolution reaction (OER) is essential for the development of renewable energy conversion and storage technologies. Eight N-doped graphenes containing variable numbers of embedded cobalt atoms (Coxy-NG, x = 1-4, y = 1-3, where x represents the number of embedded Co atoms and y represents different configurations) were designed and their OER electrocatalytic activities were systematically studied through density functional theory calculations. The significant roles of the number of Co atoms and their configuration in their OER performance were discussed in detail. Co31-NG occupies the peak of the activity volcano plot with a low overpotential of 0.31 V, which is smaller than Co11-NG with only one Co atom and even superior to the widely used IrO2 (0.56 V). The electronic structure and electron density analysis reveal that the outstanding electrocatalytic performance is due to the orbital hybridization between Co and N atoms and the increased positive charge on in-plane Co due to the out-of-plane Co atoms/clusters. This work clarifies the important role of transition atoms and provides excellent examples for reducing the overpotential through embedding several transition metal atoms onto single-atom electrocatalysts.