Four isostructural donor-acceptor alternating polymers of benzodithiophene (BDT)/naphthodifuran (NDF) and benzoselenadiazole (BSe)/benzothiadiazole (BT) have been developed and evaluated for organic photovoltaics. The substitution of one-atom (Se for S) in the accepting units exerts remarkable impact on the optoelectronic properties of polymers. Extended absorption, narrowed bandgap and higher HOMO energy levels were observed for Se-containing polymers in comparison to their S-containing counterparts. Theoretical calculations confirmed the measurable effect on energy levels as found in experimental studies. Bulk-heterojuction solar cells based on the BDT-BSe copolymer and [6,6]-phenyl-C71-butyric acid methyl ester (1 : 2, w/w) blend films deliver the best PCE of 5.40%. BSe-based polymers showed enhanced photovoltaic performances than BT-based polymers. The device performance is found to be strongly dependent on the processing conditions and morphology of the active layers.