A recent experimental study reported the successful synthesis of an orthorhombic FeB4 with a high hardness of 62(5) GPa [H. Gou et al., Phys. Rev. Lett. 111, 157002 (2013)], which has reignited extensive interests on whether transition-metal borides compounds will become superhard materials. However, it is contradicted with some theoretical studies suggesting transition-metal boron compounds are unlikely to become superhard materials. Here, we examined structural and electronic properties of FeB4 using density functional theory. The electronic calculations show the good metallicity and covalent Fe-B bonding. Meanwhile, we extensively investigated stress-strain relations of FeB4 under various tensile and shear loading directions. The calculated weakest tensile and shear stresses are 40 GPa and 25 GPa, respectively. Further simulations (e.g., electron localization function and bond length along the weakest loading direction) on FeB4 show the weak Fe-B bonding is responsible for this low hardness. Moreover, these results are consistent with the value of Vickers hardness (11.7-32.3 GPa) by employing different empirical hardness models and below the superhardness threshold of 40 GPa. Our current results suggest FeB4 is a hard material and unlikely to become superhard (>40 GPa).