Chalcogenide perovskites ABX3 (A = Ca, Sr, or Ba; B = Ti, Zr, or Hf; and X = O, S, or Se) have been considered as promising candidates for overcoming the stability and toxic issues of halide perovskites. In this work, we unveil the disparity of the nature of the band gap between halide and chalcogenide perovskites. First-principles calculations show that the prototype cubic phase of chalcogenide perovskites exhibits indirect band gaps with the valence band maximum and the conduction band minimum located at R and Γ points, respectively, in the Brillion zone. Therefore, the optical transitions near band edges of chalcogenide perovskites differ from those of its halide counterparts, although its stable orthorhombic phase embodies a direct band gap. We have further found that the direct-indirect band gap difference of chalcogenide perovskites in the cubic phase demonstrates a linear correlation with t + μ, where t and μ are the tolerance and octahedral factor, respectively, thereby providing a viable way to search chalcogenide perovskites with a quasi-direct band gap.