During the 1981 Voyager encounter, Titan's stratosphere exhibited a large thermal asymmetry, with high northern latitudes being colder than comparable southern latitudes. Given the short radiative time constant, this asymmetry would not be expected at the season of the Voyager observations (spring equinox), if the infrared and solar opacity sources were distributed symmetrically. We have investigated the radiative budget of Titan's stratosphere, using two selections of Voyager IRIS spectra recorded at symmetric northern and southern latitudes. In the region 0.1-1 mbar, temperatures are 7 K colder at 50 degrees N than at 53 degrees S and the difference reaches approximately 13 K at 5 mbar. On the other hand, the northern region is strongly enriched in nitriles and hydrocarbons, and the haze optical depth derived from the continuum emission between 8 and 15 micrometers is twice as large as in the south. Cooling rate profiles have been computed at the two locations, using the gas and haze abundances derived from the IRIS measurements. We find that, despite lower temperatures, the cooling rate profiles in the pressure range 0.15-5 mbar are 20 to 40% larger in the north than in the south, because of the enhanced concentrations of infrared radiators. Because the northern hemisphere appears darker than the southern one in the Voyager images, enhanced solar heating is also expected to take place at 50 degrees N. Solar heating rate profiles have been calculated, with two different assumptions on the origin of the hemispheric asymmetry. In the most likely case where it results from a variation in the absorbance of the haze material, the heating rates are found to be 12-15% larger at the northern location than at the southern one, a smaller increase than that in the cooling rates. If the lower albedo in the north results from an increase in the particle number density, a 55 to 75% difference is found for the pressure range 0.15-5 mbar, thus larger than that calculated for the cooling rates. Considering the uncertainties in the haze model, dynamical heat transport may significantly contribute to the meridional temperature gradients observed in the stratosphere. On the other hand, the latitudinal variation in gas and haze composition may be sufficient to explain the entire temperature asymmetry observed, without invoking a lag in the thermal response of the atmosphere due to dynamical inertia.