Malaria epidemics in regions with seasonal windows of transmission can vary greatly in size from year to year. A central question has been whether these interannual cycles are driven by climate, are instead generated by the intrinsic dynamics of the disease, or result from the resonance of these two mechanisms. This corresponds to the more general inverse problem of identifying the respective roles of external forcings vs. internal feedbacks from time series for nonlinear and noisy systems. We propose here a quantitative approach to formally compare rival hypotheses on climate vs. disease dynamics, or external forcings vs. internal feedbacks, that combines dynamical models with recently developed, computational inference methods. The interannual patterns of epidemic malaria are investigated here for desert regions of northwest India, with extensive epidemiological records for Plasmodium falciparum malaria for the past two decades. We formulate a dynamical model of malaria transmission that explicitly incorporates rainfall, and we rely on recent advances on parameter estimation for nonlinear and stochastic dynamical systems based on sequential Monte Carlo methods. Results show a significant effect of rainfall in the inter-annual variability of epidemic malaria that involves a threshold in the disease response. The model exhibits high prediction skill for yearly cases in the malaria transmission season following the monsoonal rains. Consideration of a more complex model with clinical immunity demonstrates the robustness of the findings and suggests a role of infected individuals that lack clinical symptoms as a reservoir for transmission. Our results indicate that the nonlinear dynamics of the disease itself play a role at the seasonal, but not the interannual, time scales. They illustrate the feasibility of forecasting malaria epidemics in desert and semi-arid regions of India based on climate variability. This approach should be applicable to malaria in other locations, to other infectious diseases, and to other nonlinear systems under forcing.