Upconversion luminescence (UCL) refers to nonlinear optical processes, which can convert near-infrared photons to short-wavelength emission. Recent advances in nanotechnology have contributed to the development of photon upconversion materials as promising new generation candidates of fluorescent bioprobes and spectral converters for biomedical and optoelectronic applications. Apart from the remarkable photoluminescence of the materials under photon excitation, some UCL materials may exhibit intrinsic magnetic, ferroelectric, X-ray absorption properties, and so on. These interesting characteristics provide an opportunity for us to couple a single stimulus or multiple stimuli (electric field, magnetic field, X-ray, electron beam, temperature and pH, etc.) to various types of UCL materials. In this review, we will primarily focus on the stimuli responsive properties of UCL materials beyond light-matter interaction, which can aid both fundamental research and widespread applications of the materials. The mechanisms of the response to various stimuli in the UCL materials are discussed. This article will also highlight recent advances in the development of these materials in response to various stimuli and their applications in multimodal bioimaging, drug delivery and release, electro-optical devices, magnetic, temperature and pH sensors and multiple anti-counterfeiting inks. Lastly, we will present potential directions of future research and challenging issues which arise in expanding the applications of stimuli responsive UCL materials.