Timely detection of unusual and/or unexpected events in natural and man-made systems has deep scientific and practical relevance. We show that the recently proposed conceptually simple and easily calculated measure of permutation entropy can be effectively used to detect qualitative and quantitative dynamical changes. We illustrate our results on two model systems as well as on clinically characterized brain wave data from epileptic patients.