Mesenchymal stem cells (MSCs) hold great promise for therapeutic use in regenerative medicine and tissue engineering. A detailed understanding of the molecular processes governing MSC fate determination will be instrumental in the application of MSCs. Much progress has been made in recent years in defining the epigenetic events that control the differentiation of MSCs into different lineages. A complex network of transcription factors and histone modifiers, in concert with specific transcriptional co-activators and co-repressors, activates or represses MSC differentiation. In this review, we summarize recent progress in determining the effects of histone-modifying enzymes on the multilineage differentiation of MSCs. In addition, we propose that the manipulation of histone signatures associated with lineage-specific differentiation by small molecules has immense potential for the advancement of MSC-based regenerative medicine.