Epigenetic modifications to histones may promote either activation or repression of the transcription of nearby genes. Recent experimental studies show that the promoters of many lineage-control genes in stem cells have "bivalent domains" in which the nucleosomes contain both active (H3K4me3) and repressive (H3K27me3) marks. It is generally agreed that bivalent domains play an important role in stem cell differentiation, but the underlying mechanisms remain unclear. Here we formulate a mathematical model to investigate the dynamic properties of histone modification patterns. We then illustrate that our modeling framework can be used to capture key features of experimentally observed combinatorial chromatin states.