Mammals sense odors through the gene family of olfactory receptors (ORs). Despite the enormous number of OR genes (∼1,400 in mouse), each olfactory sensory neuron expresses one, and only one, of them. In neurobiology, it remains a long-standing mystery how this singularity can be achieved despite intrinsic stochasticity of gene expression. Recent experiments showed an epigenetic mechanism for maintaining singular OR expression: Once any ORs are activated, their expression inhibits further OR activation by down-regulating a histone demethylase Lsd1 (also known as Aof2 or Kdm1a), an enzyme required for the removal of the repressive histone marker H3K9me3 on OR genes. However, it remains unclear at a quantitative level how singularity can be initiated in the first place. In particular, does a simple activation/feedback scheme suffice to generate singularity? Here we show theoretically that rare events of histone demethylation can indeed produce robust singularity by separating two timescales: slow OR activation by stepwise H3K9me3 demethylation, and fast feedback to turn off Lsd1. Given a typical 1-h response of transcriptional feedback, to achieve the observed extent of singularity (only 2% of neurons express more than one ORs), we predict that OR activation must be as slow as 5–10 d-a timescale compatible with experiments. Our model further suggests H3K9me3-to-H3K9me2 demethylation as an additional rate-limiting step responsible for OR singularity. Our conclusions may be generally applicable to other systems where monoallelic expression is desired, and provide guidelines for the design of a synthetic system of singular expression.