The evolution of viruses to escape prevailing host immunity involves selection at multiple integrative scales, from within-host viral and immune kinetics to the host population level. In order to understand how viral immune escape occurs, we develop an analytical framework that links the dynamical nature of immunity and viral variation across these scales. Our epidemiological model incorporates within-host viral evolutionary dynamics for a virus that causes acute infections (e.g. influenza and norovirus) with changes in host immunity in response to genetic changes in the virus population. We use a deterministic description of the within-host replication dynamics of the virus, the pool of susceptible host cells and the host adaptive immune response. We find that viral immune escape is most effective at intermediate values of immune strength. At very low levels of immunity, selection is too weak to drive immune escape in recovered hosts, while very high levels of immunity impose such strong selection that viral subpopulations go extinct before acquiring enough genetic diversity to escape host immunity. This result echoes the predictions of simpler models, but our formulation allows us to dissect the combination of within-host and transmission-level processes that drive immune escape.