Although looking time is used to assess infant perceptual and cognitive processing, little is known about the temporal structure of infant looking. To shed light on this temporal structure, 127 three-month-olds were assessed in an infant-controlled habituation procedure and presented with a pre-recorded display of a woman addressing the infant using infant-directed speech. Previous individual look durations positively predicted subsequent look durations over a six look window, suggesting a temporal dependency between successive infant looks. The previous look duration continued to predict the subsequent look duration after accounting for habituation-linked declines in look duration, and when looks were separated by an inter-trial interval in which no stimulus was displayed. Individual differences in temporal dependency, the strength of associations between consecutive look durations, are distinct from individual differences in mean infant look duration. Nevertheless, infants with stronger temporal dependency had briefer mean look durations, a potential index of stimulus processing. Temporal dependency was evident not only between individual infant looks but between the durations of successive habituation trials (total looking within a trial). Finally, temporal dependency was evident in associations between the last look at the habituation stimulus and the first look at a novel test stimulus. Thus temporal dependency was evident across multiple timescales (individual looks and trials comprised of multiple individual looks) and persisted across conditions including brief periods of no stimulus presentation and changes from a familiar to novel stimulus. Associations between consecutive look durations over multiple timescales and stimuli suggest a temporal structure of infant attention that has been largely ignored in previous work on infant looking.