Many nearshore fish and invertebrate populations are overexploited even when apparently coherent management structures are in place. One potential cause of mismanagement may be a poor understanding and accounting of stochasticity, particularly for stock recruitment. Many of the fishes and invertebrates that comprise nearshore fisheries are relatively sedentary as adults but have an obligate larval pelagic stage that is dispersed by ocean currents. Here, we demonstrate that larval connectivity is inherently an intermittent and heterogeneous process on annual time scales. This stochasticity arises from the advection of pelagic larvae by chaotic coastal circulations. This result departs from typical assumptions where larvae simply diffuse from one site to another or where complex connectivity patterns are created by transport within spatially complicated environments. We derive a statistical model for the expected variability in larval settlement patterns and demonstrate how larval connectivity varies as a function of different biological and physical processes. The stochastic nature of larval connectivity creates an unavoidable uncertainty in the assessment of fish recruitment and the resulting forecasts of sustainable yields.