Individuals with autism spectrum disorder (ASD) and their relatives process faces differently from typically developed (TD) individuals. In an fMRI face-viewing task, TD and undiagnosed sibling (SIB) children (5-18 years) showed face specialization in the right amygdala and ventromedial prefrontal cortex, with left fusiform and right amygdala face specialization increasing with age in TD subjects. SIBs showed extensive antero-medial temporal lobe activation for faces that was not present in any other group, suggesting a potential compensatory mechanism. In ASD, face specialization was minimal but increased with age in the right fusiform and decreased with age in the left amygdala, suggesting atypical development of a frontal-amygdala-fusiform system which is strongly linked to detecting salience and processing facial information.