The neuropeptides oxytocin and vasopressin are evolutionarily conserved regulators of social perception and behavior. Evidence is building that they are critically involved in the development of social recognition skills within rodent species, primates, and humans. We investigated whether common polymorphisms in the genes encoding the oxytocin and vasopressin 1a receptors influence social memory for faces. Our sample comprised 198 families, from the United Kingdom and Finland, in whom a single child had been diagnosed with high-functioning autism. Previous research has shown that impaired social perception, characteristic of autism, extends to the first-degree relatives of autistic individuals, implying heritable risk. Assessments of face recognition memory, discrimination of facial emotions, and direction of gaze detection were standardized for age (7-60 y) and sex. A common SNP in the oxytocin receptor (rs237887) was strongly associated with recognition memory in combined probands, parents, and siblings after correction for multiple comparisons. Homozygotes for the ancestral A allele had impairments in the range -0.6 to -1.15 SD scores, irrespective of their diagnostic status. Our findings imply that a critical role for the oxytocin system in social recognition has been conserved across perceptual boundaries through evolution, from olfaction in rodents to visual memory in humans.