Perception is inherently subjective, and individual differences in phenomenology are well illustrated by the phenomenon of synesthesia (highly specific, consistent, and automatic cross-modal experiences, in which the external stimulus corresponding to the additional sensation is absent). It is unknown why some people develop synesthesia and others do not. In the current study, we tested whether neural markers related to having synesthesia in the family were evident in brain function and structure. Relatives of synesthetes (who did not have any type of synesthesia themselves) and matched controls read specially prepared books with colored letters for several weeks and were scanned before and after reading using magnetic resonance imaging. Effects of acquired letter-color associations were evident in brain activation. Training-related activation (while viewing black letters) in the right angular gyrus of the parietal lobe was directly related to the strength of the learned letter-color associations (behavioral Stroop effect). Within this obtained angular gyrus ROI, the familial trait of synesthesia related to brain activation differences while participants viewed both black and colored letters. Finally, we compared brain structure using voxel-based morphometry and diffusion tensor imaging to test for group differences and training effects. One cluster in the left superior parietal lobe had significantly more coherent white matter in the relatives compared with controls. No evidence for experience-dependent plasticity was obtained. For the first time, we present evidence suggesting that the (nonsynesthete) relatives of grapheme-color synesthetes show atypical grapheme processing as well as increased brain connectivity.