Apolipoprotein E (ApoE) gene and primarily its allele e4 have been identified as a risk factor for Alzheimer's disease (AD). The prevalence of the gene in 25-30% in the population makes it essential to estimate its role in neuroregulation and its impact on distributed brain networks. In this study, we provide computational neuroanatomy based interpretation of large-scale and small-world cortical networks in cognitive normal (CN) subjects with differing Apolipoprotein-E4 (ApoE4) gene expression. We estimated large-scale anatomical networks from cortical thickness measurements derived from magnetic resonance imaging in 147 CN subjects explored in relation to ApoE4 genotype (e4+ carriers (n=41) versus e4- non-carriers (n=106)). Brain networks were constructed by thresholding cortical thickness correlation matrices of 68 bilateral regions of the brain analyzed using well-established graph theoretical approaches. Compared to ApoE4 non-carriers, carriers showed increased interregional correlation coefficients in regions like precentral, superior frontal and inferior temporal regions. Interestingly most of the altered connections were intra-hemispheric limited primarily to the right hemisphere. Furthermore, ApoE4 carriers demonstrated abnormal small-world architecture in the cortical networks with increased clustering coefficient and path lengths as compared to non-carrier, suggesting a less optimal topological organization. Additionally non-carriers demonstrated higher betweenness in regions such as middle temporal, para-hippocampal gyrus, posterior cingulate and insula of the default mode network (DMN), also seen in subjects with AD and mild cognitive impairment (MCI). The results suggest that the complex morphological cortical connectivity patterns are altered in ApoE4 carriers as compared to non-carriers, providing evidence for disruption of integrity in large-scale anatomical brain networks.