A functional brain-derived neurotrophic factor (BDNF) gene polymorphism (Val66Met) that alters activity-dependent secretion has previously been reported to influence cognitive functioning. A large proportion of these reports suggest that the Met allele, which results in reduced secretion of BDNF, impairs long-term memory as a direct consequence of its influence on hippocampal function but has little influence on working memory. In contrast, other studies have found that the Met allele can also play a protective role in certain neurological conditions and is associated with improved non-verbal reasoning skills in the elderly suggesting effects that appear disease, domain and age specific. We have investigated six haplotype-tagging single nucleotide polymorphisms (SNPs) using a cohort of 722 elderly individuals who have completed cognitive tests that measured the domains of fluid intelligence, processing speed and memory. We found that the presence of the Met allele reduced cognitive performance on all cognitive tests. This reached nominal significance for tests of processing speed (P = 0.001), delayed recall (P = 0.037) and general intelligence (g) (P = 0.008). No association was observed between cognitive tests and any other SNPs once the Val66Met was adjusted for. Our results support initial findings that the Met allele is associated with reduced cognitive functioning. We found no evidence that the Met allele plays a protective role in older non-demented individuals. Magnetic resonance imaging data collected from a subgroup of 61 volunteers showed that the left and right hippocampus were 5.0% and 3.9% smaller, respectively, in those possessing the Met allele, although only a non-significant trend was observed.