Aging is associated with progressive brain atrophy and declines in learning and memory, often attributed to hippocampal or cortical deterioration. The role of brain-derived neurotrophic factor (BDNF) in modulating the structural and functional changes in the brain and visual system, particularly in relation to BDNF Val66Met polymorphism, remains underexplored. In this present cross-sectional observational study, we aimed to assess the effects of BDNF polymorphism on brain structural integrity, cognitive function, and visual pathway alterations. A total of 108 older individuals with no evidence of dementia and a mean (SD) age of 67.3 (9.1) years were recruited from the Optic Nerve Decline and Cognitive Change (ONDCC) study cohort. The BDNF Met allele carriage had a significant association with lower entorhinal cortex volume (6.7% lower compared to the Val/Val genotype, P = 0.02) and posterior cingulate volume (3.2% lower than the Val/Val group, P = 0.03), after adjusting for confounding factors including age, sex and estimated total intracranial volumes (eTIV). No significant associations were identified between the BDNF Val66Met genotype and other brain volumetric or diffusion measures, cognitive performances, or vision parameters except for temporal retinal nerve fibre layer thickness. Small but significant correlations were found between visual structural and functional, cognitive, and brain morphological metrics. Our findings suggest that carriage of BDNF Val66Met polymorphism is associated with lower entorhinal cortex and posterior cingulate volumes and may be involved in modulating the cortical morphology along the aging process.