Epidemiological studies suggest that higher midlife serum total cholesterol levels are associated with an increased risk of Alzheimer's disease (AD). Using fluorodeoxyglucose positron emission tomography (PET) in the study of cognitively normal late middle-aged people, we demonstrated an association between apolipoprotein E (APOE) epsilon4 gene dose, the major genetic risk factor for late-onset AD, and lower measurements of the cerebral metabolic rate for glucose (CMRgl) in AD-affected brain regions, we proposed using PET as a pre-symptomatic endophenotype to evaluate other putative AD risk modifiers, and we then used it to support an aggregate cholesterol-related genetic risk score in the risk of AD. In the present study, we used PET to investigate the association between serum total cholesterol levels and cerebral metabolic rate for glucose metabolism (CMRgl) in 117 cognitively normal late middle-aged APOE epsilon4 homozygotes, heterozygotes and non-carriers. Higher serum total cholesterol levels were associated with lower CMRgl bilaterally in precuneus, parietotemporal and prefrontal regions previously found to be preferentially affected by AD, and in additional frontal regions previously found to be preferentially affected by normal aging. The associations were greater in APOE epsilon4 carriers than non-carriers in some of the AD-affected brain regions. We postulate that higher midlife serum total cholesterol levels accelerate brain processes associated with normal aging and conspire with other risk factors in the predisposition to AD. We propose using PET in proof-of-concept randomized controlled trials to rapidly evaluate the effects of midlife cholesterol-lowering treatments on the brain changes associated with normal aging and AD.