Differences in general cognitive function have been shown to be partly heritable and to show genetic correlations with several psychiatric and physical disease states. However, to date, few single-nucleotide polymorphisms (SNPs) have demonstrated genome-wide significance, hampering efforts aimed at determining which genetic variants are most important for cognitive function and which regions drive the genetic associations between cognitive function and disease states. Here, we combine multiple large genome-wide association study (GWAS) data sets, from the CHARGE cognitive consortium (n=53 949) and UK Biobank (n=36 035), to partition the genome into 52 functional annotations and an additional 10 annotations describing tissue-specific histone marks. Using stratified linkage disequilibrium score regression we show that, in two measures of cognitive function, SNPs associated with cognitive function cluster in regions of the genome that are under evolutionary negative selective pressure. These conserved regions contained ~2.6% of the SNPs from each GWAS but accounted for ~40% of the SNP-based heritability. The results suggest that the search for causal variants associated with cognitive function, and those variants that exert a pleiotropic effect between cognitive function and health, will be facilitated by examining these enriched regions.