Large biobanks have dramatically advanced our understanding of genetic influences on human brain anatomy. However, most studies have combined rather than compared males and females - despite theoretical grounds for potential sex differences. By systematically screening for sex differences in the common genetic architecture of > 1000 neuroanatomical phenotypes in the UK Biobank, we establish a general concordance between males and females in heritability estimates, genetic correlations and variant-level effects. Notable exceptions include: higher mean h 2 in females for regional volume and surface area phenotypes; between-sex genetic correlations that are significantly below 1 in the insula and parietal cortex; and, a male-specific effect common variant mapping to RBFOX1 - a gene linked to multiple male-biased neuropsychiatric disorders. This work suggests that common variant influences on human brain anatomy are largely consistent between males and females, with a few exceptions that will guide future research as biobanks continue to grow in size.