The independent evolution of the sexes may often be constrained if male and female homologous traits share a similar genetic architecture. Thus, cross-sex genetic covariance is assumed to play a key role in the evolution of sexual dimorphism (SD) with consequent impacts on sexual selection, population dynamics, and speciation processes. We compiled cross-sex genetic correlations (r(MF)) estimates from 114 sources to assess the extent to which the evolution of SD is typically constrained and test several specific hypotheses. First, we tested if r(MF) differed among trait types and especially between fitness components and other traits. We also tested the theoretical prediction of a negative relationship between r(MF) and SD based on the expectation that increases in SD should be facilitated by sex-specific genetic variance. We show that r(MF) is usually large and positive but that it is typically smaller for fitness components. This demonstrates that the evolution of SD is typically genetically constrained and that sex-specific selection coefficients may often be opposite in sign due to sub-optimal levels of SD. Most importantly, we confirm that sex-specific genetic variance is an important contributor to the evolution of SD by validating the prediction of a negative correlation between r(MF) and SD.