Sex differences have been observed in neurobehavioral measures and in neuroanatomic studies. Men and women differ in emotion processing, including perception, experience and expression, most notably reflected in greater male aggression. We examine temporo-limbic and prefrontal structures volumetrically in a large well-characterized sample of healthy adults, applying morphometric methods across cerebral regions that regulate emotions. Quantitative magnetic resonance imaging (MRI) was performed in 116 healthy adults, 57 men and 59 women, age range 18-49 years. We used reliable methods of region of interest identification to examine sex differences in volume of temporo-limbic and frontal regions. An automated tissue segmentation procedure was used to obtain separate measurements for gray and white matter. After correcting for cranial volume, men and women had identical volumes of amygdala and hippocampus, as well as dorsal prefrontal cortex. However, women had larger orbital frontal cortices than men, resulting in highly significant difference in the ratio of orbital gray to amygdala volume (P = 0.002). The larger volume of cortex devoted to emotional modulation may relate to behavioral evidence for sex differences in emotion processing.