Although lesion and functional imaging studies have broadly implicated the right hemisphere in the recognition of emotion, neither the underlying processes nor the precise anatomical correlates are well understood. We addressed these two issues in a quantitative study of 108 subjects with focal brain lesions, using three different tasks that assessed the recognition and naming of six basic emotions from facial expressions. Lesions were analyzed as a function of task performance by coregistration in a common brain space, and statistical analyses of their joint volumetric density revealed specific regions in which damage was significantly associated with impairment. We show that recognizing emotions from visually presented facial expressions requires right somatosensory-related cortices. The findings are consistent with the idea that we recognize another individual's emotional state by internally generating somatosensory representations that simulate how the other individual would feel when displaying a certain facial expression. Follow-up experiments revealed that conceptual knowledge and knowledge of the name of the emotion draw on neuroanatomically separable systems. Right somatosensory-related cortices thus constitute an additional critical component that functions together with structures such as the amygdala and right visual cortices in retrieving socially relevant information from faces.