Previously reported leftward asymmetry in language-related gray and white matter areas of the brain has been proposed as a structural correlate of left-sided functional hemispheric language lateralization. However, structural asymmetry in non-left-sided functional language lateralization has as yet not been studied. Furthermore, the neuroanatomical basis of the reported volumetric white matter asymmetry is not fully understood. In 20 healthy volunteers, including 13 left-handers, we performed functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). We studied the relative fiber density (RFD) of the arcuate fasciculus (AF), using DT-tractography, in relation to functional hemispheric language lateralization. Hemispheric language lateralization was right-sided in five left-handed individuals. We demonstrated an overall significant leftward asymmetry in RFD of the AF, irrespective of handedness or functional language lateralization. Furthermore, in right-handers, the degree of structural asymmetry was found to be correlated with the degree of functional lateralization. We conclude that structural asymmetry in the AF does not seem to reflect functional hemispheric language lateralization, as has been proposed previously. Our findings suggest that the previously reported white matter asymmetry may be explained by a structural asymmetry in the arcuate fasciculus. These findings have important implications for the understanding of the functional and structural lateralization of brain regions as well as for the clinical evaluation of language function.