Onset timing of letter processing in auditory and visual sensory cortices

Abstract

Here, we report onset latencies for multisensory processing of letters in the primary auditory and visual sensory cortices. Healthy adults were presented with 300-ms visual and/or auditory letters (uppercase Roman alphabet and the corresponding auditory letter names in English). Magnetoencephalography (MEG) evoked response generators were extracted from the auditory and visual sensory cortices for both within-modality and cross-sensory activations; these locations were mainly consistent with functional magnetic resonance imaging (fMRI) results in the same subjects. In the primary auditory cortices (Heschl's gyri) activity to auditory stimuli commenced at 25 ms and to visual stimuli at 65 ms (median values). In the primary visual cortex (Calcarine fissure) the activations started at 48 ms to visual and at 62 ms to auditory stimuli. This timing pattern suggests that the origins of the cross-sensory activations may be in the primary sensory cortices of the opposite modality, with conduction delays (from one sensory cortex to another) of 17-37 ms. Audiovisual interactions for letters started at 125 ms in the auditory and at 133 ms in the visual cortex (60-71 ms after inputs from both modalities converged). Multivariate pattern analysis suggested similar latency differences between the sensory cortices. Combined with our earlier findings for simpler stimuli (noise bursts and checkerboards), these results suggest that primary sensory cortices participate in early cross-modal and interaction processes similarly for different stimulus materials, but previously learned audiovisual associations and stimulus complexity may delay the start of the audiovisual interaction stage.

Keywords: MEG; audiovisual interaction; cross-modal; language; multisensory.