Influential psychological models of autism spectrum disorder (ASD) have proposed that this prevalent developmental disorder results from impairment of global (integrative) information processing and overload of local (sensory) information. However, little neuroanatomical evidence consistent with this account has been reported. Here, we examined relative grey matter volumes (rGMVs) between three cortical networks, how they changed with age, and their relationship with core symptomatology. Using public neuroimaging data of high-functioning ASD males and age-/sex-/IQ-matched controls, we first identified age-associated atypical increases in rGMVs of the regions of two sensory systems (auditory and visual networks), and an age-related aberrant decrease in rGMV of a task-control system (fronto-parietal network, FPN) in ASD children. While the enlarged rGMV of the auditory network in ASD adults was associated with the severity of autistic socio-communicational core symptom, that of the visual network was instead correlated with the severity of restricted and repetitive behaviours in ASD. Notably, the atypically decreased rGMV of FPN predicted both of the two core symptoms. These findings suggest that disproportionate undergrowth of a task-control system (FPN) may be a common anatomical basis for the two ASD core symptoms, and relative overgrowth of the two different sensory systems selectively compounds the distinct symptoms.